Skip to content
This repository was archived by the owner on Sep 9, 2025. It is now read-only.

Conversation

@jwm4
Copy link
Contributor

@jwm4 jwm4 commented Dec 13, 2024

#163 failed due to some github / DCO problem, so we're trying again.

@jwm4 jwm4 changed the title Second attempt to submit this ADR because the last one failed Second attempt to submit RAG repo location ADR because the last one failed Dec 13, 2024
Copy link
Contributor

@cdoern cdoern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with the conclusion to put this in instructlab/instructlab for the time being, left a few other comments.


## Decision

- For now, RAG will be located in the core repository in its own directory: `src/instructlab/rag` in the core InstructLab repository (<https://github.com/instructlab/instructlab>).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense to me


### Future phases

- In the near future, RAG might be moved to the existing <https://github.com/instructlab/rag> repository.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to (maybe in a separate doc or a future PR to this doc) to figure out why we'd move it out. If we are just moving things to separate into units, I think you can have a clearly defined unit inside of the core repo.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the purposes of this document, I think the pros and cons of listed for alternatives is probably sufficient. I don't think we need a stronger argument for a thing we might do in the future since we're explicitly committing to open a new ADR if and when we do.


## Risks

- Putting the RAG functionality in the core repository requires any application that wants to use this functionality to bring in the entire core which then brings in all of the libraries it depend on, so this becomes an enormous dependency. This discourages reuse in other applications. It *encourages* either of the following behaviors that would be unfortunate:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

realistically though, no one is using RAG for ilab without ilab

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I disagree, but let's try to resolve that disagreement in another venue.

@nathan-weinberg nathan-weinberg merged commit d6f77b1 into instructlab:main Dec 16, 2024
4 checks passed
@mairin
Copy link
Member

mairin commented Dec 19, 2024

Discussed this in the oversight committee and this is approved.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants