Skip to content

Conversation

dqbd
Copy link
Collaborator

@dqbd dqbd commented Mar 23, 2023

Add an implementation of Prisma vectorstore, with type safety in mind.

1. Prisma vectorstore implements a mock interface, as it depends on the types generated via prisma generate
This does mean that the code might be unnecessary brittle compared to hardcoding @prisma/client directly in user codebases.

2. Added optional generic type argument for Document metadata
Developers can now specify a concrete type for metadata prop in Document. AFAIK this does not propagate in chains etc, but I believe this to be an useful starting block.

3. fromDocuments and fromTexts is currently unsupported
As there is a significant difficulty upserting entire documents, I've marked these methods as @deprecated for now.

Copy link
Collaborator

@hwchase17 hwchase17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh nice i like this a lot

an example would probably hel[

@dqbd dqbd force-pushed the prisma-vectorstore branch from 43cd9ed to b5306ad Compare March 25, 2023 14:33
@vercel
Copy link

vercel bot commented Mar 25, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
langchainjs-docs ⬜️ Ignored (Inspect) Visit Preview 💬 Add feedback Apr 8, 2023 0:30am

Copy link
Collaborator

@nfcampos nfcampos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments, looks good otherwise

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we'd want to remove the pageContent from metadata?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the moment, it returns the subset a Prisma model (TModel and TSelectModel), which do include the this.contentColumn).

We could remove the content column from the metadata, but I would argue that the metadata should contain the same columns, as user specifies in the dbConfig.

@MaximeThoonsen
Copy link

hello @dqbd, I was wondering if we could use typeorm to avoid adding another orm in the dependencies?

@dqbd
Copy link
Collaborator Author

dqbd commented Apr 8, 2023

Hi @MaximeThoonsen!

I was wondering if we could use typeorm to avoid adding another orm in the dependencies?

Both prisma and @prisma/client were added as a peer-dependency, it is not being actively installed when consuming langchain package. That being said, agreed, it can be removed entirely from the langchain project, as I'm stubbing the definitions.

Regardless, I think a TypeORM vectorstore implementation is certainly doable. Will look into it and possibly create a separate PR.

Copy link
Collaborator

@nfcampos nfcampos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great, thanks!

@nfcampos nfcampos merged commit 5ce8f2b into langchain-ai:main Apr 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants