Issues running npm:chromadb inside nextjs
#953
Comments
|
@jeffchuber chromadb is not installed in the example repo you provided. Did you forget to push something? |
|
@perzeuss i did indeed! pushed |
|
The error you're encountering is stemming from Merge Request #929. This merge request introduced a feature designed to operate in a browser environment and to catch a failed dynamic import. Even if The current design of @jeffchuber to have chromadb running in next.js you could define const { ChromaClient } = require('chromadb');
export default function handler(req, res) {
const client = new ChromaClient();
const heartbeatFn = async () => {
return await client.heartbeat();
}
let heartbeat = heartbeatFn();
res.status(200).json({ heartbeat })
}This will work when the dynamic import ( |
|
I noticed that we're already employing a workaround for dynamic imports in the We could apply a similar solution to the To implement this workaround, we need to modify the asynchronous import |
|
I am trying to run this file in nextjs: /// app/api/chat/route.ts
import { Message, StreamingTextResponse } from 'ai'
import { ConversationalRetrievalQAChain } from 'langchain/chains'
import { ChatOpenAI } from 'langchain/chat_models/openai'
import { OpenAIEmbeddings } from 'langchain/embeddings/openai'
import {
AIMessage,
FunctionMessage,
HumanMessage,
SystemMessage,
} from 'langchain/schema'
import { Chroma } from 'langchain/vectorstores/chroma'
export const initVectorDB = async (collection: string) => {
const vectorStore = await Chroma.fromExistingCollection(
new OpenAIEmbeddings(),
{ collectionName: collection }
)
return vectorStore
}
export const convertMessagesToLangChain = (messages: Message[]) => {
const allMessages = [...messages]
const lastMessage = allMessages.splice(-1, 1)
const newMessages = []
for (const message of allMessages) {
if (message.role === 'user') {
newMessages.push(new HumanMessage(message.content))
} else if (message.role === 'assistant') {
newMessages.push(new AIMessage(message.content))
} else if (message.role === 'function') {
newMessages.push(new SystemMessage(message.content))
} else {
newMessages.push(new FunctionMessage(message.content, message.name))
}
}
return {
langChainMessages: newMessages,
question: lastMessage[0].content,
}
}
export const runtime = 'nodejs'
export async function POST(req: Request) {
const json = await req.json()
const messages: Message[] = json.messages
const vectorStore = await initVectorDB('zustand')
console.log(messages)
const model = new ChatOpenAI({
openAIApiKey: process.env.OPENAI_API_KEY as string,
modelName: 'gpt-3.5-turbo-16k-0613',
})
const chain = ConversationalRetrievalQAChain.fromLLM(
model,
vectorStore.asRetriever(),
{}
)
const { langChainMessages, question } = convertMessagesToLangChain(messages)
console.log(langChainMessages, question, 'question')
const stream = await chain.stream({
chat_history: langChainMessages,
question,
})
return new StreamingTextResponse(stream)
}And I keep getting: which suggests that its thinking chroma is running in the browser, however AFAIK this is only running on the server (am new to the app directory) |
|
@BjoernRave the issue is that the build tools try to locate the module regardless of the fact that the import only takes effect in browser environments. The chromadb package does not provide an own client for node and browser environments. That means the code for the browser environment is also loaded when you use chromadb on nextjs server routes / node environment. The scripts in the chromadb package just detect that you are runing in a node environment and only execute code for the node environment. Same for browser environments. |
|
@jeffchuber I just noticed that there is actually an ESM build, but it is not generated. I overlooked that because I just wanted to quickly fix the error here 😅 This error occurs only in my setup because I have installed chroma as git submodule and yarn has issues with a node_module in the parent and in the js client folder. |
|
when I tried to run it in pure node with So is this sth that needs to be fixed on the chroma side? Or langchain, or nextjs? (So many parts :D) |
You are using the CJS build, At the moment you have to import chroma from And then to make it work you'd have to install |
|
Now I get this error: There is a PR which should fix this issue though, right? So I guess I will have to wait for that to land |
|
Any updates? |
resolves #953 ## Description of changes - Bug fixes - implement a workaround in clients/js/src/embeddings/WebAIEmbeddingFunction.ts to resolve #953 ## Test plan At present, we lack a testing setup specifically tailored for a browser environment. Implementing this would be a necessary step to implement tests for changes in this branch. I've tested it locally using this change: https://github.com/jeffchuber/nextjs-chroma/pull/1/files - http://localhost:3000/api/hello (node version) works fine - http://localhost:3000 (browser version) works fine
resolves chroma-core#953 ## Description of changes - Bug fixes - implement a workaround in clients/js/src/embeddings/WebAIEmbeddingFunction.ts to resolve chroma-core#953 ## Test plan At present, we lack a testing setup specifically tailored for a browser environment. Implementing this would be a necessary step to implement tests for changes in this branch. I've tested it locally using this change: https://github.com/jeffchuber/nextjs-chroma/pull/1/files - http://localhost:3000/api/hello (node version) works fine - http://localhost:3000 (browser version) works fine
resolves chroma-core#953 ## Description of changes - Bug fixes - implement a workaround in clients/js/src/embeddings/WebAIEmbeddingFunction.ts to resolve chroma-core#953 ## Test plan At present, we lack a testing setup specifically tailored for a browser environment. Implementing this would be a necessary step to implement tests for changes in this branch. I've tested it locally using this change: https://github.com/jeffchuber/nextjs-chroma/pull/1/files - http://localhost:3000/api/hello (node version) works fine - http://localhost:3000 (browser version) works fine
resolves chroma-core#953 ## Description of changes - Bug fixes - implement a workaround in clients/js/src/embeddings/WebAIEmbeddingFunction.ts to resolve chroma-core#953 ## Test plan At present, we lack a testing setup specifically tailored for a browser environment. Implementing this would be a necessary step to implement tests for changes in this branch. I've tested it locally using this change: https://github.com/jeffchuber/nextjs-chroma/pull/1/files - http://localhost:3000/api/hello (node version) works fine - http://localhost:3000 (browser version) works fine
|
after installing npm i @visheratin/web-ai onnxruntime-web @visheratin/tokenizers jimp. I still got more errors. |
|
@helxsz the warnings can be ignored. Edit 1: Are you sure the webpack errors are caused by installing chromadb? Edit 2: And do you still have the issue when you uninstall webai related dependencies and using chromadb without an embedding function? |
|
I just realize the webai embedding plugin is initialized in node mode. Edit 1: |
What's wrong
Chroma's
npmpackagechromadbis currently incompatible with nextjs builds.Here is a repo you can very minimal clone and run - https://github.com/jeffchuber/nextjs-chroma
Relevant logs
Inclusion breaks nextjs.
If you remove this file, we also logspam a bunch of crap into
nextjsThe text was updated successfully, but these errors were encountered: