Knowledge #1567
Knowledge #1567
Conversation
| from crewai.knowledge.source.base_knowledge_source import BaseKnowledgeSource | ||
|
|
||
|
|
||
| class Knowledge(BaseModel): |
There was a problem hiding this comment.
Lets write some docs about:
- how to use this
- setting your own custom embedder for this
| @@ -0,0 +1,82 @@ | |||
| import os | |||
There was a problem hiding this comment.
i'd drop the ollama version. support openai, then let anyone bring their own embedder function (super easy) then have the knowledge_config setup like embedder_config setup for our rag storage
| @abstractmethod | ||
| def add(self, embedder: BaseEmbedder) -> None: | ||
| """Process content, chunk it, compute embeddings, and save them.""" | ||
| pass | ||
|
|
||
| def get_embeddings(self) -> List[np.ndarray]: | ||
| """Return the list of embeddings for the chunks.""" |
There was a problem hiding this comment.
lets make this save to the project directory instead of the root.
| # Compute embeddings for the new chunks | ||
| new_embeddings = embedder.embed_chunks(new_chunks) | ||
| # Save the embeddings | ||
| self.chunk_embeddings.extend(new_embeddings) |
There was a problem hiding this comment.
We should also be saving this to a db and persist it. like our ragstorage
There was a problem hiding this comment.
We should do this as we can generate the embeddings for files once, then just query if they already exist. otherwise, users will spend tokens generating embeddings that already exist.
| from pydantic import BaseModel, ConfigDict, Field | ||
|
|
||
| from crewai.knowledge.embedder.base_embedder import BaseEmbedder | ||
|
|
There was a problem hiding this comment.
extending this to save embeddings to db, then using knowledge class to query from there
src/crewai/agent.py
Outdated
| @@ -85,6 +88,10 @@ class Agent(BaseAgent): | |||
| llm: Union[str, InstanceOf[LLM], Any] = Field( | |||
| description="Language model that will run the agent.", default=None | |||
| ) | |||
| knowledge_sources: Optional[List[BaseKnowledgeSource]] = Field( | |||
| default=None, | |||
| description="Knowledge sources for the agent.", | |||
There was a problem hiding this comment.
Would be better to declare this on the crew class. the task prompt will query from the relevant trickling down to the agent level, then defining here on the agent level
No description provided.