Build Enterprise-grade Generative AI applications with microservice architecture
This initiative empowers the development of high-quality Generative AI applications for enterprises via microservices, simplifying the scaling and deployment process for production. It abstracts away infrastructure complexities, facilitating the seamless development and deployment of Enterprise AI services.
GenAIComps provides a suite of microservices, leveraging a service composer to assemble a mega-service tailored for real-world Enterprise AI applications. All the microservices are containerized, allowing cloud native deployment. Checkout how the microservices are used in GenAIExamples.
The initially supported microservices are described in the below table. More microservices are on the way.
| MicroService | Framework | Model | Serving | HW | Description |
| Embedding | LangChain | BAAI/bge-large-en-v1.5 | TEI-Habana | Gaudi2 | Embedding on Gaudi2 |
| Embedding | LangChain | BAAI/bge-base-en-v1.5 | TEI | Xeon | Embedding on Xeon CPU |
| Retriever | LangChain | BAAI/bge-base-en-v1.5 | TEI | Xeon | Retriever on Xeon CPU |
| Reranking | LangChain | BAAI/bge-reranker-large | TEI-Habana | Gaudi2 | Reranking on Gaudi2 |
| Reranking | LangChain | BBAAI/bge-reranker-base | TEI | Xeon | Reranking on Xeon CPU |
| LLM | LangChain | Intel/neural-chat-7b-v3-3 | TGI Gaudi | Gaudi2 | LLM on Gaudi2 |
| LLM | LangChain | Intel/neural-chat-7b-v3-3 | TGI | Xeon | LLM on Xeon CPU |