MicroCore is a collection of python adapters for Large Language Models and Semantic Search APIs allowing to communicate with these services in a convenient way, make them easily switchable and separate business logic from the implementation details.
It defines interfaces for features typically used in AI applications, which allows you to keep your application as simple as possible and try various models & services without need to change your application code.
You even can switch between text completion and chat completion models only using configuration.
The basic example of usage is as follows:
from microcore import llm
while user_msg := input('Enter message: '):
print('AI: ' + llm(user_msg))Install as PyPi package:
pip install ai-microcore
Alternatively, you may just copy microcore folder to your project sources root.
git clone git@github.com:Nayjest/ai-microcore.git && mv ai-microcore/microcore ./ && rm -rf ai-microcorePython 3.10 / 3.11 / 3.12
Both v0.28+ and v1.X OpenAI package versions are supported.
Having OPENAI_API_KEY in OS environment variables is enough for basic usage.
Similarity search features will work out of the box if you have the chromadb pip package installed.
There are a few options available for configuring microcore:
- Use
microcore.configure(**params)
💡 All configuration options should be available in IDE autocompletion tooltips - Create a
.envfile in your project root; examples: basic.env, Mistral Large.env, Anthropic Claude 3 Opus.env, Gemini on Vertex AI.env, Gemini on AI Studio.env - Use a custom configuration file:
mc.configure(DOT_ENV_FILE='dev-config.ini') - Define OS environment variables
For the full list of available configuration options, you may also check microcore/config.py.
For the models working not via OpenAI API, you may need to install additional packages:
pip install anthropicpip install google-generativeaipip install vertexai📌Additonaly for working through Vertex AI you need to install the Google Cloud CLI and configure the authorization.
You will need to install transformers and deep learning library of your choice (PyTorch, TensorFlow, Flax, etc).
See transformers installation.
- Configuration options passed as arguments to
microcore.configure()have the highest priority. - The priority of configuration file options (
.envby default or the value ofDOT_ENV_FILE) is higher than OS environment variables.
💡 SettingUSE_DOT_ENVtofalsedisables reading configuration files. - OS environment variables have the lowest priority.
Performs a request to a large language model (LLM).
Asynchronous variant: allm(prompt: str, **kwargs)
from microcore import *
# Will print all requests and responses to console
use_logging()
# Basic usage
ai_response = llm('What is your model name?')
# You also may pass a list of strings as prompt
# - For chat completion models elements are treated as separate messages
# - For completion LLMs elements are treated as text lines
llm(['1+2', '='])
llm('1+2=', model='gpt-4')
# To specify a message role, you can use dictionary or classes
llm(dict(role='system', content='1+2='))
# equivalent
llm(SysMsg('1+2='))
# The returned value is a string
assert '7' == llm([
SysMsg('You are a calculator'),
UserMsg('1+2='),
AssistantMsg('3'),
UserMsg('3+4=')]
).strip()
# But it contains all fields of the LLM response in additional attributes
for i in llm('1+2=?', n=3, temperature=2).choices:
print('RESPONSE:', i.message.content)
# To use response streaming you may specify the callback function:
llm('Hi there', callback=lambda x: print(x, end=''))
# Or multiple callbacks:
output = []
llm('Hi there', callbacks=[
lambda x: print(x, end=''),
lambda x: output.append(x),
])Renders prompt template with params.
Full-featured Jinja2 templates are used by default.
Related configuration options:
from microcore import configure
configure(
# 'tpl' folder in current working directory by default
PROMPT_TEMPLATES_PATH = 'my_templates_folder'
)texts.search(collection: str, query: str | list, n_results: int = 5, where: dict = None, **kwargs) → list[str]
Similarity search
Find most similar text
Return collection of texts
Store text and related metadata in embeddings database
Store multiple texts and related metadata in the embeddings database
Clear collection
LLM Microcore supports all models & API providers having OpenAI API.
| API Provider | Models |
|---|---|
| OpenAI | All GPT-4 and GTP-3.5-Turbo models all text completion models (davinci, gpt-3.5-turbo-instruct, etc) |
| Microsoft Azure | All OpenAI models, Mistral Large |
| Anthropic | Claude 3 models |
| MistralAI | All Mistral models |
| Google AI Studio | Google Gemini models |
| Google Vertex AI | Gemini Pro & other models |
| Deep Infra | deepinfra/airoboros-70b jondurbin/airoboros-l2-70b-gpt4-1.4.1 meta-llama/Llama-2-70b-chat-hf and other models having OpenAI API |
| Anyscale | meta-llama/Llama-2-70b-chat-hf meta-llama/Llama-2-13b-chat-hf meta-llama/Llama-7b-chat-hf |
| Groq | LLaMA2 70b Mixtral 8x7b Gemma 7b |
| Fireworks | Over 50 open-source language models |
- HuggingFace Transformers (see configuration examples here).
- Custom local models by providing own function for chat / text completion, sync / async inference.
Performs code review by LLM for changes in git .patch files in any programming languages.
Image analysis (Google Colab)
Determine the number of petals and the color of the flower from a photo (gpt-4-turbo)
Banchmark LLMs on math problems (Kaggle Notebook)
Benchmark accuracy of 20+ state of the art models on solving olympiad math problems. Inferencing local language models via HuggingFace Transformers, parallel inference.
@TODO
This is experimental feature.
Tweaks the Python import system to provide automatic setup of MicroCore environment based on metadata in module docstrings.
import microcore.ai_modules- Automatically registers template folders of AI modules in Jinja2 environment
Please see CONTRIBUTING for details.
Licensed under the MIT License © 2023 Vitalii Stepanenko