outlines: Guided Text Generation

[irthomasthomas]

• outlines-dev/outlines: Guided Text Generation

  Made with ❤👷️ by the team at .txt. We'd love to have your feedback!

pip install outlines
First time here? Go to our setup guide

Features

🤖 Multiple model integrations: OpenAI, transformers, AutoGPTQ, AutoAWQ
🖍️ Simple and powerful prompting primitives based on the Jinja templating engine
🚄 Multiple choices, type constraints and dynamic stopping
⚡ Fast regex-guided generation
🔥 Fast JSON generation following a JSON schema or a Pydantic model
📝 Grammar-guided generation
🐍 Interleave completions with loops, conditionals, and custom Python functions
💾 Caching of generations
🗂️ Batch inference
🚀 Serve with vLLM
Outlines 〰 has new releases and features coming every week. Make sure to ⭐ star and 👀 watch this repository, follow @dottxtai to stay up to date!

Guided generation

The first step towards reliability of systems that include large language models is to ensure that there is a well-defined interface between their output and user-defined code. Outlines provides ways to control the generation of language models to make their output more predictable.

Multiple choices

You can reduce the completion to a choice between multiple possibilities:

import outlines

model = outlines.models.transformers("mistralai/Mistral-7B-v0.1")

prompt = """You are a sentiment-labelling assistant.
Is the following review positive or negative?

Review: This restaurant is just awesome!
"""
answer = outlines.generate.choice(model, ["Positive", "Negative"])(prompt)
Type constraint