Roger Kibbe's Unparsed 2024 Talk

Welcome to the repository for the "Unparsed 2024 Talk" by Roger Kibbe. This repository contains all the materials and resources for the presentation delivered at the Unparsed 2024 conference.

Table of Contents

• Introduction
• Presentation Slides
• Code Examples
• Installation
• Usage

Introduction

This repository hosts the content for Roger Kibbe's talk at Unparsed 2024. The presentation explores the differences between prompt engineering, RAG and fine-tuning and explores the when, what and how of fine-tuning LLM

Presentation Slides

The presentation slides can be found in the presentation directory.

• Presentation

Code Examples

Mistral 7B Fine Tune Juypter Notebook

Fine-Tuning Data

Simple Mistral CLI chatbot

Simple Mistral GUI Chatbot

Installation

To run the code examples locally, you will need to have Python installed. Follow the instructions below to set up your environment:

1. Clone the repository:

   git clone https://github.com/rogerkibbe/unparsed-2024-talk.git
   cd unparsed-2024-talk

2. Create a virtual environment and activate it:

   python3 -m venv venv
   source venv/bin/activate

3. Install the required dependencies:

   pip install -r requirements.txt

4. Get a Mistral API key and create a .env file with the key:

MISTRAL_API_KEY=[Mistral API Key]

Usage

The fine-tuning code is located in the mistral_finetune.ipynb Juypter notebook. Open it to run the fine-tune.

The fine-tuning data is IT support Q&A for a Bastard Operator from Hell type chatbot. The data was created by Anthropic's Claude and hand edited.

gui_client.py is a python panel app. To run:

panel serve [file name]

cli_compare.py is a simple command line tool to compare the outputs of the BOFH fine-tuned model and the base Mistral 7B Model. To run:

python cli_compare.py