Merge pull request #13 from tensorplex-labs/feat/expand-readme

feat: updated readme with technical summary

README.md

@@ -1,12 +1,28 @@
 # dojo-synthetic-api
 
-Before running the synthetic data api, you will need the following keys:
+This repository generates question & answer pairs (QAP), which are retrieved and assessed by [Dojo Subnet](https://github.com/tensorplex-labs/dojo).
+
+In terms of implementation, an asynchronous task scheduler is used to schedule QAP generation. Redis is used to store generated pairs. Dojo retrieve QAPs by querying the `/synthetic-gen` endpoint hosted by this repo. Successful retrieval will delete that QAP from redis. When the number of stored pairs falls under a given threshold, the task scheduler will generate new QAPs to replenish the database, ensuring that there is always a healthy supply of QAPs for use by Dojo.
+
+QAPs are generated by prompting and querying LLMs. This process is underimprovement and is subject to change in the future.
+
+1. Query an LLM for a list of common objects used in animation.
+2. Prompt an LLM with these objects to create a coding question.
+3. Prompt an LLM to answer the generated question. The generated question and output code make up a QAP.
+
+Currently, synthetic-api only generates code output as Javascript. We are in the process of expanding support for other programming languages.
+
+## Setup
+
+Before running the synthetic-api, you will need the following keys:
 
 - Openrouter (from https://openrouter.ai/)
 - E2B (from https://e2b.dev/)
 
 Copy the .env.example file to a .env file and fill in the blanks, here we will use Openrouter as our LLM API provider.
 
+Docker will create a redis instance using the specified `REDIS_USERNAME` and `REDIS_PASSWORD`. You will need these to manually interact with your redis on docker.
+
 ```bash
 cp .env.example .env
 
@@ -19,9 +35,9 @@ E2B_API_KEY=
 
 ## Run with docker-compose
 
-Install docker and docker-compose
+Install docker and docker-compose:
 
-```
+```bash
 sudo apt-get install \
     ca-certificates \
     curl \
@@ -36,13 +52,9 @@ sudo apt-get update
 sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
 ```
 
-Fill in the environment variables in `docker-compose.yml` and run `docker compose up`
-
-```
-# Go into the editor for docker-compose.yml and edit the environment variables
-# CHANGE THOSE LINES THAT ARE MARKED WITH # CHANGE
-vim docker-compose.yml
+Run docker to launch synthetic-api:
 
+```bash
 # Run the service
 docker compose up -d
 ```