test: record and replay streaming LLM calls in e2e tests

[codeincontext]

Description

This PR supersedes #130, as that PR didn't mock the moderation OpenAI calls

• Playwright intercepts API calls to /chat and adds x-fixture-name and x-fixture-mode headers
• Chat API handler looks for those headers and passes custom LLMService and modetaions OpenAi clients to Aila
  • The FixtureRecord LLMService proxies the OpenAI service and saves the chunks to a file. Moderation has an equivalent
  • The FixtureReplay LLMService streams the chunks in a fixture file. Moderation has an equivalent

Known issues:

• There's (what I think is) a race condition which requires some explicit waits, otherwise the number of cpleted sections regresses between messages
  • "These waits are an antipattern. If we don't allow enough time before sending the next message, the completed section count goes backwards. I think it's a race condition with the useEffects in the chat and fetching state from tRPC after streaming. It was happening before and after feat: allow the user to interact while moderation is in progress #147"
  • I think we should just move forward and address that at a later date if it becomes important

How to test

1. Check out the branch locally
2. Pull doppler, or set AILA_FIXTURES_ENABLED=true in .env
3. Run pnpm run test-e2e-ui
4. Click through to aila-chat => full-romans.test.ts => Authenticated => "Full aila flow with Romans fixture"
5. Run the test. If you make sure it's selected in the sidebar you should see the output on the right. Note that the auth setup will run first
6. You should see the same content as the fixtures. Look at the .formatted.txt files to confirm

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
oak-ai-lesson-assistant	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Sep 23, 2024 4:46pm

[github-actions]

Playwright e2e tests

Job summary

Download report

To view traces locally, unzip the report and run:

npx playwright show-report ~/Downloads/playwright-report

[codeincontext]

These waits are an antipattern. If we don't allow enough time before sending the next message, the completed section count goes backwards. I think it's a race condition with the useEffects in the chat and fetching state from tRPC after streaming. It was happening before and after #147

[codeincontext]

Don't worry about this too much, as I'm putting together a new PR to split it out into a separate test

[sonarcloud]

Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[mantagen]

Brilliant!