feat: shortened text responses, basedOn selection and generate mode prompt tidy-up #124

stefl · 2024-09-12T12:40:21Z

Description

Adapts the prompt so that we more explicitly state that we want the user to select a lesson from the ones pulled from the database
Removes the relevant lessons section and prompt entirely if there are no lessons found
Attempts to skip the first "shall we start from scratch" (as per the Miro spec) but this doesn't seem to be consistent
Attempts to reduce the text response length to meet the Miro spec by providing examples in-line with each step
Numbers each step so that it's clearer what the happy path should look like
Fixes a few issues with the prompt which were mentioning out of date things
Restructures the prompt further so that interactive mode and generate mode are more separate - we were mixing up various user-interaction parts of the prompt with what would be used for non-interactive generation
Renames some files to be clearer
Breaks out the voice and language into a separate prompt file
Tries to ensure that the additionalMaterials section does not get overwritten when making edits

Issue(s)

Fixes the issue where Aila immediately picks a lesson plan to base upon.

Reduces text response length since it grew longer after merging SO

How to test

Try making a lesson base on an existing one. It should ask you before defaulting to one of the options.
Check that the text output in the lesson plan meets the spec in terms of text length.
Check that the text responses on the left hand side look okay. They are somewhat different, but shorter now.

vercel · 2024-09-12T12:40:25Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
oak-ai-lesson-assistant	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Sep 18, 2024 0:36am

stefl · 2024-09-12T12:41:21Z

packages/core/src/prompts/lesson-assistant/parts/currentLessonPlan.ts

@@ -0,0 +1,10 @@
+import { TemplateProps } from "..";


This is shared by both interactive and generate mode

stefl · 2024-09-12T12:41:59Z

packages/core/src/prompts/lesson-assistant/parts/interactingWithTheUser.ts

@@ -0,0 +1,120 @@
+export const interactingWithTheUser =


Move all the instructions for interaction into a single file with examples at each step.

More clearly mark out the steps the plan should go through with numbers, examples and explicit logic.

stefl · 2024-09-12T12:42:40Z

packages/core/src/prompts/lesson-assistant/parts/interactingWithTheUser.ts

+STEP 1: FIX AMERICANISMS, INCONSISTENCIES AND MISSING SECTIONS
+First, apply any corrections to the lesson plan by checking for Americanisms or inconsistencies between sections. You should do this automatically within each request, and not require user input.
+
+* ENSURE THAT THE LESSON PLAN IS GENERATED IN THE CORRECT ORDER


So that exitQuiz does not come before anything else for example.

stefl · 2024-09-12T12:42:52Z

packages/core/src/prompts/lesson-assistant/parts/interactingWithTheUser.ts

+* ENSURE THAT THE LESSON PLAN IS GENERATED IN THE CORRECT ORDER
+The sections of the lesson plan should be generated in this order: title, subject, topic, keyStage, basedOn (optional), learningOutcome, learningCycles, priorKnowledge, keyLearningPoints, misconceptions, keywords, starterQuiz, cycle1, cycle2, cycle3, exitQuiz, additionalMaterials.
+
+* ENSURE THAT THERE ARE NO MISSING PAST SECTIONS


Try to handle the case where we have missing sections in the plan.

stefl · 2024-09-12T12:43:02Z

packages/core/src/prompts/lesson-assistant/parts/interactingWithTheUser.ts

+For instance, if you have a lesson plan with all sections complete up until the exitQuiz, except for cycle1, this is indicative of an error and you should generate cycle1 again.
+Always trust the supplied lesson plan over the provided message history, because this represents the state of the lesson plan as it currently stands.
+
+* INCLUDING REFERENCES TO OTHER LESSON PLANS


I've left this in, but not sure we are using it?

stefl · 2024-09-12T12:43:15Z

packages/core/src/prompts/lesson-assistant/parts/interactingWithTheUser.ts

+If they are not present, ask the user for them.
+You can skip this step if the user has provided the title, key stage, subject and topic in the lesson plan.
+
+STEP 2 ASK THE USER IF THEY WANT TO ADAPT AN EXISTING LESSON IF THERE ARE RELEVANT LESSONS INCLUDED


This is the key step about adapting a lesson

github-actions · 2024-09-12T12:43:33Z

Playwright e2e tests

Job summary

Download report

To view traces locally, unzip the report and run:

npx playwright show-report ~/Downloads/playwright-report

stefl · 2024-09-12T12:43:47Z

packages/core/src/prompts/lesson-assistant/parts/interactingWithTheUser.ts

+
+STEP 6. additionalMaterials
+Ask the user if they would like to add any additional materials to the lesson plan. If they do, generate the additionalMaterials section. This is an open-ended section, so the user could ask you to generate anything that relates to the lesson plan.
+When generating this section ensure that if you are adding something new, ensure that you do not overwrite what already exists. You may need to respond with the existing content together with the new content so the user does not lose their work.


Try to get it not to send a patch which overwrites the additionalMaterials section. This should be an array.

stefl · 2024-09-12T12:43:58Z

packages/core/src/prompts/lesson-assistant/parts/interactingWithTheUser.ts

+STEP 7: final checks
+Ask the user if they want to edit anything, add anything to the additional materials. Offer to run a consistency check for language and content. Once complete, they can download their slides!
+
+SPECIAL RULE: ALLOW THE USER TO GENERATE AN ENTIRE LESSON PLAN WITHOUT ASKING QUESTIONS


Retain this feature - I am not sure it is working

stefl · 2024-09-12T12:44:11Z

packages/core/src/prompts/lesson-assistant/parts/interactingWithTheUser.ts

+
+The user may say something like "Generate the entire lesson plan without asking me any questions". In this case, you should proceed by generating all of the sections in the lesson plan, ignoring the instructions about doing it in steps and not checking if the user is happy after each step. This is to allow the user to quickly generate an entire lesson. Only once you have generated the whole lesson, ask the user if they are happy or would like to edit anything.
+
+SPECIAL RULE: BASING THE LESSON PLAN ON USER-PROVIDED CONTENT


Retain this feature - I'm not sure it works. Untested.

stefl · 2024-09-12T12:44:19Z

packages/core/src/prompts/lesson-assistant/parts/languageAndVoice.ts

@@ -0,0 +1,38 @@
+export const languageAndVoice = () => `LANGUAGE


Move language and voice to its own file

stefl · 2024-09-12T12:44:39Z

packages/core/src/prompts/lesson-assistant/parts/lessonComplete.ts

@@ -0,0 +1,17 @@
+export const lessonComplete = () => `ONCE THE LESSON IS COMPLETE


Move this to a separate file. Not edited.

stefl · 2024-09-16T09:40:12Z

packages/core/src/prompts/lesson-assistant/parts/interactingWithTheUser.ts

+Here is an example of how you should respond:
+
+START OF EXAMPLE HAPPINESS CHECK
+Are the learning outcome and learning cycles appropriate for your pupils? If not, suggest an edit below.


This and the other examples are from the Miro spec for how Aila should respond

mantagen

Looks good, though not clear what of the prompt is new vs moved

mantagen · 2024-09-18T11:30:48Z

packages/core/src/prompts/lesson-assistant/parts/currentLessonPlan.ts

+This represents the state of the application at the moment, as a JSON document.
+Use this as the basis for making any changes to the lesson plan.
+If you have content in past messages, the state represented here overrides that.
+${currentLessonPlan}`;


Was about to say should this be JSON.stringified but just looked and it's expected as a string

sonarcloud · 2024-09-18T12:34:59Z

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

oak-machine-user · 2024-09-30T11:58:41Z

🎉 This PR is included in version 1.8.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

…rompt tidy-up (#124)

feat: basedOn selection and prompt tidy-up

eccb05a

stefl commented Sep 12, 2024

View reviewed changes

vercel bot deployed to Preview September 12, 2024 12:41 View deployment

stefl commented Sep 12, 2024

View reviewed changes

Ensure sections are in a sensible order

25626da

vercel bot deployed to Preview September 12, 2024 12:54 View deployment

Shortened text responses

abb0efc

stefl changed the title ~~feat: basedOn selection and prompt tidy-up~~ feat: shortened text responses, basedOn selection and prompt tidy-up Sep 16, 2024

stefl changed the title ~~feat: shortened text responses, basedOn selection and prompt tidy-up~~ feat: shortened text responses, basedOn selection and generate mode prompt tidy-up Sep 16, 2024

stefl commented Sep 16, 2024

View reviewed changes

vercel bot deployed to Preview September 16, 2024 09:40 View deployment

stefl added 2 commits September 16, 2024 10:46

Remove language from the consistency check so it is in one place

5b94e18

Tap to continue example

e29e24c

vercel bot deployed to Preview September 16, 2024 09:54 View deployment

More explicity confirm that the user wants a base lesson

5a10a57

vercel bot deployed to Preview September 16, 2024 10:00 View deployment

Dynamically build the prompt based on having relevant lessons

ba9126c

vercel bot deployed to Preview September 16, 2024 10:49 View deployment

Bold

af2cea7

stefl requested a review from a team September 16, 2024 10:55

vercel bot deployed to Preview September 16, 2024 10:56 View deployment

Additional instruction not to repeat lesson content in the text response

79b885b

vercel bot deployed to Preview September 16, 2024 11:09 View deployment

mantagen approved these changes Sep 18, 2024

View reviewed changes

Merge changes to interactive/protocol prompt

1d5132d

vercel bot deployed to Preview September 18, 2024 11:58 View deployment

stefl added 2 commits September 18, 2024 13:05

Ensure that all cycles are generated in one go

889d7fe

Try to force basedOn not to be chosen by default

154d6af

vercel bot deployed to Preview September 18, 2024 12:10 View deployment

Do not do basedOn in two steps

82d3821

vercel bot deployed to Preview September 18, 2024 12:29 View deployment

reiterate basedon

7655147

vercel bot deployed to Preview September 18, 2024 12:36 View deployment

stefl merged commit 940c97a into main Sep 18, 2024
12 checks passed

stefl deleted the fix/based_on_selection branch September 18, 2024 13:46

oak-machine-user added the released label Sep 30, 2024

mantagen pushed a commit that referenced this pull request Oct 3, 2024

feat: shortened text responses, basedOn selection and generate mode p…

66a816b

…rompt tidy-up (#124)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: shortened text responses, basedOn selection and generate mode prompt tidy-up #124

feat: shortened text responses, basedOn selection and generate mode prompt tidy-up #124

stefl commented Sep 12, 2024 •

edited

Loading

vercel bot commented Sep 12, 2024 •

edited

Loading

stefl Sep 12, 2024

stefl Sep 12, 2024

stefl Sep 12, 2024

stefl Sep 12, 2024

stefl Sep 12, 2024

stefl Sep 12, 2024

github-actions bot commented Sep 12, 2024 •

edited

Loading

stefl Sep 12, 2024

stefl Sep 12, 2024

stefl Sep 12, 2024

stefl Sep 12, 2024

stefl Sep 12, 2024

stefl Sep 16, 2024

mantagen left a comment

mantagen Sep 18, 2024

sonarcloud bot commented Sep 18, 2024

oak-machine-user commented Sep 30, 2024


		The user may say something like "Generate the entire lesson plan without asking me any questions". In this case, you should proceed by generating all of the sections in the lesson plan, ignoring the instructions about doing it in steps and not checking if the user is happy after each step. This is to allow the user to quickly generate an entire lesson. Only once you have generated the whole lesson, ask the user if they are happy or would like to edit anything.

		SPECIAL RULE: BASING THE LESSON PLAN ON USER-PROVIDED CONTENT

		@@ -0,0 +1,38 @@
		export const languageAndVoice = () => `LANGUAGE

		@@ -0,0 +1,17 @@
		export const lessonComplete = () => `ONCE THE LESSON IS COMPLETE

feat: shortened text responses, basedOn selection and generate mode prompt tidy-up #124

feat: shortened text responses, basedOn selection and generate mode prompt tidy-up #124

Conversation

stefl commented Sep 12, 2024 • edited Loading

Description

Issue(s)

How to test

vercel bot commented Sep 12, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Sep 12, 2024 • edited Loading

Playwright e2e tests

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mantagen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sonarcloud bot commented Sep 18, 2024

Quality Gate passed

oak-machine-user commented Sep 30, 2024

stefl commented Sep 12, 2024 •

edited

Loading

vercel bot commented Sep 12, 2024 •

edited

Loading

github-actions bot commented Sep 12, 2024 •

edited

Loading