Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance conv prompt and train #2388

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Trangle
Copy link
Contributor

@Trangle Trangle commented Sep 8, 2023

Why are these changes needed?

  1. The process of constructing prompts in the conversation has been separated, allowing for more general circumstances such as when the GPT speaks first or when the human speaks multiple times.
  2. A prompt generator and manager have been constructed, with each generator handling its own training token_ids. This makes definitions more clear and concatenation more universal.
  3. An additional 'train_general' has been introduced, providing support for more general SFT models.

I've already validated the code logic several times and haven't found any potential hidden issues, please help me test it further.

Related issue number (if applicable)

Checks

  • I've run format.sh to lint the changes in this PR.
  • I've included any doc changes needed.
  • I've made sure the relevant tests are passing (if applicable).

@Trangle Trangle force-pushed the enhance_conv_prompt_and_train branch from 718d8bc to e4460fc Compare December 9, 2023 05:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant