chat_template parameter

[prasiyer]

Can you please explain when to use each of the following chat_template parameters?
tokenizer, chatml, zephyr, None

I am working to prepare the dataset for finetuning Llama-3-8B-Instruct

[splanker]

Yes, this would be helpful. Autotrain seems appealing but the lack of documentation, lack of how to videos, lack of basic guides blows my mind.

[abhishekkrthakur]

does this help: https://huggingface.co/blog/abhishek/phi3-finetune-macbook?

[splanker]

Yeah, not really. Same old same old. Even at the bottom where it says "extensive" documentation can be found here (link to huggingface documentation) it really is just a couple of pages explaining what AuoTrain is and how much it costs. I wouldn't exactly call that "extensive".

A colab dataset validator with some code to detect and possibly offer solutions for correcting the issue would be fantastic. I've looked, but can't find anything.

So, as it stands the process is:

• make adjustments to the dataset
• upload
• run
• fail
• search for explanation of error
• find none
• repeat
  It's kind of a time consuming approach for something that has the word "auto" in it.

[abhishekkrthakur]

@splanker i read your rant :D thanks! that means more things need to be improved.
may i ask you to post a few lines from the dataset that is giving you the error? so i know a bit more

[prasiyer]

@abhishekkrthakur -- Appreciate your efforts to create open source tools. Autotrain makes fine-tuning of llms, a bit less intimidating. I have 2 questions
Question 1: Is it possible to get information similar to the table below?

chat-template parameter value | Example dataset | Training method | Applicable open llms
None |openassistant-guanaco, alpaca | SFT |Phi-3-mini-4k-instruct, Meta-Llama-3-8B-Instruct
Chatml |?? |?? |??
Zephyr |?? |?? |??

Question 2: When chat-template = "tokenizer", is the functionality similar to using tokenizer.apply_chat_template(..,..,tokenize = False) to convert the prompt to a appropriate format for the llm?

Thanks in advance.

[prasiyer]

@abhishekkrthakur - Can you please provide feedback to the above question?

[Stealthwriter]

use tokenizer always and make sure the dataset has a proper format for chat template