Update special token handling in conversion scripts for gpt2 derived tokenizers#3746

Merged

goerch merged 2 commits intoggerganov:masterfrom Galunid:update-conversion-scripts

Oct 23, 2023

+56-19

Commits on Oct 23, 2023