Nanogpt implementation

[thistleknot]

Is it possible to provide an implementation of this using nanogpt and or litgpt?

[gaotianyu1350]

Hi,

Thanks for your interest in MeZO! If they are in the transformers package, then it should be fairly easy to use -- just load these models and for prefix-tuning/lora, change the layer names accordingly.

[thistleknot]

It's not. It's a complete rewrite

[gaotianyu1350]

Then you can refer to the read me in the large_model folder. We highlighted the core part of the MeZO code which you can copy to your own codebase.