Implement top_p / nucleus sampling

[jbochi]

This is an implementation of nucleus sampling, introduced in the paper The Curious Case of Neural Text Degeneration.

It's supported by llama-cpp, llama2.c, open ai and others.

[LaurentMazare]

Looks great, thanks! Could you describe a bit the testing that you've done? Just want to be sure that it covers running all the modified CLIs as well as the different web UIs.
Also could you add an entry to the top level CHANGELOG.md file?

[jbochi]

Looks great, thanks!

Thank you! And thank you for creating candle!

Could you describe a bit the testing that you've done? Just want to be sure that it covers running all the modified CLIs as well as the different web UIs.

Here's trunk serve:

I also tested that even with high temperature, the output is close to greedy decoding (temperature=0) if top_p is low:

And here's lib-example.html:

Also could you add an entry to the top level CHANGELOG.md file?

Done

[LaurentMazare]

Great thanks, will merge once the CI has run!

[LaurentMazare]

rustfmt seems to fail, could you have a look please?

[jbochi]

rustfmt seems to fail, could you have a look please?

sorry about that! Should be fixed now.

I also added the new test file I forgot to commit before

[LaurentMazare]

Thank for apply rustfmt, clippy seems to be having some failures now :)

[jbochi]

Sorry again. Hopefully, it's all fixed now 🙏

[LaurentMazare]

Thanks, merged!