Speed up sample_topp

[dror-cyata]

This PR should allow using topp logic while avoiding the significant performance hit

Sampling probabilities tend to be sharp, so filtering out low probabilities before sorting the list result with significant time improvement for sample_topp.
We use a threshold that guaranties the filtered entries do not belong to topp (except for maybe a very rare edge case, need to think on it), but anyway we verify it to allow higher threshold.

[kroggen]

I suspect that #270 is faster

[dror-cyata]

I suspect that #270 is faster

Just looked on #270. Both indeed look similar and filtering out entries with low probabilities, so they should take similar time (threshold of 1e-5 is used there, and ~3e-6 is used here, by default). However, the threshold used there is arbitrary, and here we used a threshold that assures that the filtered out entries do not belong to topp. Note that in #270 there might be edge cases where entries that are part of topp are unintentionally filtered out.

[rdentato]

I agree with @dror-cyata, my constant in PR#270 was arbitrary and I have now replaced it with (1-topp)/(n-1).
The benefit are still substantial. I hope @karpathy will merge one of these PR (there is also #276 on the same topic)

[karpathy]

Merged #276 ty