Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rand BFloat16 sampling without conversion #73

Merged
merged 2 commits into from
Sep 16, 2024
Merged

rand BFloat16 sampling without conversion #73

merged 2 commits into from
Sep 16, 2024

Conversation

milankl
Copy link
Member

@milankl milankl commented Apr 12, 2024

fixes #71

Technically randn, randexp should be written directly for BFloat16 too to avoid the rounding errors from conversion but they are 1) more complicted (see ziggurat algorithm for randn) 2) randn is continuous so rounding from conversion might be fine, randexp doesn't seem to produce very small numbers (<<1e-10) let alone 0 in practice for Float32/64, so rounding because of the conversion might be fine here too

Edit: I was wrong on randexp being complicated, from wikipedia

image

@ViralBShah
Copy link
Member

@oscardssmith should we merge?

@oscardssmith
Copy link
Member

oscardssmith commented Sep 9, 2024

This implementation is slightly less precise than necessary, why not use the same one used by Float16? (e.g. BFloat16(Float32(rand(r, UInt8)) * Float32(0x1.0p-8))

@JeffreySarnoff
Copy link
Member

just saw this -- thank you

@JeffreySarnoff JeffreySarnoff merged commit 2811afa into master Sep 16, 2024
9 checks passed
@JeffreySarnoff JeffreySarnoff deleted the mk/rand branch September 16, 2024 20:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

rand(BFloat16, n) appears shifted from [0, 1) into (0, 1]
4 participants