Release 0.3?

[josephlr]

We have a few changes coming up (#425, #414 , #432) which will make the following changes to the compatibility of this crate:

• Increase the Minimum Supported Rust Version (MSRV) from 1.36 (released July 4, 2019) to 1.56 (released October 21, 2021).
• Increase the supported OS version of *-pc-windows-{gnu,msvc} from Windows Vista (released November 30, 2006) to Windows 10 (released July 15, 2015).
  • Rust increased its minimum Windows OS version from Windows 7 to Windows 10 in February 26th, 2024.
  • Older versions of Windows are still supported under the x86_64-win7-windows-msvc and i686-win7-windows-msvc targets
  • We would intend to continue to support the *-win7-* targets (via RtlGenRandom)
• We are not making any changes to the API of this crate
• We are not making any changes to the features of this crate

The Rust Semver Compatibility Guide does not specify if such actions would be breaking changes. Both:

• changing the minimum version of Rust required
• changing the platform and environment requirements

are listed as "Possibly-breaking" changes.

My personal preference would be to release these changes as 0.2.16 (not consider this a breaking change) because:

• getrandom is widely used throughout the Rust ecosystem so updating all callers can be somewhat tedious.
• It's not clear how a user can have a custom implementation work with both versions of getrandom.
• Both the MSRV increases and the Windows platform increases are less disruptive than an Semver-breaking change.

However, if we do decide to have this be a breaking change, we should discuss if there are any other semver-breaking changes we wish to make (see #346 and #230). Another option would just be to get consensus on #232 and release these changes as 1.0

[josephlr]

CC: @dhardy @vks for how this might affect rand

[newpavlov]

We are not making any changes to the API of this crate

There was a proposal to rename getrandom(_uninit) to fill(_uninit), plus there is #365, but I do not have a concrete proposal for it now. Associated Error constants could also use a bit of work, e.g. I am not sure it's worth to expose constants like VXWORKS_RAND_SECURE on all platforms. We also can use this release to reconsider our cfgs (e.g. #230 and #346). It's also worth to tweak __getrandom_custom's signature to remove this hack.

Moreover, we can use this release to relax our MSRV policy in the vein suggested by @briansmith, but this should be done together with rand.

getrandom is widely used throughout the Rust ecosystem so updating all callers can be somewhat tedious.

I think it's rarely used directly, usually it's done through rand crates.

It's not clear how a user can have a custom implementation work with both versions of getrandom.

It's possible to do this:

[dependencies]
getrandom02 = { version = "0.2", package = "getrandom" }
getrandom = "0.3"

And call the registering macros from each crate (assuming extern functions are called differently).

[dhardy]

As @josephlr says, pushing these changes to a 0.2.x release does not break any formal stability guarantees and likely won't cause a large number of issues, though it's impossible to be certain.

I have a slight preference for bumping the version to 0.3 since the drawbacks are small:

• Custom backends can support multiple versions of getrandom as @newpavlov says. An example of this in action: winit supports three versions of raw-window-handle (optionally).
• No other dependants will be affected. Some build trees may end up using multiple getrandom versions, but this hardly matters.

[briansmith]

• No other dependants will be affected. Some build trees may end up using multiple getrandom versions, but this hardly matters.

I understand what you're saying, but in practice it does matter. Many people have their CI set up to reject multiple versions of a dependency by default. so they have to either make an exception to their CI rules, and/or they submit PRs upstream to try to upgrade everybody to the new version, which is extra work for everybody. Then those upstream projects (such as some I maintain) end up having to choose to drop support for 0.2 to help these people, or spend time trying to explain to them (over and over, indefinitely) why it "hardly matters." Either way it ends up being disproportionate amount of effort.

If we release 0.3:

• What will we cap the MSRV to for the 0.3 series? 1.63 and insist on a 0.4 for any higher MSRV? Otherwise, I guess many projects will want to stick with 0.2.
• When we add new OS support for 0.3, will we backport those to 0.2? In general will we keep maintaining 0.2? Depending on the MSRV policy for 0.3, there is likely going to be pressure to maintain 0.2.

[briansmith]

There was a proposal to rename getrandom(_uninit) to fill(_uninit),

It would be nice but (a) it can be done in a backward-compatible way by renaming it to fill and providing a wrapper named getrandom[_uninit] that is optionally marked #[deprecated].

#365

I think nonblocking getrandom should be a separate API and so it could be added in a SemVer-capable manner. And/or we could implement the old API in terms of the new API, optionally eventually marking the old API #[deprecated].

Associated Error constants could also use a bit of work, e.g. I am not sure it's worth to expose constants like VXWORKS_RAND_SECURE on all platforms.

Better to discuss it in its own issue to reach a decision.

We also can use this release to reconsider our cfgs (e.g. #230 and #346). It's also worth to tweak __getrandom_custom's signature to remove this hack.

Yes, these are basically bugs that we've deferred fixes for until we make a SemVer-breaking version bump, so we should do this for 0.3.

[josephlr]

I opened #438 and #439 to discuss breaking changes to the API and feature flags respectively. I'm convinced by the arguments above, we should release these changes as a breaking change.

• What will we cap the MSRV to for the 0.3 series?

Personally, I think that we should cap the MSRV for the upcoming release at 1.57 (1.56 is Rust 2021, and 1.57 gives us panicking in const which is super nice for static checks). My preference would actually be to release these upcoming changes as 1.0. I know @newpavlov wants to wait for MSRV-aware dependency resolution, but I'm not sure why we should block releasing 1.0 on that feature.

For 1.0 and onward, we can just say that removing platform support and increasing the MSRV can be done in a minor version, and that:

• we won't drop platform support until Rust libstd does
• our MSRV will always be "conservative" (maybe 2 years of old compiler versions?)

That way folks can put getrandom = "1.0" in their dependency file if they are fine with an MSRV of "2 years ago" and if they really want to for-sure not be broken, they can do getrandom = "~1.0"

• When we add new OS support for 0.3, will we backport those to 0.2? In general will we keep maintaining 0.2? Depending on the MSRV policy for 0.3, there is likely going to be pressure to maintain 0.2.

I really wouldn't want to bother back-porting changes. I think we could have backports be "contributions welcome". I'd be fine reviewing backports, but I don't want to do the backporting (especially if nobody's actually using it).

[newpavlov]

I don't think we should release v1.0, not only because of the MSRV-aware resolver, but also because we will introduce new APIs. There are also potential for new features like "global crate features", which can be quite useful for us. Remember that we always can use the semver trick to release v0.3-compatible v1.0, assuming the APIs stay stable enough.

I really wouldn't want to bother back-porting changes. I think we could have backports be "contributions welcome". I'd be fine reviewing backports, but I don't want to do the backporting (especially if nobody's actually using it).

I agree with this, IIRC we acted similarly with v0.1 backports.

[briansmith]

The main problem with trying to make the next release be 1.0: The release will be long-delayed because there's too much perfecting and too many potential compatibility-breaking changes.

I think we need to accept the MSRV increases for 0.2 and do a 0.2 release soon, so that we don't get into a state where we can't do a release because master isn't good enough to be 1.0.

[newpavlov]

@josephlr
FYI I temporarily disabled the review requirement as we discussed in the email. Unreviewed changes start from this commit and have to reviewed before we release v0.3.

[dhardy]

To reply to @newpavlov here instead of in #544:

@dhardy My plan is to release getrandom v0.3 before before rand v0.9. You will not be able to silently migrate from v0.2 to v0.3 even with rust-random/rand#1537 because of the changes around optional backend handling.

So there are breaking changes, but not in the API. We already tell people using rand to use a direct dependency on getrandom when needing to customise the implementation so in a certain sense it would not be a breaking change to update getrandom, but on the other hand it could still cause issues.

I'd like to be able to release rand v1.0 soon after v0.9; I don't know if getrandom has plans for 1.0 soon?

[newpavlov]

I think we could release v1.0 relatively soon (in a year or two after the v0.3 release) assuming no issues will be found with the new APIs. One thing which I would like to do with v1.0 (here and with rand) is to migrate to the 2024 edition and relax our MSRV policy by telling users to rely on the new MSRV-dependent resolver. But since it involves a substantial MSRV bump, I don't think we should rush it.

[dhardy]

From the docs,

The resolver is a global setting for a workspace, and the setting is ignored in dependencies.

So IIUC rand / getrandom doesn't need to use Edition 2024 or resolver=3 to benefit from this; it's enough for a dependant application to use Edition 2024 / resolver=3. This means we don't need to raise our MSRV to benefit.

With that in mind, I see no point in bumping the MSRV to use Edition 2024.

---

Do you have a plan for getrandom v0.3 / something you need help with? I'd like to get rand v0.9 out soon because of gen being a reserved keyword in Edition 2024.

[newpavlov]

So IIUC rand / getrandom doesn't need to use Edition 2024 or resolver=3 to benefit from this

In my understanding, the problem is that resolver=3 will be available only on Rust 1.85 and later. So if rand v0.9.0 and v0.9.1 will have MSRV equal to 1.63 and 1.70, it will create problems for hypothetical users of 1.65 tollchain. So I think it's easier to just bump MSRV to 1.85 in a hypothetical v1.0 release.

Do you have a plan for getrandom v0.3 / something you need help with?

I plan to cut a pre-release after finishing #546. It's probably worth to do a PSA on URLO and Reddit, since the new opt-in target handling could be somewhat controversial with WASM developers.

[josephlr]

Sorry @newpavlov for my delay in reviewing stuff for v0.3. I should have some time over the next week to review stuff. I started by reviewing #559 is there anything in particular I should look at next, or should I just start at the last time a reviewed a PR (which is baae5fc I think)?

[josephlr]

I also really like the organizational changes in #538 it made the code much easier to read.

[newpavlov]

@josephlr
You also can just take a look at the current state of the crate. You may have new ideas/suggestions after looking at it with fresh eyes.

[dhardy]

I would like to get this + rand released early next week if possible. Are there any blocking issues / reviews required / other to-do items?

[newpavlov]

I don't have anything in mind, so we only need a green light from @josephlr.

[josephlr]

I'm mostly done with my review, just a few more things I want to think about (and get other's feedback on):

• Should we use cargo features for purely "additive" backends?
  • This would turn wasm_js and esp_idf --cfg flags into "js" and "esp_idf" features
  • The remaining getrandom_backends would stay as-is.
  • My concern is that enabling JS support is (by far) the most common non-default configuration of this crate, and changing it might be confusing. The addition of targets is "additive" so seems like a better fit for cargo features.
• Should we use a "nightly" feature (or "sanitizer" feature) for enabling cfg_sanitizer?

[josephlr]

Alternatively, now that we can use getrandom_backend = "custom" for any target and it will override the default implementation, we can simply directly support wasm32-unknown-unknown with the JS implementation. Then, in the rare senario someone is using wasm32-unknown-unknown without JS, they can create a custom backend (which they would have to do regardless).

[newpavlov]

I believe we should be consistent and use configuration flags for all optional backends. Enabling the flag is relatively easy and automatically resolves potential issues with some library crates enabling the js crate feature unconditionally. During the pre-release announcement (URLO, Reddit) I did not get any push-back, so hopefully this change will not be too controversial.

Should we use a "nightly" feature (or "sanitizer" feature) for enabling cfg_sanitizer?

The problem with this hypothetical feature is that it will be used only for the sanitizer support. I think thst for sanitizer users it will be a bit easier to enable the configuration flag, than the nightly crate feature for a dependency buried deep in their dependency tree. It would've been a different story if we had some other Nightly-only functionality.

[hanna-kruppe]

As a user who struggled to keep unwanted wasm-bindgen-related code out of non-web wasm32-unknown-unknown builds in the past, I appreciate the move to a cfg flag for that. However, an unfortunate side effect of how it's done in 0.3.0-rc.0 is that Cargo.lock now contains the union of all dependencies that may be needed for any combination of cfg(..)s. That is, if I create an empty library and add getrandom as a dependency (without enabling any features):

• With getrandom 0.2.15, there's five packages in my lockfile (my project, getrandom, cfg-if, libc, and wasi).
• With getrandom 0.3.0-rc.0, the lockfile ends up with 33 packages in total. Including some transitive dependencies that are virtually impossible to enable (e.g., windows-sys is pulled in via rustix, but getrandom only uses rustix on Linux and Android).

The set of dependencies actually built by default hasn't changed, but polluting the lockfile like this has negative side effects. It creates more false positives for tools like cargo vendor, cargo deny, cargo audit, security advisory scans, etc. that parse the lockfile or look at the output of cargo metadata. It can also cause incorrect, unavoidable errors from Cargo itself. For example, adding a dependency on bitflags = "=2.3.3" gives the following error when attempting to regenerate the lockfile, even if the code would never be build in a configuration that actually pulls in both versions of bitflags:

error: failed to select a version for `bitflags`.
    ... required by package `rustix v0.38.38`
    ... which satisfies dependency `rustix = "^0.38.38"` of package `getrandom v0.3.0-rc.0`
    ... which satisfies dependency `getrandom = "=0.3.0-rc.0"` of package `try-getrandom v0.1.0 (/home/hanna/try-getrandom)`
versions that meet the requirements `^2.4.0` are: 2.6.0, 2.5.0, 2.4.2, 2.4.1, 2.4.0

all possible versions conflict with previously selected packages.

  previously selected package `bitflags v2.3.3`
    ... which satisfies dependency `bitflags = "=2.3.3"` of package `try-getrandom v0.1.0  (/home/hanna/try-getrandom)

Edit: also see the discussion at rust-lang/cargo#10801 and the many references to that issue across GitHub for illustration how much trouble overly broad dependencies in lockfiles can cause. At least that's a Cargo issue that should get fixed eventually, while the getrandom 0.3.0-rc.0 design interacts badly with Cargo "as designed" (one lockfile is supposed to work for any target platform and set of cfgs).

One way to mitigate these problem would be to add back feature flags in addition to the cfg(getrandom_backend=...)-gating. This has the downside that anyone who wants to use a non-default backend (that requires extra dependencies, fully custom backends can probably be exempt) has to both enable the getrandom crate's feature and set --cfg getrandom_backend=.... But from my own, biased POV that seems like the lesser evil for such a foundational, widely-used crate.

I'm raising this now because changing the rules from "set this cfg" to "set this cfg and also enable a Cargo feature" would on its face be a backwards incompatible change.

[josephlr]

Thank you very much @hanna-kruppe for raising this concern. This is exactly the type of feedback we want for how to best enable alternative backends for this crate. I am especially sensitive to this concern because in over half of the issues I find complaining about this sort of problem, we are the ultimate cause of said problem.

I am wondering if this could be done using a single cargo feature ("use an alternative backend") + a cfg specifying which alternative backend to use.

I'll think more about how we might address this.

[dhardy]

• With getrandom 0.3.0-rc.0, the lockfile ends up with 33 packages in total. Including some transitive dependencies that are virtually impossible to enable (e.g., windows-sys is pulled in via rustix, but getrandom only uses rustix on Linux and Android).

It may be that the Cargo dependency resolver pulls in packages which can never be used in practice. That is presumably cargo#10801.

But at the same time, many of the dependencies can be used on some platform, if I understand correctly. According to this comment, Cargo.lock must be the same on all platforms, thus many of the entries would appear to be legitimate from the point of Cargo's dependency resolver.

[hanna-kruppe]

Yes, the lockfile is intended to cover all platforms, that's why e.g. wasi ends up in the lockfile even with getrandom 0.2. I don't take any issue with that aspect! But the lockfile does take into account which crate features are enabled and what that entails for dependencies. The Cargo bug 10801 only affects "weak" feature implications (myfeature = ["somedep?/somefeature"]), which aren't relevant here. All of the crates in the lockfile could actually be built with the right combination of cfgs, but in most cases that combination must include a certain value for cfg(getrandom_backend=...).

However, most of these dependencies are not actually used on any platform, unless someone opts into a non-default backend. Cargo can reason about this when the opt-in happens via features, but it intentionally and quite reasonably doesn't do any reasoning about cfg expressions. I understand why y'all moved away from feature flags and support that. But always having all dependencies for all getrandom backends in the lockfile is undesirable:

• When I work on an application that I compile and run on my own infrastructure or distribute to end users, I'll pick one of the getrandom backend per platform I build for. In these cases, I'd want my lockfile to reflect the getrandom backends I'm actually using. (I may also want to tailor my lockfile to just the platforms I care about, but that's a feature request for Cargo.)
• For applications that are built from source by various people, there will be cases where someone has a good reason to build "basically upstream, just with a different getrandom backend". However, this still builds substantially different code than upstream intended and tested, so I don't see much value in using the exact same lockfile. Regenerating the lockfile may even be helpful because the diff tells you more about what effect the change of backend had.
• When I work on a library using getrandom, I can't say which backend will be chosen by applications using my library. But those applications don't care about my lockfile anyway. I'm generally only going to develop and test my library with the default getrandom backends, so any dependencies for opt-in backends are unused for my purposes and shouldn't go into the lockfile.
• Both applications and libraries are affected by the negative side effects of spurious lockfile entries described in my previous comment.

To be clear: any new dependencies that are used by default on some platform are totally fine. I mean, I'd still appreciate this set to be as small as possible for other reasons, but it is valuable to have lockfile reflect what will be built on any unforseen platform. I'm only talking about dependencies that were or would have been feature-gated in version 0.2 but are gated on cfg(getrandom_backend=...) in version 0.3.0-rc.0.

[newpavlov]

I think the main issue is that cargo is able to exclude feature-dependent project dependencies from Cargo.lock if it can see that the feature is not enabled anywhere in the project. For example, with just getrandom = "0.2" cargo will not include into lock file wasm-bindgen and js-sys dependencies since it can see that the project does not enable the js crate feature. But with the configuration flag approach it has to include them since user may pass the flag during compilation, thus implicitly pulling those dependencies.

getrandom v0.3 has the following dependency tree right now:

foo v0.1.0 (/tmp/foo)
└── getrandom v0.3.0-rc.0
    ├── cfg-if v1.0.0
    ├── js-sys v0.3.76
    │   ├── once_cell v1.20.2
    │   └── wasm-bindgen v0.2.99
    │       ├── cfg-if v1.0.0
    │       ├── once_cell v1.20.2
    │       └── wasm-bindgen-macro v0.2.99 (proc-macro)
    │           ├── quote v1.0.37
    │           │   └── proc-macro2 v1.0.92
    │           │       └── unicode-ident v1.0.14
    │           └── wasm-bindgen-macro-support v0.2.99
    │               ├── proc-macro2 v1.0.92 (*)
    │               ├── quote v1.0.37 (*)
    │               ├── syn v2.0.90
    │               │   ├── proc-macro2 v1.0.92 (*)
    │               │   ├── quote v1.0.37 (*)
    │               │   └── unicode-ident v1.0.14
    │               ├── wasm-bindgen-backend v0.2.99
    │               │   ├── bumpalo v3.16.0
    │               │   ├── log v0.4.22
    │               │   ├── proc-macro2 v1.0.92 (*)
    │               │   ├── quote v1.0.37 (*)
    │               │   ├── syn v2.0.90 (*)
    │               │   └── wasm-bindgen-shared v0.2.99
    │               └── wasm-bindgen-shared v0.2.99
    ├── libc v0.2.169
    ├── rustix v0.38.42
    │   ├── bitflags v2.6.0
    │   ├── errno v0.3.10
    │   │   ├── libc v0.2.169
    │   │   └── windows-sys v0.59.0
    │   │       └── windows-targets v0.52.6
    │   │           ├── windows_aarch64_gnullvm v0.52.6
    │   │           ├── windows_aarch64_msvc v0.52.6
    │   │           ├── windows_i686_gnu v0.52.6
    │   │           ├── windows_i686_gnullvm v0.52.6
    │   │           ├── windows_i686_msvc v0.52.6
    │   │           ├── windows_x86_64_gnu v0.52.6
    │   │           ├── windows_x86_64_gnullvm v0.52.6
    │   │           └── windows_x86_64_msvc v0.52.6
    │   ├── libc v0.2.169
    │   ├── linux-raw-sys v0.4.14
    │   └── windows-sys v0.59.0 (*)
    ├── wasi v0.13.3+wasi-0.2.2
    │   └── wit-bindgen-rt v0.33.0
    │       └── bitflags v2.6.0
    ├── wasm-bindgen v0.2.99 (*)
    └── windows-targets v0.52.6 (*)

Some of the "bloat" (windows-targets and wasi v0.13) is caused by changes in the default backends and will stay regardless of the opt-in backends.

The rustix backend adds 4 dependencies (rustix, windows-sys, linux-raw-sys, errno). It's unfortunate (and one of the reasons why I argued that we need a "raw Linux syscall" crate), but I guess it's tolerable. This part can be improved by replacing rustix with a leaner crate or we could implement the "raw syscall" functionality ourselves.

js-sys/wasm-bindgen are the biggest offenders here, but, on the other hand, they are de facto "standard" and get pulled by many non-trivial projects.

Overall, while I agree that the lock file bloat is unfortunate, I don't think we should introduce hacks like requiring to enable both configuration flag and crate feature simultaneously. It may be worth to look for a better approach, but I don't have any ideas right now.

It can also cause incorrect, unavoidable errors from Cargo itself. For example, adding a dependency on bitflags = "=2.3.3" gives the following error when attempting to regenerate the lockfile

Such dependency bounds are notoriously fragile and should not be used. And, as you can see above, bitflags gets pulled by the WASI backend, so you would have this problem even without the opt-in backends.

[hanna-kruppe]

js-sys/wasm-bindgen are the biggest offenders here, but, on the other hand, they are de facto "standard" and get pulled by most non-trivial projects.

I don't know how to quantify this but it doesn't feel correct to me (and I do have some non-trivial projects that avoid them). These crates have a lot of reverse dependencies, but many of those are feature-gated or come from applications/libraries specific to the wasm-bindgen ecosystem. General-purpose libraries that unconditionally depend on wasm-bindgen/js-sys on wasm32-unknown-unknown (directly or by enabling features of other libraries) are simply buggy. There were and still are some popular/high-profile libraries that have such bugs, but their number has decreased and hopefully continues to decrease. Please don't make it worse again.

Such dependency bounds are notoriously fragile and should not be used. And, as you can see above, bitflags gets pulled by the WASI backend, so you would have this problem even without the opt-in backends.

Sorry for any confusion caused by picking bitflags without realizing it's used by a default backend. I don't care about that specific package, I just wanted to trigger the conflicting versions error and bitflags was listed first in the lockfile. I completely agree that such bounds are extremely fragile should be avoided at almost any cost. Unfortunately this is not a universally accepted position (e.g., some libraries pin versions or add upper bounds to support a MSRV policy), so large dependency graphs can easily end up with an instance of it somewhere. This usually gets resolved eventually by relaxing the version bound, but it causes a bunch of pain in the meantime and feels especially silly when the offending library isn't actually used.

Aside: a similar problem that's more difficult to fix but also triggers more rarely is with the links = "..." field for *-sys crates. Cargo will error if two crates (usually, two semver-incompatible versions of the same crate) set that to the same value. This happens more rarely, but it does happen and can't always be fixed by just patching version constraints. I don't see anything in the current getrandom dependency tree that seems likely to run into this, but if it ever happens then getrandom's popularity will make it very painful. Like other problems we're discussing, it's not exclusive to opt-in backends, but any extra lockfile entry beyond what's actually necessary increases the probability and blast radius.