python310Packages.urllib3: 1.26.14 -> 2.0.3

[brucearctor]

https://github.com/urllib3/urllib3/releases/tag/2.0.3

Description of changes

upgrade to include 2.0.3

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 23.11 Release Notes (or backporting 23.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

[fabaff]

This triggers over 5000+ rebuilds, thus can't go to master. Also, it will probably break a couple of hundreds packages as they have a constrain for urllib3<2.

https://github.com/urllib3/urllib3/releases/tag/2.0.0

[brucearctor]

This triggers over 5000+ rebuilds, thus can't go to master. Also, it will probably break a couple of hundreds packages as they have a constrain for urllib3<2.

https://github.com/urllib3/urllib3/releases/tag/2.0.0

🤦 ; @fabaff , thanks!

1. Looks like I have a more to learn about where to merge this. Will read the docs. In case it helps place where I'm at in nix journey ... still figuring out the relation of release branches [ ex: 23.05 ] - if not point in time snapshots - to master. Will start with https://github.com/NixOS/nixpkgs/blob/master/README.md and then find other.

2. Breaking other packages: Are packages that tightly coupled? Meaning is that a problem for the other packages to eventually fix? Or would need to be part of a PR to make sure to fix the hundreds of other things that don't haven't updated their constraints to urllib3<3 -- any pointers for reading here?

LOL -->

b53d782

and

c033f34 [ a PR of yours ]

made it seem like upgrading versions was simple :-)

[tjni]

This is a good resource for understanding the staging workflow: https://nixos.org/manual/nixpkgs/stable/#submitting-changes-commit-policy.

Breaking other packages: Are packages that tightly coupled? Meaning is that a problem for the other packages to eventually fix? Or would need to be part of a PR to make sure to fix the hundreds of other things that don't haven't updated their constraints to urllib3<3 -- any pointers for reading here?

From my observation, it's unfortunately very common for upper bound constraints to exist on urllib3. A saving grace is that many of the rebuilds needed probably depend on urllib3 transitively, so perhaps fewer packages need fixing.

I like to think we're all on the same team pushing changes to balance keeping packages working with other goals (such as fixing bugs, closing security issues, etc.). So we have to do due diligence to not break many packages without a seriously compelling reason.

---

Onto the PR itself, thank you putting it up! I've been slowly working on the side on it too, so I'd be interested in combining efforts to get this done.

To start with, here are 3 PRs that will be needed to minimize breakages:

• python310Packages.requests-unixsocket: patch to make compatible with urllib3 2+ #240785
• python310Packages.requests-toolbelt: 0.10.1 -> 1.0.0 #240796
• python310Packages.requests-mock: 1.10.0 -> 1.11.0 #241075

Where I need help the most currently is testing botocore (and, relatedly, awscli2) with the new version. It is unfortunately not ready upstream (see boto/botocore#2926), but they need to take into account cloud (e.g. AWS) infrastructure, whereas we might be able to do this faster in nixpkgs since we carry all newer versions of OpenSSL.

[fabaff]

Breaking other packages: Are packages that tightly coupled? Meaning is that a problem for the other packages to eventually fix? Or would need to be part of a PR to make sure to fix the hundreds of other things that don't haven't updated their constraints to urllib3<3 -- any pointers for reading here?

I guess that there are a little over 250 packages present which depends directly on urllib3. urllib3does semantic versioning as far as I can tell and so many modules declare something like urllib3 <2.0, >=1.x.x in their requirements. Unlike PyPI we have only one version of a Python module (there are exceptions but only as a last resort).

Getting widely used modules updated like urllib3, httpx or paramiko updated usually means to get their consumers updated first.

made it seem like upgrading versions was simple :-)

Those urllib3 releases didn't contains breaking changes.

[SuperSandro2000]

Suggested change

	format = "setuptools";
	format = "setuptools";
	diabled = pythonOlder "3.7";

[SuperSandro2000]

If we are doing a major update which most likely requires extra fixups, we can also remove this.

[SuperSandro2000]

Below we should replace pytest-freezegun with freezegun https://github.com/urllib3/urllib3/blob/2.0.3/dev-requirements.txt

[tjni]

I discovered that botocore truly isn't ready yet. urllib3 made changes to its implementation of HTTPConnection in urllib3/urllib3#2565 which breaks botocore's implementation to support "100 Continue" HTTP responses from servers. I will try to work on a fix upstream.

[tjni]

I'll use this comment to record any additional PRs that will be needed before we update this.

• python310Packages.botocore: 1.29.79 -> 1.31.9 #244753
• Handle 100 Continue in urllib3>=2. boto/botocore#2984

[tjni]

We can update botocore to 1.31.15 now and unpin its urllib3 dependency to get us one more step closer to bumping this. Please feel free to keep going on this if anyone has the time.

[wegank]

Already on master.