Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ResourceWarning due to unclosed response in http.client.HTTPConnection._tunnel #103472

Closed
graingert opened this issue Apr 12, 2023 · 1 comment
Closed
Labels
3.11 only security fixes 3.12 bugs and security fixes 3.13 bugs and security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@graingert
Copy link
Contributor

graingert commented Apr 12, 2023

Bug report

calling http.client.HTTPConnection._tunnel() may result in a ResourceWarning because the response is not closed

this results in an intermittent failure on the urllib3 test test/with_dummyserver/test_proxy_poolmanager.py::TestHTTPProxyManager::test_tunneling_proxy_request_timeout[https-https] with ResourceWarnings enabled, see https://github.com/urllib3/urllib3/actions/runs/4638944124/jobs/8209257011#step:6:2471 and https://github.com/urllib3/urllib3/blob/4d425a2d767ce2e0c988fff4385046ea57458c49/test/with_dummyserver/test_proxy_poolmanager.py#L480-L498

Your environment

  • CPython versions tested on: 3.8-3.11

Linked PRs

@graingert graingert added type-bug An unexpected behavior, bug, or error stdlib Python modules in the Lib dir 3.11 only security fixes 3.12 bugs and security fixes 3.13 bugs and security fixes labels Apr 12, 2023
graingert added a commit to graingert/cpython that referenced this issue Apr 12, 2023
@graingert graingert changed the title ResourceWarning due to unclosed request in http.client.HTTPConnection._tunnel ResourceWarning due to unclosed response in http.client.HTTPConnection._tunnel Apr 12, 2023
gpshead added a commit that referenced this issue May 2, 2023
Avoid a potential `ResourceWarning` in `http.client.HTTPConnection`
by closing the proxy / tunnel's CONNECT response explicitly.

---------

Co-authored-by: Gregory P. Smith <greg@krypto.org>
miss-islington pushed a commit to miss-islington/cpython that referenced this issue May 2, 2023
…03473)

Avoid a potential `ResourceWarning` in `http.client.HTTPConnection`
by closing the proxy / tunnel's CONNECT response explicitly.

---------

(cherry picked from commit 9de0cf2)

Co-authored-by: Thomas Grainger <tagrain@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
gpshead added a commit that referenced this issue May 2, 2023
#104077)

GH-103472: close response in HTTPConnection._tunnel (GH-103473)

Avoid a potential `ResourceWarning` in `http.client.HTTPConnection`
by closing the proxy / tunnel's CONNECT response explicitly.

---------

(cherry picked from commit 9de0cf2)

Co-authored-by: Thomas Grainger <tagrain@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
carljm added a commit to carljm/cpython that referenced this issue May 2, 2023
* main:
  pythongh-103822: [Calendar] change return value to enum for day and month APIs (pythonGH-103827)
  pythongh-65022: Fix description of tuple return value in copyreg (python#103892)
  pythonGH-103525: Improve exception message from `pathlib.PurePath()` (pythonGH-103526)
  pythongh-84436: Add integration C API tests for immortal objects (pythongh-103962)
  pythongh-103743: Add PyUnstable_Object_GC_NewWithExtraData (pythonGH-103744)
  pythongh-102997: Update Windows installer to SQLite 3.41.2. (python#102999)
  pythonGH-103484: Fix redirected permanently URLs (python#104001)
  Improve assert_type phrasing (python#104081)
  pythongh-102997: Update macOS installer to SQLite 3.41.2. (pythonGH-102998)
  pythonGH-103472: close response in HTTPConnection._tunnel (python#103473)
  pythongh-88496: IDLE - fix another test on macOS (python#104075)
  pythongh-94673: Hide Objects in PyTypeObject Behind Accessors (pythongh-104074)
  pythongh-94673: Properly Initialize and Finalize Static Builtin Types for Each Interpreter (pythongh-104072)
  pythongh-104016: Skip test for deeply neste f-strings on wasi (python#104071)
carljm added a commit to carljm/cpython that referenced this issue May 2, 2023
* main: (760 commits)
  pythonGH-104102: Optimize `pathlib.Path.glob()` handling of `../` pattern segments (pythonGH-104103)
  pythonGH-104104: Optimize `pathlib.Path.glob()` by avoiding repeated calls to `os.path.normcase()` (pythonGH-104105)
  pythongh-103822: [Calendar] change return value to enum for day and month APIs (pythonGH-103827)
  pythongh-65022: Fix description of tuple return value in copyreg (python#103892)
  pythonGH-103525: Improve exception message from `pathlib.PurePath()` (pythonGH-103526)
  pythongh-84436: Add integration C API tests for immortal objects (pythongh-103962)
  pythongh-103743: Add PyUnstable_Object_GC_NewWithExtraData (pythonGH-103744)
  pythongh-102997: Update Windows installer to SQLite 3.41.2. (python#102999)
  pythonGH-103484: Fix redirected permanently URLs (python#104001)
  Improve assert_type phrasing (python#104081)
  pythongh-102997: Update macOS installer to SQLite 3.41.2. (pythonGH-102998)
  pythonGH-103472: close response in HTTPConnection._tunnel (python#103473)
  pythongh-88496: IDLE - fix another test on macOS (python#104075)
  pythongh-94673: Hide Objects in PyTypeObject Behind Accessors (pythongh-104074)
  pythongh-94673: Properly Initialize and Finalize Static Builtin Types for Each Interpreter (pythongh-104072)
  pythongh-104016: Skip test for deeply neste f-strings on wasi (python#104071)
  pythongh-104057: Fix direct invocation of test_super (python#104064)
  pythongh-87092: Expose assembler to unit tests (python#103988)
  pythongh-97696: asyncio eager tasks factory (python#102853)
  pythongh-84436: Immortalize in _PyStructSequence_InitBuiltinWithFlags() (pythongh-104054)
  ...
@hauntsaninja
Copy link
Contributor

Looks like this was fixed, thanks all!

netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Sep 12, 2024
2.2.3 (2024-09-12)
==================

Features
--------

- Added support for Python 3.13.

Bugfixes
--------

- Fixed the default encoding of chunked request bodies to be UTF-8 instead of ISO-8859-1.
  All other methods of supplying a request body already use UTF-8 starting in urllib3 v2.0.
- Fixed ResourceWarning on CONNECT with Python < 3.11.4 by backporting python/cpython#103472.
- Adjust tolerance for floating-point comparison on Windows to avoid flakiness in CI
- Fixed a crash where certain standard library hash functions were absent in restricted environments.
- Fixed mypy error when adding to ``HTTPConnection.default_socket_options``.

HTTP/2 (experimental)
---------------------

HTTP/2 support is still in early development.

- Excluded Transfer-Encoding: chunked from HTTP/2 request body
- Added version checking for ``h2`` (https://pypi.org/project/h2/) usage.

  Now only accepting supported h2 major version 4.x.x.
- Added a probing mechanism for determining whether a given target origin
  supports HTTP/2 via ALPN.
- Add support for sending a request body with HTTP/2

Deprecations and Removals
-------------------------

- Note for downstream distributors: the ``_version.py`` file has been removed and is now created at build time by hatch-vcs.
- Drop support for end-of-life PyPy3.8 and PyPy3.9.
apereocas-bot referenced this issue in apereo/cas Sep 13, 2024
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [urllib3](https://redirect.github.com/urllib3/urllib3) ([changelog](https://redirect.github.com/urllib3/urllib3/blob/main/CHANGES.rst)) | `==2.2.2` -> `==2.2.3` | [![age](https://developer.mend.io/api/mc/badges/age/pypi/urllib3/2.2.3?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/urllib3/2.2.3?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/urllib3/2.2.2/2.2.3?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/urllib3/2.2.2/2.2.3?slim=true)](https://docs.renovatebot.com/merge-confidence/) |

---

### Release Notes

<details>
<summary>urllib3/urllib3 (urllib3)</summary>

### [`v2.2.3`](https://redirect.github.com/urllib3/urllib3/blob/HEAD/CHANGES.rst#223-2024-09-12)

[Compare Source](https://redirect.github.com/urllib3/urllib3/compare/2.2.2...2.2.3)

\==================

## Features

-   Added support for Python 3.13. (`#&#8203;3473 <https://github.com/urllib3/urllib3/issues/3473>`\__)

## Bugfixes

-   Fixed the default encoding of chunked request bodies to be UTF-8 instead of ISO-8859-1.
    All other methods of supplying a request body already use UTF-8 starting in urllib3 v2.0. (`#&#8203;3053 <https://github.com/urllib3/urllib3/issues/3053>`\__)
-   Fixed ResourceWarning on CONNECT with Python < 3.11.4 by backporting [https://github.com/python/cpython/issues/103472](https://redirect.github.com/python/cpython/issues/103472). (`#&#8203;3252 <https://github.com/urllib3/urllib3/issues/3252>`\__)
-   Adjust tolerance for floating-point comparison on Windows to avoid flakiness in CI (`#&#8203;3413 <https://github.com/urllib3/urllib3/issues/3413>`\__)
-   Fixed a crash where certain standard library hash functions were absent in restricted environments. (`#&#8203;3432 <https://github.com/urllib3/urllib3/issues/3432>`\__)
-   Fixed mypy error when adding to `HTTPConnection.default_socket_options`. (`#&#8203;3448 <https://github.com/urllib3/urllib3/issues/3448>`\__)

## HTTP/2 (experimental)

HTTP/2 support is still in early development.

-   Excluded Transfer-Encoding: chunked from HTTP/2 request body (`#&#8203;3425 <https://github.com/urllib3/urllib3/issues/3425>`\__)
-   Added version checking for `h2` (https://pypi.org/project/h2/) usage.

    Now only accepting supported h2 major version 4.x.x. (`#&#8203;3290 <https://github.com/urllib3/urllib3/issues/3290>`\__)
-   Added a probing mechanism for determining whether a given target origin
    supports HTTP/2 via ALPN. (`#&#8203;3301 <https://github.com/urllib3/urllib3/issues/3301>`\__)
-   Add support for sending a request body with HTTP/2 (`#&#8203;3302 <https://github.com/urllib3/urllib3/issues/3302>`\__)

## Deprecations and Removals

-   Note for downstream distributors: the `_version.py` file has been removed and is now created at build time by hatch-vcs. (`#&#8203;3412 <https://github.com/urllib3/urllib3/issues/3412>`\__)
-   Drop support for end-of-life PyPy3.8 and PyPy3.9. (`#&#8203;3475 <https://github.com/urllib3/urllib3/issues/3475>`\__)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "after 10pm every weekday,before 6am every weekday" (UTC), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/apereo/cas).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOC43NC4xIiwidXBkYXRlZEluVmVyIjoiMzguNzQuMSIsInRhcmdldEJyYW5jaCI6Im1hc3RlciIsImxhYmVscyI6WyJCb3QiLCJSZW5vdmF0ZSJdfQ==-->
lqiu96 referenced this issue in googleapis/sdk-platform-java Sep 16, 2024
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [urllib3](https://redirect.github.com/urllib3/urllib3)
([changelog](https://redirect.github.com/urllib3/urllib3/blob/main/CHANGES.rst))
| `==2.2.2` -> `==2.2.3` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/urllib3/2.2.3?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/urllib3/2.2.3?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/urllib3/2.2.2/2.2.3?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/urllib3/2.2.2/2.2.3?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|

---

> [!WARNING]
> Some dependencies could not be looked up. Check the Dependency
Dashboard for more information.

---

### Release Notes

<details>
<summary>urllib3/urllib3 (urllib3)</summary>

###
[`v2.2.3`](https://redirect.github.com/urllib3/urllib3/blob/HEAD/CHANGES.rst#223-2024-09-12)

[Compare
Source](https://redirect.github.com/urllib3/urllib3/compare/2.2.2...2.2.3)

\==================

## Features

- Added support for Python 3.13. (`#&#8203;3473
<https://github.com/urllib3/urllib3/issues/3473>`\__)

## Bugfixes

- Fixed the default encoding of chunked request bodies to be UTF-8
instead of ISO-8859-1.
All other methods of supplying a request body already use UTF-8 starting
in urllib3 v2.0. (`#&#8203;3053
<https://github.com/urllib3/urllib3/issues/3053>`\__)
- Fixed ResourceWarning on CONNECT with Python < 3.11.4 by backporting
[https://github.com/python/cpython/issues/103472](https://redirect.github.com/python/cpython/issues/103472).
(`#&#8203;3252 <https://github.com/urllib3/urllib3/issues/3252>`\__)
- Adjust tolerance for floating-point comparison on Windows to avoid
flakiness in CI (`#&#8203;3413
<https://github.com/urllib3/urllib3/issues/3413>`\__)
- Fixed a crash where certain standard library hash functions were
absent in restricted environments. (`#&#8203;3432
<https://github.com/urllib3/urllib3/issues/3432>`\__)
- Fixed mypy error when adding to
`HTTPConnection.default_socket_options`. (`#&#8203;3448
<https://github.com/urllib3/urllib3/issues/3448>`\__)

## HTTP/2 (experimental)

HTTP/2 support is still in early development.

- Excluded Transfer-Encoding: chunked from HTTP/2 request body
(`#&#8203;3425 <https://github.com/urllib3/urllib3/issues/3425>`\__)
- Added version checking for `h2` (https://pypi.org/project/h2/) usage.

Now only accepting supported h2 major version 4.x.x. (`#&#8203;3290
<https://github.com/urllib3/urllib3/issues/3290>`\__)
- Added a probing mechanism for determining whether a given target
origin
supports HTTP/2 via ALPN. (`#&#8203;3301
<https://github.com/urllib3/urllib3/issues/3301>`\__)
- Add support for sending a request body with HTTP/2 (`#&#8203;3302
<https://github.com/urllib3/urllib3/issues/3302>`\__)

## Deprecations and Removals

- Note for downstream distributors: the `_version.py` file has been
removed and is now created at build time by hatch-vcs. (`#&#8203;3412
<https://github.com/urllib3/urllib3/issues/3412>`\__)
- Drop support for end-of-life PyPy3.8 and PyPy3.9. (`#&#8203;3475
<https://github.com/urllib3/urllib3/issues/3475>`\__)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/googleapis/sdk-platform-java).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOC43NC4xIiwidXBkYXRlZEluVmVyIjoiMzguNzQuMSIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOltdfQ==-->
ldetmer referenced this issue in googleapis/sdk-platform-java Sep 17, 2024
This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [urllib3](https://redirect.github.com/urllib3/urllib3)
([changelog](https://redirect.github.com/urllib3/urllib3/blob/main/CHANGES.rst))
| `==2.2.2` -> `==2.2.3` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/urllib3/2.2.3?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/urllib3/2.2.3?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/urllib3/2.2.2/2.2.3?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/urllib3/2.2.2/2.2.3?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|

---

> [!WARNING]
> Some dependencies could not be looked up. Check the Dependency
Dashboard for more information.

---

### Release Notes

<details>
<summary>urllib3/urllib3 (urllib3)</summary>

###
[`v2.2.3`](https://redirect.github.com/urllib3/urllib3/blob/HEAD/CHANGES.rst#223-2024-09-12)

[Compare
Source](https://redirect.github.com/urllib3/urllib3/compare/2.2.2...2.2.3)

\==================

## Features

- Added support for Python 3.13. (`#&#8203;3473
<https://github.com/urllib3/urllib3/issues/3473>`\__)

## Bugfixes

- Fixed the default encoding of chunked request bodies to be UTF-8
instead of ISO-8859-1.
All other methods of supplying a request body already use UTF-8 starting
in urllib3 v2.0. (`#&#8203;3053
<https://github.com/urllib3/urllib3/issues/3053>`\__)
- Fixed ResourceWarning on CONNECT with Python < 3.11.4 by backporting
[https://github.com/python/cpython/issues/103472](https://redirect.github.com/python/cpython/issues/103472).
(`#&#8203;3252 <https://github.com/urllib3/urllib3/issues/3252>`\__)
- Adjust tolerance for floating-point comparison on Windows to avoid
flakiness in CI (`#&#8203;3413
<https://github.com/urllib3/urllib3/issues/3413>`\__)
- Fixed a crash where certain standard library hash functions were
absent in restricted environments. (`#&#8203;3432
<https://github.com/urllib3/urllib3/issues/3432>`\__)
- Fixed mypy error when adding to
`HTTPConnection.default_socket_options`. (`#&#8203;3448
<https://github.com/urllib3/urllib3/issues/3448>`\__)

## HTTP/2 (experimental)

HTTP/2 support is still in early development.

- Excluded Transfer-Encoding: chunked from HTTP/2 request body
(`#&#8203;3425 <https://github.com/urllib3/urllib3/issues/3425>`\__)
- Added version checking for `h2` (https://pypi.org/project/h2/) usage.

Now only accepting supported h2 major version 4.x.x. (`#&#8203;3290
<https://github.com/urllib3/urllib3/issues/3290>`\__)
- Added a probing mechanism for determining whether a given target
origin
supports HTTP/2 via ALPN. (`#&#8203;3301
<https://github.com/urllib3/urllib3/issues/3301>`\__)
- Add support for sending a request body with HTTP/2 (`#&#8203;3302
<https://github.com/urllib3/urllib3/issues/3302>`\__)

## Deprecations and Removals

- Note for downstream distributors: the `_version.py` file has been
removed and is now created at build time by hatch-vcs. (`#&#8203;3412
<https://github.com/urllib3/urllib3/issues/3412>`\__)
- Drop support for end-of-life PyPy3.8 and PyPy3.9. (`#&#8203;3475
<https://github.com/urllib3/urllib3/issues/3475>`\__)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR was generated by [Mend Renovate](https://mend.io/renovate/).
View the [repository job
log](https://developer.mend.io/github/googleapis/sdk-platform-java).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOC43NC4xIiwidXBkYXRlZEluVmVyIjoiMzguNzQuMSIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOltdfQ==-->
bmwiedemann pushed a commit to bmwiedemann/openSUSE that referenced this issue Oct 3, 2024
https://build.opensuse.org/request/show/1205339
by user StevenK + anag+factory
- Update to 2.2.3:
  * Features
    + Added support for Python 3.13.
  * Bugfixes
    + Fixed the default encoding of chunked request bodies to be UTF-8
      instead of ISO-8859-1. All other methods of supplying a request body
      already use UTF-8 starting in urllib3 v2.0.
    + Fixed ResourceWarning on CONNECT with Python < 3.11.4 by backporting
      python/cpython#103472.
    + Fixed a crash where certain standard library hash functions were absent
      in restricted environments.
    + Added the Proxy-Authorization header to the list of headers to strip
      from requests when redirecting to a different host. As before,
      different headers can be set via Retry.remove_headers_on_redirect.
    + Allowed passing negative integers as amt to read methods of
      http.client.HTTPRe
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.11 only security fixes 3.12 bugs and security fixes 3.13 bugs and security fixes stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

2 participants