Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GH-103472: close response in HTTPConnection._tunnel #103473

Merged
merged 8 commits into from
May 2, 2023
Merged

GH-103472: close response in HTTPConnection._tunnel #103473

merged 8 commits into from
May 2, 2023

Conversation

graingert
Copy link
Contributor

@graingert graingert commented Apr 12, 2023 •
edited by bedevere-bot
Loading

@bedevere-bot bedevere-bot mentioned this pull request Apr 12, 2023
Closed
@graingert graingert marked this pull request as ready for review April 12, 2023 13:07
@arhadthedev
Copy link
Member

@gpshead (as a committer into Lib/http/client.py)

@graingert graingert force-pushed the close-response-in-http-client-tunnel branch from 86c29f5 to 2aa460c Compare April 25, 2023 14:55
orsenthil
orsenthil requested changes Apr 30, 2023
View reviewed changes
Copy link
Member

@orsenthil orsenthil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This piece of code has been there for a while, so making changes here for intermittent failure seems to require bit more understanding on the nature of the error.

Lib/http/client.py Show resolved Hide resolved
Lib/http/client.py Show resolved Hide resolved
@bedevere-bot
Copy link

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@orsenthil orsenthil requested a review from pquentin April 30, 2023 00:39
@graingert
Copy link
Contributor Author

I have made the requested changes; please review again

@bedevere-bot
Copy link

Thanks for making the requested changes!

@orsenthil: please review the changes made to this pull request.

@orsenthil
Copy link
Member

I have made the requested changes; please review again

Did you push any commits? I don't see it.

@graingert
Copy link
Contributor Author

@orsenthil I responded to your comments I didn't need to make any changes

@gpshead
Copy link
Member

gpshead commented May 1, 2023

overall, having tried to review the http.client logic in depth, I think what you're doing in this PR makes sense. Our http.client code is... quite complicated.

As you noted, it is the HTTPResponse sock.makefile() having a side "copy" of the socket as a file object that makes always calling response.close() not mess things up in the _tunnel() case. ("Closing the file object returned by makefile() won’t close the original socket unless all other file objects have been closed and socket.close() has been called on the socket object." - socket docs)

@gpshead gpshead self-assigned this May 1, 2023
@gpshead gpshead added the needs backport to 3.11 only security fixes label May 1, 2023
@gpshead gpshead enabled auto-merge (squash) May 2, 2023 03:36
orsenthil
orsenthil approved these changes May 2, 2023
View reviewed changes
Copy link
Member

@orsenthil orsenthil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM too. Thank you.

@gpshead gpshead merged commit 9de0cf2 into python:main May 2, 2023
@miss-islington
Copy link
Contributor

Thanks @graingert for the PR, and @gpshead for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

@bedevere-bot
Copy link

GH-104077 is a backport of this pull request to the 3.11 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.11 only security fixes label May 2, 2023
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request May 2, 2023
@graingert @gpshead @miss-islington
pythonGH-103472: close response in HTTPConnection._tunnel (pythonGH-1…
8f1ce8f 
…03473)

Avoid a potential `ResourceWarning` in `http.client.HTTPConnection`
by closing the proxy / tunnel's CONNECT response explicitly.

---------

(cherry picked from commit 9de0cf2)

Co-authored-by: Thomas Grainger <tagrain@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
gpshead added a commit that referenced this pull request May 2, 2023
@miss-islington @graingert @gpshead
[3.11] GH-103472: close response in HTTPConnection._tunnel (GH-103473) (
0d40264 
#104077)

GH-103472: close response in HTTPConnection._tunnel (GH-103473)

Avoid a potential `ResourceWarning` in `http.client.HTTPConnection`
by closing the proxy / tunnel's CONNECT response explicitly.

---------

(cherry picked from commit 9de0cf2)

Co-authored-by: Thomas Grainger <tagrain@gmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
@graingert graingert deleted the close-response-in-http-client-tunnel branch May 2, 2023 11:26
carljm added a commit to carljm/cpython that referenced this pull request May 2, 2023
@carljm
Merge branch 'main' into inlinecomp2
0727d6f 
* main:
  pythongh-103822: [Calendar] change return value to enum for day and month APIs (pythonGH-103827)
  pythongh-65022: Fix description of tuple return value in copyreg (python#103892)
  pythonGH-103525: Improve exception message from `pathlib.PurePath()` (pythonGH-103526)
  pythongh-84436: Add integration C API tests for immortal objects (pythongh-103962)
  pythongh-103743: Add PyUnstable_Object_GC_NewWithExtraData (pythonGH-103744)
  pythongh-102997: Update Windows installer to SQLite 3.41.2. (python#102999)
  pythonGH-103484: Fix redirected permanently URLs (python#104001)
  Improve assert_type phrasing (python#104081)
  pythongh-102997: Update macOS installer to SQLite 3.41.2. (pythonGH-102998)
  pythonGH-103472: close response in HTTPConnection._tunnel (python#103473)
  pythongh-88496: IDLE - fix another test on macOS (python#104075)
  pythongh-94673: Hide Objects in PyTypeObject Behind Accessors (pythongh-104074)
  pythongh-94673: Properly Initialize and Finalize Static Builtin Types for Each Interpreter (pythongh-104072)
  pythongh-104016: Skip test for deeply neste f-strings on wasi (python#104071)
carljm added a commit to carljm/cpython that referenced this pull request May 2, 2023
@carljm
Merge branch 'main' into cached_property_docs
b9c39e3 
* main: (760 commits)
  pythonGH-104102: Optimize `pathlib.Path.glob()` handling of `../` pattern segments (pythonGH-104103)
  pythonGH-104104: Optimize `pathlib.Path.glob()` by avoiding repeated calls to `os.path.normcase()` (pythonGH-104105)
  pythongh-103822: [Calendar] change return value to enum for day and month APIs (pythonGH-103827)
  pythongh-65022: Fix description of tuple return value in copyreg (python#103892)
  pythonGH-103525: Improve exception message from `pathlib.PurePath()` (pythonGH-103526)
  pythongh-84436: Add integration C API tests for immortal objects (pythongh-103962)
  pythongh-103743: Add PyUnstable_Object_GC_NewWithExtraData (pythonGH-103744)
  pythongh-102997: Update Windows installer to SQLite 3.41.2. (python#102999)
  pythonGH-103484: Fix redirected permanently URLs (python#104001)
  Improve assert_type phrasing (python#104081)
  pythongh-102997: Update macOS installer to SQLite 3.41.2. (pythonGH-102998)
  pythonGH-103472: close response in HTTPConnection._tunnel (python#103473)
  pythongh-88496: IDLE - fix another test on macOS (python#104075)
  pythongh-94673: Hide Objects in PyTypeObject Behind Accessors (pythongh-104074)
  pythongh-94673: Properly Initialize and Finalize Static Builtin Types for Each Interpreter (pythongh-104072)
  pythongh-104016: Skip test for deeply neste f-strings on wasi (python#104071)
  pythongh-104057: Fix direct invocation of test_super (python#104064)
  pythongh-87092: Expose assembler to unit tests (python#103988)
  pythongh-97696: asyncio eager tasks factory (python#102853)
  pythongh-84436: Immortalize in _PyStructSequence_InitBuiltinWithFlags() (pythongh-104054)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@orsenthil orsenthil orsenthil approved these changes

@gpshead gpshead gpshead approved these changes

@pquentin pquentin Awaiting requested review from pquentin

Assignees

@gpshead gpshead

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@graingert @arhadthedev @bedevere-bot @orsenthil @gpshead @miss-islington @pquentin