Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect work on the http2 #113

Open
rasimx opened this issue Oct 20, 2023 · 3 comments
Open

Incorrect work on the http2 #113

rasimx opened this issue Oct 20, 2023 · 3 comments
Assignees
Labels
t-tooling Issues with this label are in the ownership of the tooling team.

Comments

@rasimx
Copy link

rasimx commented Oct 20, 2023

It seems to me that the library does not work correctly over the http/2 when using a proxy. Each request create a new agent, which, in turn, create of a new connection. I tried to reuse an existing agent in Got, this significantly reduces the time to receive responses from the server.

const http2Agent = new Http2OverHttp(wrapperOptions)

// multiple requests
const response = await got(
    `https://some.site`,
    {
      http2: true,
      resolveBodyOnly: true,
      responseType: 'json',
      agent: {
        http2: http2Agent,
      },
    },
  );
@B4nan B4nan added the t-tooling Issues with this label are in the ownership of the tooling team. label Oct 23, 2023
@B4nan B4nan added this to the 75th sprint - Tooling team milestone Oct 23, 2023
@vladfrangu
Copy link
Member

hi! Taking a look at this issue right now, do you have a reproduction sample we can use to try to debug this? 🙏

Looking at the code, got-scraping only overwrites the agents provided by you when you also set a proxyUrl.
Also, what are the options you gave to the Http2OverHttp agent?

@rasimx
Copy link
Author

rasimx commented Nov 3, 2023

Apparently, I expressed myself incorrectly. I meant, when using the proxyUrl option, got-scraping creates a new agent for each request. As a result, the new agent creates a new connection for each request. But this does not correspond to the concept of http2, when a new connection is created only at the first request, and all other requests use an existing connection.
It should use an existing agent instead of creating a new one for each request.
I gave the code above as an example of how I used an existing agent in got

@tonybruess
Copy link

Please see my comment which explains this issue: #112 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
t-tooling Issues with this label are in the ownership of the tooling team.
Projects
None yet
Development

No branches or pull requests

5 participants