-
-
Notifications
You must be signed in to change notification settings - Fork 72
-
-
Notifications
You must be signed in to change notification settings - Fork 72
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Protocol error (Network.getResponseBody): No resource with given identifier found #89
Comments
Probably this code - I think shot-scraper/shot_scraper/cli.py Lines 728 to 734 in bf34a76
|
Added this debugging code: diff --git a/shot_scraper/cli.py b/shot_scraper/cli.py
index d104725..f5636a8 100644
--- a/shot_scraper/cli.py
+++ b/shot_scraper/cli.py
@@ -726,6 +726,12 @@ def take_shot(
if log_requests:
def on_response(response):
+ try:
+ body = response.body()
+ except Exception as ex:
+ print(ex)
+ print(response.url)
+ return
log_requests.write(
json.dumps(
{ And got this:
|
puppeteer/puppeteer#2258 (comment) says "resources get dumped after page commits navigation" - so presumably what's happening here is that a page navigation has occurred which clears those resources from memory before my Python code gets a chance to call |
My hunch is that it's a lot harder to reliably access the size of the resource than I had expected. |
I'm going to try my best, but return I'll mention this in the documentation. |
This seems to do the right thing: diff --git a/shot_scraper/cli.py b/shot_scraper/cli.py
index d104725..a19e878 100644
--- a/shot_scraper/cli.py
+++ b/shot_scraper/cli.py
@@ -726,12 +726,20 @@ def take_shot(
if log_requests:
def on_response(response):
+ try:
+ body = response.body()
+ size = len(body)
+ except Error as ex:
+ if "Network.getResponseBody" in ex.message:
+ size = None
+ else:
+ raise
log_requests.write(
json.dumps(
{
"method": response.request.method,
"url": response.url,
- "size": len(response.body()),
+ "size": size,
"timing": response.request.timing,
}
) |
Got this error when running:
This was logged out a bunch of times, even though the command itself ran to completion.
I think this is likely caused by the new log requests feature from:
The text was updated successfully, but these errors were encountered: