After processing a fair amount of data, Deno hangs when reading the response body

[NiceGuyIT]

I'm downloading data from an API and Deno is freezing after a fair amount of data. It seems to be related to the amount of data because it will hang somewhere between 200 and 400 pages with a page size of 100, but will hang before downloading 100 pages with a page size of 1000. They all exhibit the same problem. The amount of data downloaded before the hang is random, but the hang is inevitable.

By "hang" I mean reading the data from the response will not return. it does not matter if the function is response.arrayBuffer(), response.text() or response.json(). An exception is not thrown and the promise to read the response never resolves.

Adding --inspect-wait to Deno's command line and attaching a debugger shows the code is hanging in the first for loop (line 171) of eventLoopTick() in core/01_core.js. When the hang happens, CPU usage is 0%.

I compiled Deno using commit 3db9c87 (latest as of writing this bug report) and the problem is still present.

deno 1.39.1 (debug, x86_64-unknown-linux-gnu)
v8 12.0.267.8
typescript 5.3.3

The earliest Deno version I could compile was v1.18.1. The problem is still present.

deno 1.18.1 (debug, x86_64-unknown-linux-gnu)
v8 9.8.177.6
typescript 4.5.2

The problem exists on Windows...

deno 1.39.1 (release, x86_64-pc-windows-msvc)
v8 12.0.267.8
typescript 5.3.3

and macOS.

deno 1.39.1 (release, aarch64-apple-darwin)
v8 12.0.267.8
typescript 5.3.3

To reproduce the problem, save the TypeScript below as bug-client.ts and run this command. The code only downloads data, it does not save the data.

deno run --unstable --allow-all ./bug-client.ts

I also downloaded all the data and presented this using Deno as a web server but could not reproduce the problem. (Code can be provided if needed.) I don't know if this is because localhost has near 0 latency or some other factor.

bug-client.ts:

const pageSize = 1000;
let pageNum = 1;
let nextPageToken: string | undefined = undefined;

const server = `https://clinicaltrials.gov/api/v2/studies`
let jsonResponse: any;
let response: Response;
let buff: ArrayBuffer;
let data: string;

do {
	const url = `${server}?pageSize=${pageSize}${nextPageToken ? `&pageToken=${nextPageToken}` : ""}`;

	console.debug(`nextPageToken:`, nextPageToken, `Fetching URL:`, url);
	response = await fetch(url, {
		headers: {
			accept: "application/json",
		},
	});

	if (!response.ok) {
		console.error(`HTTP error! Status: ${response.status}`);
		Deno.exit(1)
	}

	try {
		console.debug(`Reading text from response`);
		// This is the code that hangs and never returns.
		buff = await response.arrayBuffer();
		console.debug(`arrayBuffer length:`, buff.byteLength);
		data = new TextDecoder().decode(buff);
		jsonResponse = JSON.parse(data);
	} catch (err: unknown) {
		console.error(`Failed to parse JSON: ${err}`);
		Deno.exit(1)
	}

	nextPageToken = jsonResponse.nextPageToken;
	console.info(`pageNum:`, pageNum, `count:`, jsonResponse.studies.length, `nextPageToken:`, nextPageToken);
	pageNum++;
} while (nextPageToken);

[kt3k]

Ran the same script with type annotations stripped in Node.js and it was able to retrieve all 478 pages. This looks like Deno issue to me.

@bartlomieju @mmastrac Do you think of any cause of this?

[mmastrac]

@bartlomieju and I looked into this and we suspect that it's something to do with reqwest's connection pool and potentially an interaction w/the HTTP/2 connection.

We tried forcing the TCP connection closed and the fetches did not make progress, which suggests that a waker may be getting lost inside reqwest, or a protocol error is not being surfaced.