perf(gatsby): fix performance regression with query dependency cleaning

[vladar]

Some Context First

A recent refactor of dirty query implementation #27504 caused a performance regression in the query running step. The old implementation before that PR could wipe all data dependencies for queries at once after all of them were executed.

With queries on demand, we will run queries individually and so can't remove data dependencies in a single batch - we have to wipe them for individual queries.

It requires slightly different data structures to make it fast.

Description

Before this PR we were tracking data dependencies per node/connection:

const queriesByNode. = new Map<string, Set<string>>()
const queryIds: Set<string> = queriesByNode.get(nodeId)

This is enough to efficiently mark all queries related to a given node as dirty. But if you want to remove a single query from node dependencies you have to loop through all nodes in this map.

This PR introduces an inverse map for tracked data dependencies to solve this.

const queriesByNode = new Map<string, Set<string>>()
const queryNodes = new Map<string, Set<string>>()
const queries = queriesByNode.get(nodeId)
const nodes = queryNodes.get(queryId)

Now we can first get IDs of all nodes associated with a query. And then remove query <-> node dependency for those individual nodes (so no need to loop through all nodes).

Note: we do not use an inverse map for connections tracking because there is a limited number of connections and in my tests, the overhead associated with inverse maps for connections makes performance worse. We can revisit this later if we find it necessary.

Results

32k pages on my laptop before/after:

success run page queries - 73.472s - 31997/31997 435.50/s
success run page queries - 52.695s - 31994/31994 607.16/s

[pvdz]

Great :D Thanks!

[pvdz]

Suggested change

	const nodeQueries = state.byNode.get(nodeId)
	if (nodeQueries) {
	nodeQueries.delete(queryId)
	}
	state.byNode.get(nodeId)?.delete(queryId)

[vladar]

This was my first attempt. But it gives an Eslint error.