optimize dep-graph serialization

[nikomatsakis]

Looking at the syntex-syntax crate, I see that dep-graph serialization is pretty expensive:

time: 13.588; rss: 424MB        serialize dep graph
incremental: re-using 50 out of 50 modules

We're saving way more data than we really need to. I expect we can make this a lot cheaper by pruning the graph. But it'd be worth doing some profiling.

cc @michaelwoerister

[nikomatsakis]

Also surprising, if unrelated: even with 100% reuse, translation takes ~4s on my machine. I think the only thing it's doing is re-generating the metadata.

[nikomatsakis]

OK, so, my first attempt at optimizing this was kind of a failure, in that it caused us to take more time than before. =)

Some things I learned:

• the expensive part is the meta-data hash computation
• some part of this is computing the sets of HIR inputs for a given metadata, some part of it is probably creating the strings and sorting and so forth; haven't quite gotten to the bottom of it

Also worth nothing:

When I rewrote this code, some of the problems I've encountered with non-determinstic re-use basically went away. So that's interesting.

[michaelwoerister]

Those seem like very valuable pieces of information! Sounds like some kind of caching on intermediate graph nodes might be a speedup opportunity. But that's just off the top of my head, I haven't really investigated.

[nikomatsakis]

I have some thoughts about how to improve performance. I'll experiment
tomorrow (and/or we can discuss)

On Aug 3, 2016 9:30 PM, "Michael Woerister" notifications@github.com
wrote:

Those seem like very valuable pieces of information! Sounds like some kind
of caching on intermediate graph nodes might be a speedup opportunity. But
that's just off the top of my head, I haven't really investigated.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#35232 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAJeZmXINxKcVOjHsyEvRfu7arbEj8lzks5qcUDPgaJpZM4JbLQQ
.

[nikomatsakis]

My latest version runs in 2.5s -- but it still spoils the unit tests. I have to fix that next, as we talked about @michaelwoerister.

[nikomatsakis]

(Still seems like we could go faster...)

[nikomatsakis]

OK, got it down to 1.8s.