Github resolver is slow

[spmason]

I'm seeing much slowness in volo, and through liberal use of console.time I've narrowed it down to the github resolver:

github resolve jrburke/requirejs/2.1.6: 546ms
github resolve jquery/jquery/1.7.2: 1479ms

Any ideas on how to speed this up?

[jrburke]

It does the following right now for a github ID like jrburke/requirejs/2.1.6:

• API call to get list of tags for the repo, since some repos use 'v2.1.6' tag names instead of just plain '2.1.6', so volo inspects the tags to find out if it should use a 'v' prefix.
• Looks for a package.json with a "volo" section, in case the add target has dependencies
  • Fetches versionTag's package.json from github.
  • If that does not exist or no volo info in it, fetch master's package.json.
  • If still no volo info, check the volojs/repos override repo for an explicit package.json override.
• If there is a volo.url property, confirm its target exists via a HEAD request.

In your examples, the requirejs one is quicker because it has a package.json with a volo property for the given tag, where jquery does not. So, in general, things will get quicker if more projects add a "volo" property to their package.json. In the meantime, the slowness is a tradeoff for not requiring all projects to buy into volo right away, but still allow them to be usable.

I am open to trying other things though. In particular, I can see perhaps discarding the master package.json check and just rely on the tagged version or the overrides directory. Will need to think about it a bit more, going to move this to 3.0.1 which has some other tickets for possible speed ups.

[spmason]

What about cloning the actual gh repo locally and doing the checks on that? It may be a bigger one-time hit but then subsequent actions would be a lot faster (including getting the actual code when necessary). You would also get implicit 'caching' for free..

Failing that, some other kind of local cache for api responses maybe?

[jrburke]

A local cache is planned, tracking in #114. No plans to do git clones though.