Reference a whole GITHUB repo in paket.dependencies

[forki]

At the moment we can do:

github forki/FsUnit FsUnit.fs

and link this file in a project by adding File:FsUnit.fs to the paket.references file. That's really really cool and I documented this at http://fsprojects.github.io/Paket/github_dependencies.html.

No I'd like to go one step further and just have

github forki/FsUnit

in paket.dependencies.

This should download all files from the repo. For this step I'd keep the paket.references untouched.

Ideas how to download the whole repo without using git?

/cc @agross @isaacabraham

[jwood803]

This may be a bit too much with what you're thinking, but perhaps it can download the archive and extract it?

[forki]

that sounds like a good idea. Anything else?

[isaacabraham]

Yeah I looked at this. The only two options I came up with was either: -

1. downloading the entire archive
2. Using Git to create a local repo and do it that way.
3. Using the API to get a list of files in the repo and then falling back to the HTTP endpoints to get the files seprately.

I'm somewhat wary of the API because of the rate limiting - I fell foul of it quite quickly when playing around with it.

Also - downloading the whole archive isn't a bad idea but might people want to do folder paths etc. in the future? Or perhaps leave that for a separate issue.

[jwood803]

For the folder paths, perhaps that can be an additional option to specify the location?

[forki]

maybe @haacked or @shiftkey can help us here.

• Is there a way to download the current state of a repo without hitting the API rate limit? Without a git client?
• Is there a way to whitelist Paket?

Thanks.

[shiftkey]

Is there a way to download the current state of a repo without hitting the API rate limit? Without a git client?

A quick brain dump:

• authenticated requests: you get 5000 requests per hour instead of 60. Of course no-one wants to do that, but it's an option.
• download zip: if you only care about master, that's a great way to get the stuff you need in one go.
• shallow clone: I had this same discussion with someone today and you could do a shallow clone of the repository rather than interacting with the API. However libgit2sharp doesn't support this and I've only heard whispers of NGit supporting it so I think you're stuck with using a git client here.

Is there a way to whitelist Paket?

I'm not sure about this, I'd suggest getting in touch with support@github.com.

[bartelink]

@forki I believe the bulk of the concerns in here apply across all file sources and the basic functionality can be expressed as a subsyntax of #154.

Hence I'd suggest either closing this to shorten the list or rebasing it to express just a set of concerns related ot multiple files or a syntax proposal (either in terms of #125 or #154 or something else)?

[forki]

No the problem here is not syntax in any of the paket files. It's how to get the remote files. That's more technical aspect.

[bartelink]

@forki my point is that there are 2 source syntax proposals and this touches on syntax and on the surface this issue as it is described looks like one but isnt

Download rate limiting and caching / fallback support are shared with single file downloads (and genning / caching .nupkg might be an element of an approach to that).

So I was just saying that it would be best for this isssue to have a clear purpose, be that or fallbck/cachingn or, most likely dealing with rate limiting re github. In which case I'd retitle this "Handle Github rate limiting gracefully for files and/or full-repo source references", not "Reference a whole GITHUB repo in paket.dependencies"

[blacktaxi]

@forki what are the drawbacks to using git to download the whole repo, except for need of git to be installed locally?

[blacktaxi]

@forki BTW, it is possible to download zip archive for a specific revision at https://github.com/{username}/{projectname}/archive/{sha}.zip.

[forki]

cool https://github.com/fsprojects/Paket/archive/80795aa62ef8efbb97993e18fc52dd40f7d837c7.zip works.

then it will be easy.