package management system

[shabbychef]

This is probably v 2.0 or beyond, but a good system for contributing/installing/maintaining packages (or modules, libraries, whatever they are called in Julia), like that in R via CRAN, would allow developers to contribute to the project without having to touch the core repo. Making the package system frictionless goes a long way towards growing the community. (Compare R to Matlab in this regard, where the Mathworks has a vested interest in selling toolboxes, so community packages take a back seat.)

[StefanKarpinski]

Yes, absolutely. We're not quite at that point yet, but when we are, we most definitely need this. I would actually like for this to be git-based. Not because git is the end-all-be-all, but because it's what we use, it's popular, and it allows much easier management of different versions.

[ViralBShah]

We need to decide if the package management is source-based or binary-based or both. Source based is perhaps easiest, but requires the user to have compilers installed, and you can base it on git.

Also, I think it would be great if pure julia packages can simply be loaded using a URL and cached. For example:
load("http://foo.com/bar.j")

The bar package can then be cached locally, so that you don't need to load it remotely every single time.

[ViralBShah]

It would be nice to have remote loading of modules as well. We can even have rules, so that the package provider can provide binary versions as well, but if a certain binary package is not available, the source gets pulled and compiled.

Also, we'd need good dependency management, like Debian's APT.

[o-jasper]

I think it might be a good idea to allow users to say what their dependencies are, and maybe then have checked. If methods are extended in other modules than they are created would probably make this impossible though.

Another idea is to allow users to make modules point to where tests/guied versions/documentation is. If namespacing/modules are hierarchical, some-module/test could be a way. (just ftr Made a mention here)

Also probably a good idea to (optionally) add a version of julia. (somewhat stupidly)i failed at running some python programs because i didnt realize i needed python2.

[adamramadhan]

but why in git? why don't make something new like julia? i love git, i use git. but i think a good and strong language should be independent.

[StefanKarpinski]

@adamramadhan: git already solves the problem we want to solve (distributed version control) and using it allows us to leverage amazing tools like github to accelerate the pace of development. If a Julia package is a git repo by definition, then people are forced to use the de facto standard distributed version control system for Julia packages. Submitting improvements and fixes to packages becomes trivial and friction-free. I'm also certainly not about to invent my own version control system when a perfectly good one is already does everything I want.

[johnmyleswhite]

I'm totally on board with the Git strategy for managing packages, but I think the community will need to force people to use tags to create a useful version system. It's really helpful to be able to tell people to use a version of your package that's 0.5-7 or higher. Having to use commit hashes seems a lot uglier.

[StefanKarpinski]

Indeed. We have full support of semantic version numbers — see base/version.jl. This is how Julia packages will be numbered and tagged.

[edwindj]

Another source of inspiration imho is npm (node.js package manager):

It allows for:

• per package dependencies (which can be a pain in R, having installed two packages each depending on a different version of a third package)
• Semantic versioning
• directory-based, local, global, tar,gz-based and git-based repositories (http://npmjs.org/doc/developers.html)

and of course a lot more.

[adamramadhan]

yes npm for node.js is quite good. i know git have all the features, but still for me, (a beginner programmer) it would be very easy to deal with. npm install x and make a folder that can we include seamlessly. its just so easy.

[adamramadhan]

thanks for the 1.0 milestone :)

[dcampbell24]

I would also suggest looking at how Go handles this. They have a simple yet elegant system that allows for packages to be pulled from several different version control systems and uses urls as a way of avoiding namespace clashes. The way they handle dependencies is designed to ensure fast compilation.

I also like how they determine what is and is not exported from packages, but using capitalization does not look like it will work with Julia's current conventions. Using short names for packages and never eliding the package name also seems like a really good design choice. This makes maintaining code much easier as it is easy to figure out where a function or variable came from and avoids problems that occur from having name clashes between variable names across packages.

http://golang.org/ref/spec#Packages

[adambom]

I've worked a lot with npm and a little bit with gems and they are both pretty good. Wouldn't it make sense to fork one of those systems and just use that, rather than starting from scratch?

[pao]

Little late for that. https://groups.google.com/d/topic/julia-dev/zgXcBlxmng4/discussion

[ViralBShah]

Closing this since the package management system is already in place.