+github.com/mamba-org/mamba

[ddobrinskiy]

Intro

mamba is a FAST package manager for data science

While mainly aimed at python, it supports most scientific computing languages: julia, R, python, etc

It is now the default tool for most data scientists to manage their environments.

Mamba has now de-facto replaced conda, because:

• it mimics the API
• conda is very slow, while mamba is not

Motivation

As a Data Scientist I often have to take my code written locally with mamba as the pkg manager, and run it in CI/prod.

Getting the dependencies and docker images correct is a pain, especially since most mamba users are data scientists, not developers.

No dependencies

mamba's core idea is to be 100% self-contained distribution for python/julia/R/etc package management

So having external dependencies goes against how it is structured, we need to build it using only the .sh distribution files from mamba-forge

By packaging mamba with tea, I am hoping to simplify my workflows, and maybe even get wider adoption of tea within the Data Science community

p.s.

resolves #6

[jhheider]

If this is a python-venv, you might want to use python-venv.sh; see https://github.com/teaxyz/pantry.extra/blob/main/projects/youtube-dl.org/package.yml. In any case, it looks like we'll need python.org: ^3.something as a runtime dependency.

[ddobrinskiy]

@jhheider mamba's core idea is to be 100% self-contained distribution for python/julia/R/etc package management

Having external dependencies goes against how it is structured.

So I'll do more experiments on how to build it properly.

p.s. I also updated the PR description to make the purpose of this PR clearer

[ddobrinskiy]

Current status: builds and works locally

# add mamba and it's bin folder to shell
❯ ~/proj/pkgdev/pantry.extra/tea.out/github.com/mamba-org/mamba/v22.11.1.4/bin/mamba init zsh
modified      /home/david/.zshrc

==> For changes to take effect, close and re-open your current shell. <==

Added mamba to /home/david/.zshrc

❯ mamba --version
mamba 1.1.0
conda 22.11.1

❯ echo $CONDA_PREFIX
/home/david/proj/pkgdev/pantry.extra/tea.out/github.com/mamba-org/mamba/v22.11.1.4

❯ which python
/home/david/proj/pkgdev/pantry.extra/tea.out/github.com/mamba-org/mamba/v22.11.1.4/bin/python

[mxcl]

This is awesome! Thank you.

Use our fix-shebangs.ts script. See other package.ymls for usage examples.

[ddobrinskiy]

Current status:

• DONE:

  • passes CI
  • builds & tests ok locally
  • did some testing of tea-packaged mamba/python in action locally
  • move from curl to aria2c to simplify code and local testing (aria2c supports download caching, could not get it to work with curl)

• TBD

  • extensive testing of mamba/python locally (on linux)
  • test locally on macos m1
  • clean up code

@mxcl FYI

[ddobrinskiy]

UPD: this whole comment is no longer relevant, see #388 (comment)

@mxcl This PR is ready for review.

There is one thing that bothers me though:

After installing mamba, the user is expected to run mamba init "$(basename "${SHELL}")" to add it to their shell properly.

First, remember that mamba is not a binary - it is a python script.

So for this to work, it is very important that the version of python that mamba script finds is using exactly the python distribution was packaged with it. By default, this is solved by hard-coded shebangs that the installer autogenerates at installation.

However this breaks tea tests. So we did fix-shebangs.ts, but now mamba does not have hardcoded shebangs, and so when running it after a fresh install, it does not see the very specific python distribution from above.

Currently I fix it by modifying PATH (see below), and after that everything just works without a hitch.

But this bothers me, and should be somehow fixed in the future - I will experiment more after this PR is merged (I want to see how tea behaves when I'm installing mamba via running the magic)

❯ export PATH="$HOME/proj/pkgdev/pantry.extra/tea.out/github.com/mamba-org/mamba/v22.11/bin/python:$PATH"
❯ mamba init zsh

[ddobrinskiy]

Second thing that bothers me is that the code downloads installation .sh script at build time (different files depending on platform), and it does not depend on the distributable.url

So another thing to ideally fix in the future

[ddobrinskiy]

Upd: The correct way to init mamba for daily usage is the following:

tea +github.com/mamba-org/mamba sh          # activate a tea venv with mamba & conda-packaged-python
tea $ mamba init "$(basename "${SHELL}")"   # updates your .zshrc or .bashrc to make mamba & conda usable

Also added this to source code as a comment

[jhheider]

Does that produce a different outcome from just running mamba init $shell_name? The tea environment should be present during the invocation.

[ddobrinskiy]

Does that produce a different outcome from just running mamba init $shell_name? The tea environment should be present during the invocation.

@jhheider Yeah, this actually seems to work. I simplified the comment sections in the code.

p.s. the behavior below is exactly what I would want as a data scientist. Especially if I was less experienced with paths et cetera, I would want things to "just work"

❯ pwd
/home/david/proj/pkgdev/pantry.extra
❯ tea build github.com/mamba-org/mamba
❯ mamba init zsh # changes ~/.zshrc
❯ which python
python not found
❯ source ~/.zshrc
❯ which python
/home/david/proj/pkgdev/pantry.extra/tea.out/github.com/mamba-org/mamba/v22.11.1.4/bin/python

[ddobrinskiy]

If I understand tea cli correctly so was, this whole installation page with it's various instructions can be replaced by a tea one-liner:

sh <(curl tea.xyz) mamba init "$(basename "${SHELL}")"

[mxcl]

If I understand tea cli correctly so was, this whole installation page with it's various instructions can be replaced by a tea one-liner:

sh <(curl tea.xyz) mamba init "$(basename "${SHELL}")"

Awesome :) The dream is coming true.

[mxcl]

yeah it's on my “TODO” to figure out how we can communicate this sort of thing.

[mxcl]

Fabulous contribution.

[kendu3310]

nice