Python integration

[albertdb]

As a devon IDE user, I want to have Python available in the devon IDE so that I and my team can easily ramp up our development environment and we are using the same and correct versions of the tool.

Acceptance criteria:

• Mirror is added.
• There is a new commandlet that allows to install and launch Python from the CLI
• The new commandlet is a wrapper for Python CLI
• Python (Version 3.x) can be installed automatically (during setup via settings) as well as via the commandlet call - only for windows
• During the installation process I get information about the current status of the installation

17.05:

• Python (Version 3.x) can be installed automatically (during setup via settings) as well as via the commandlet call - open for Windows & Linux
• The new commandlet is documented on the website
• Python can be installed and used together with any of the devon IDE standard tools without negative interference (tested for windows and linux)
• Python is integrated in a way that complies with the multi-tenancy of the devon IDE - Windows and Linux
• PIP (Package manager) is installed - Linux & Windows

14.06:

• Python (Version 3.x) can be installed automatically (during setup via settings) as well as via the commandlet call - open for MAC
• Review for windows and linux commandlet documentation
• Python can be installed and used together with any of the devon IDE standard tools without negative interference (must be tested for MAC)
• Python is integrated in a way that complies with the multi-tenancy of the devon IDE - MAC
• Add licence documentation
• PIP (Package manager) is installed - MAC

[maybeec]

please make sure, that pip is also installed! I made it once for python integration into devonfw ide and there is some script which needs to be executed and some path adaptations to make python portable work properly. https://stackoverflow.com/questions/23888109/portable-python-cant-install-pip

[maybeec]

@albertdb, just have seen the requirement is coming from hangar https://github.com/devonfw/hangar/blob/f651d23d53f74c8c9904da82313a7a878aab2f92/scripts/accounts/aws/custom-policies-check.py

is there a reason, why there is just one python file? Couldn't this be implemented in bash?

[albertdb]

@albertdb, just have seen the requirement is coming from hangar https://github.com/devonfw/hangar/blob/f651d23d53f74c8c9904da82313a7a878aab2f92/scripts/accounts/aws/custom-policies-check.py

is there a reason, why there is just one python file? Couldn't this be implemented in bash?

The usage of python within Hangar is broader, but it is embedded in shell scripts (which is a good thing from Python, you can run one-liners easily).

The first (non-working) version of that algorithm was is Bash and it was unreadable.

[Bene90]

HI @BizarreBits
@alfeilex already handled the multitendency topic within the Azure CLI integration. Maybe he can help here.

[hohwille]

@albertdb did you observe the things happening here? Could we get in touch? In case you are a "python expert" you could give us some assistance for small design decisions.
Actually Python is a disaster as its installation process is simply not working for all platforms and e.g. on MacOs it is just broken.
There are no binary distributions for MacOS and also no portable binaries for linux. Further, Python does not follow cross-platform standards (e.g. POSIX) and uses bin folder on Linux / Mac but Scripts folder on Windows.
We have already gone through all this and learned how Python seems to work with a lot of sweat and tears.
Almost all issues could be resolved but we currently have some design decisions to take and choose between option A and B but do not know if maybe option A will cause future bugs or problems for later extension of devonfw-ide with other python based software while B is the perfect way to go (or vice versa).
If you would have time for a short meeting with me this would be awesome.

[hohwille]

So actually the entire idea of devonfw-ide is following:

1. automate the installation of tools on any machine with any OS and any architecture (mainly windows x86, mac x86 + hopefully A1/2, linux mainstream on x68).
2. provide a local sandbox installation of all tools in software/«tool» subfolder where the binary executables should exist in software/«tool»/bin/ folder.
3. Every tool should have its own software/«tool» folder where it is actualyl installed.

We managed to make this work for java, mvn (maven), gradle, node, npm, yarn, ng (angular CLI), docker, kubectl (kubernetes), eclipse, vscode, intellij, dotnet, jasypt, ionic, sonar (sonarqube), graalvm, aws, helm, terraform.

Why on earth can we get this working for python.

• Python on windows puts the binaries directly into the toplevel installation without using a bin folder. This would also work but ...
• Pip on windows puts itself and other tools it installs into a Scripts folder. I could try to wrokaround this using --prefix option that then this puts the exeutables in a bin folder inside the installtion but ...
• I can not make python work if the binaries are moved to a bin subfolder
• If I separate pip (and each additional tool installed via pip) from the python installation folder each into their own software folder (what would actually be desired and a clean design) it is not working either as python then does not find according modules.

[maybeec]

I am not sure whether I got it 100% to work, but I was able to get a portable solution at least for cobigen quite some years ago with this hin: https://stackoverflow.com/a/55271031

It feels more like a hack but could work

[hohwille]

@maybeec thanks for your suggestion. That is what we are using on windows. Maybe our problems reside from that we are using a "hack" as you call it.

[hohwille]

FYI: There is an interesting discussion regarding this problem in a parallel thread in #841.