Support of Python3

[dclong]

It's great to see the Docker for Python added. I've tried it and it works well. However, it is for Python2 only currently. Can we add support of Python3?

[jgroom33]

go for it: https://github.com/theia-ide/theia-apps/blob/master/theia-python-docker/Dockerfile#L4

[dclong]

@jgroom33

I tried to replace Python 2 with Python 3 in the Dockerfile, but it doesn't work as expected. Python 3 syntax is not recognized. There were an error message on startup (cannot remember exactly, will report later).

[marcdumais-work]

@dclong Do I understand correctly that the idea situation, for Python, would be to support both version 2.x and 3.x in the same image(s)? It seems this might be tricky, since I think we would need to know which version of the LS we need to start, when opening a .py file.

Maybe we could have a preference for the name of the python executable Theia will attempt to use as a LS.

[marcdumais-work]

@simark I know you're quite good in Python, so you may have wisdom to share?

What would be a typical environment setup for a Python dev that has projects for both Python 2 and 3, and wants to be able to switch between the two? Let's say on Linux Ubuntu and Debian, that we use as bases in our images.

[simark]

It is typical in Python to work with virtual environments, which are self-contained Python installations that you create to work on a particular project. It's useful (and sometimes necessary) even if you use the same version of Python with multiple projects, since you might need different packages for each project. So in a way, it also acts kind of as the node_modules directory in node projects, except it's usually created out of tree. So let's say I work on two unrelated projects, one which uses Python 2 and the other Python 3, I would create two virtualenvs:

$ virtualenv -p /usr/bin/python2 python2-env
$ virtualenv -p /usr/bin/python3 python3-env

Then, you source the activate file in the virtual environment you want to use to have access to it in that shell.

Now, in Theia, I'm not sure what would the most convenient. The few times I tried IDEs for Python (mostly PyDev in Eclipse), it allowed you to list your existing virtualenvs, and let you select which one to use. We can probably do the same. In some way, it reminds me a bit of C/C++ build configurations, since you can have multiple "builds" with a single source.

Because each user creates their virtualenv wherever they want, I guess that the list of virtualenvs would most likely reside in the user's home config (since the workspace config is expected to be checked in). But I can imagine a team where everybody uses the same paths, so they would want to put the list in the workspace config. In that case, we probably want the list of virtualenvs in the workspace config to be appended to the list from the user's config, not overwrite/shadow it.

[dclong]

I know this might arouses debates. If it's hard to support both Python 2 and Python 3 in Theia, then we should support Python 3 rather than Python 2 in Theia. Official support of Python 2 will end in a couple of years anyway.

[jgroom33]

Or just have two images

[marcdumais-work]

Or just have two images

+1. I was just thinking the same thing. Maybe we could have a Dockerfile ARG, that can be set one the command-line when building the image, that controls which version of the Python packages gets installed ,in a given image. When releasing, we might build, eventually test, then and publish two versions of the image, each dedicated to a specific Python version. e.g.:

• theia-python2
• theia-python3

[svenefftinge]

I don't think there are users relying on python2 but Theia needs it.

https://github.com/nodejs/node-gyp#on-unix

So we don't need a python2 docker, but one that has both, defaults to python3 and make node-gyp use python2:

npm config set python /path/to/executable/python2.7

[marcdumais-work]

Maybe we could refactor the image so it's built in two steps, like theia-docker. Then the Theia build dependencies can be part of the first part, and runtime dependencies the second.

[marcdumais-work]

Theia items related to this issue:

• Python virtual env support
• Related discussion

[sr229]

@marcdumais-work @svenefftinge We can add Python 3.6 via APT PPA on Ubuntu (for the case of theia-full)

[jpambrun]

For those interested, this seems to work for me:

FROM node:8-stretch  as builder
RUN apt-get update \
    && apt-get install -y python \ 
    && apt-get clean && rm -rf /var/cache/apt/* && rm -rf /var/lib/apt/lists/* && rm -rf /tmp/*

WORKDIR /home/thei
ADD package.json ./package.json
RUN yarn --cache-folder ./ycache && rm -rf ./ycache
RUN yarn theia build

FROM node:8-stretch
WORKDIR /home/thei
COPY --from=builder /home/thei .

RUN apt-get update \
    && apt-get install -y python3 python3-dev python3-pip \ 
    && apt-get clean && rm -rf /var/cache/apt/* && rm -rf /var/lib/apt/lists/* && rm -rf /tmp/*

RUN rm /usr/bin/python && \
    ln -s /usr/bin/python3 /usr/bin/python && \
    ln -s /usr/bin/pip3 /usr/bin/pip
    
RUN pip install \
    python-language-server \
    flake8 \
    autopep8 \
    futures \
    configparser

EXPOSE 3000
ENV SHELL /bin/bash
ENTRYPOINT [ "yarn", "theia", "start", "/src", "--hostname=0.0.0.0" ]

[jgroom33]

it looks good enough to be a PR with a new directory: theia-python3-docker

suggested updates:
&& rm -rf ./ycache is not necessary in a staged build
spelling: WORKDIR /home/thei
ADD package.json ./package.json
I think the CI system uses $version: https://github.com/theia-ide/theia-apps/blob/master/theia-java-docker/Dockerfile#L26
CI also probably uses the GITHUB_TOKEN env var

[sr229]

I'll open a PR to do this

[dclong]

It seems to me that Gitpod (theia-based) supports Python3.

[Leletir]

@jpambrun we could also

RUN  update-alternatives --install /usr/bin/python python /usr/bin/python3 1

instead of deleting /usr/bin/python

and

RUN pip3 install \
    python-language-server \
    flake8 \
    autopep8 \
    futures \
    configparser

Because some os have stong dependencies with python 2 (rhel and yum for example)

Full dockerfile:

FROM node:8-stretch  as builder
RUN apt-get update \
    && apt-get install -y python \ 
    && apt-get clean && rm -rf /var/cache/apt/* && rm -rf /var/lib/apt/lists/* && rm -rf /tmp/*

WORKDIR /home/thei
ADD package.json ./package.json
RUN yarn --cache-folder ./ycache && rm -rf ./ycache
RUN yarn theia build

FROM node:8-stretch
WORKDIR /home/thei
COPY --from=builder /home/thei .

RUN apt-get update \
    && apt-get install -y python3 python3-dev python3-pip \ 
    && apt-get clean && rm -rf /var/cache/apt/* && rm -rf /var/lib/apt/lists/* && rm -rf /tmp/*

RUN  update-alternatives --install /usr/bin/python python /usr/bin/python3 1
    #ln -s /usr/bin/python3 /usr/bin/python && \
    #ln -s /usr/bin/pip3 /usr/bin/pip
    
RUN pip3 install \
    python-language-server \
    flake8 \
    autopep8 \
    futures \
    configparser \
    rope \
    pydocstyle \
    yapf

EXPOSE 3000
ENV SHELL /bin/bash
ENTRYPOINT [ "yarn", "theia", "start", "/src", "--hostname=0.0.0.0" ]

What do you think ?

[sr229]

I think we can use the Python 3.6 Docker image and just install node.js, that way we have a standard Theia with upstream Python

[jpambrun]

I am not using theia these days so I don't have a strong opinion. I just posted what worked for me hoping it could help others.

[stale]

This contribution has been automatically marked as stale due to inactivity, and it will be closed if no further activity occurs. Thank you for contributing to Theia!

[dclong]

Can we keep this issue alive?

[stale]

This contribution has been automatically marked as stale due to inactivity, and it will be closed if no further activity occurs. Thank you for contributing to Theia!