Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port to Python3 #2475

Closed
BryanQuigley opened this issue Apr 19, 2018 · 26 comments
Closed

Port to Python3 #2475

BryanQuigley opened this issue Apr 19, 2018 · 26 comments

Comments

@BryanQuigley
Copy link
Contributor

Python(2) is only supported until 2020, maybe a good time to make a plan to port to python3. I'm not sure if it's important to support both 2 and 3 at once. I noticed because boinc-client depends on python2 in Ubuntu 18.04 (and python2 is no longer installed by default)

My initial check indicates python is used quite a bit.
py/*
samples/vm_wrapper/
sched/
test/
tools/

Any obvious places to start porting?

@SETIguy
Copy link
Contributor

SETIguy commented Apr 19, 2018 via email

@davidpanderson
Copy link
Contributor

davidpanderson commented Apr 19, 2018 via email

@BryanQuigley
Copy link
Contributor Author

For starters, print statements are now functions.. (I only see the old style print statements), so at least the code I checked in py/* and test/* would fail to run at all. (There are many other changes I haven't checked)

It appears that the python2 dependency in boinc-client may be a Debian specific modification.. for /usr/bin/update-boinc-applinks. Investigating.. as I don't see any others in boinc-client upstream..

@Ferroin
Copy link

Ferroin commented Apr 20, 2018

The print statements becoming functions can be handled in a backwards compatible manner (supporting Python versions 2.6 and newer) by just converting all the print statements to functions and adding this:

from __future__ import print_function

to the top of each file (before any other python statements, but it can be after the docstring).

@BryanQuigley
Copy link
Contributor Author

Agreed, almost all of the changes can be done in a backward compatible way. The key thing is if it's worth that extra work for specific scripts.

@adamradocz
Copy link
Member

What can we lose if we drop the python 2? Is it worth the effort making the script?
Do you know any project which relies on python2?

@ChristianBeer
Copy link
Member

ChristianBeer commented Apr 21, 2018

The problem is with projects that use a Linux distribution that has no python3 or it is difficult to install python3. For example CentOS 6/7 which comes with python2 and installing python3 is a bit difficult and you can't set python3 as the system default. So we sould need to retain python2 support.

Please note that PR #2477 was opened to address this.

@SETIguy
Copy link
Contributor

SETIguy commented Apr 21, 2018 via email

@marius311
Copy link
Member

Just in case people aren't aware of it, but quite a lot of the things mentioned in this thread are automatically fixed by the "2to3" tool. I've found that running that then fixing a few remaining things by hand is usually a pretty painless path to upgrading. There's also the "six" package with aids in keeping 2/3 compatibility (which we definitely need to do).

Anyway, I very much support getting python3 working sooner rather than later, and am happy to offer whatever input I can.

@Ferroin
Copy link

Ferroin commented Apr 23, 2018

The biggest thing I can see that would be lost is support for certain old versions of RHEL and CentOS. I don't remember the exact versions where they finally included Python 3, but I do know at least a couple of other software projects have delayed full Python 3 support to keep support for these older platforms (see for example https://github.com/ansible/ansible, which is just now seeing the beginnings of proper Python 3 support).

@Ferroin
Copy link

Ferroin commented Apr 23, 2018

Also, do keep in mind that support past a certain point for older CentOS versions means having to deal with Python 2.6, and getting code to work properly there and on Python 3 gets rather complicated fast.

@SETIguy
Copy link
Contributor

SETIguy commented Apr 24, 2018 via email

@Ferroin
Copy link

Ferroin commented Apr 24, 2018

RHEL 6 is approximately 7 years old. That's old, period, regardless of whether or not it's an enterprise release. It's in the ELS support phase on top of that, which is their way of saying you need to upgrade (and that you should not be deploying new systems running it). Personally, I also consider RHEL 7 old despite it still being in the first production phase, but I'm also the type who hates the standard enterprise practice of only handling API/ABI breaks all at once every half decade.

I'm absolutely serious though that it is extremely complicated to support both Python 2.6 and Python 3.x with the same codebase once you get past truly trivial scripts. There are enough differences in the core language that it pretty severely restricts what you can actually do, and that's without having to deal with the library differences.

@SETIguy
Copy link
Contributor

SETIguy commented Apr 24, 2018 via email

@BryanQuigley
Copy link
Contributor Author

FYI, the boinc client python2 dependency for Debian/Ubuntu was an old script we don't think is necessary anymore - https://bugs.launchpad.net/ubuntu/+source/boinc/+bug/1765576. Sorry for the noise on that..

extremely complicated to support both Python 2.6 and Python 3.x
To add: Python 2.7 and Python 3.x support at the same time is much much easier.. So that really makes SL6 the biggest limitation.

@SETIguy What do the remaining SL6 servers specifically do (or what part of boinc python code is touched)? Any other reasons to upgrade them (performance, other program updates)?

@jessefowers
Copy link

I would be willing to help with this. Even with 2to3 its going to take a lot of manual labor. If/when the decision is made to drop support for 2 and port all python code to 3, what would that process entail? Would we have to port all the code in one big patch? What volume of code are we looking at here?

@davidpanderson
Copy link
Contributor

davidpanderson commented Apr 28, 2018 via email

@jessefowers
Copy link

jessefowers commented Apr 28, 2018

The goal here is to make the code work with both 2 and 3.
Straddling 2 and 3 sounds do-able.

python-future might be a good resource. It is both a python module that builds on 2to3 and a great website detailing how to write 2/3 compatible code.

From their website:

python-future is the missing compatibility layer between Python 2 and Python 3. It allows you to use a single, clean Python 3.x-compatible codebase to support both Python 2 and Python 3 with minimal overhead.

@jessefowers
Copy link

Right now I am assuming that the existing Python code runs fine on 2.x. If we use only use 2to3 we could break the code against 2.x. What is hopefully achievable is code that runs on both 2 and 3 without any dependence on a compatibility module. If the existing code requires changes that go beyond syntax adjustments to run on 3 then we need to consider adding a dependency on a compatibility module like future, if we want to continue to support 2.x.

I suggest we exhaust all of the 'easy' syntax only adjustments and see how far that gets us.

I have read through the contributors guide and would like to help with this. I have not contributed to this project before so please tell me the preferred way to proceed. I can fork the project and begin work adjusting syntax in the scripts and modules that @davidpanderson listed so that it works with 2 and 3, but it looks like some of that is already started with #2477. Please advise.

@jessefowers
Copy link

Hi again - I just wanted to check in on this issue. I would love assist with this issue if their is a real need. Please advise on what you would need from me. Thanks!

@AenBleidd
Copy link
Member

Could this be closed in favor of #3241?

CC: @TheAspens

@smoe
Copy link
Contributor

smoe commented Aug 27, 2019

With a bit of luck, my #3259 also closes (most of?) this.

@Delizald
Copy link
Contributor

Did #3259 fixed the Py2 and Py3 compatibility? I can see that it was merged

@AenBleidd
Copy link
Member

It would be nice to verify that there are no other incompatibilities

@davidpanderson
Copy link
Contributor

The code itself works with python 2 and 3.
The issue is the MySQL interface, and distutils.
We need python3 versions of these

@AenBleidd
Copy link
Member

There are two open issues (mentioned in the comment above): #5428 and #5429.
I see no reason to keep this one open since it's a general ticket, and two above a exact issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

No branches or pull requests