Accept Python 3 by default

[joaocgreis]

Checklist

• npm install && npm test passes
• commit message follows commit guidelines

Description of change

This depends on #1843, I will rebase when that PR lands. Only a971947 needs to be reviewed here.

This makes node-gyp accept Python 3 as compatible without needing EXPERIMENTAL_NODE_GYP_PYTHON3 in the environment. It also looks for python, python3, then python2 (in that order). Note that python is usually Python 2.

@nodejs/node-gyp please not that we cover little more than a basic hello world in our tests, there are probably parts of Gyp that are not covered and are not compatible with Python 3. We will have to be extra careful shipping the node-gyp version that includes this.

[joaocgreis]

CI: https://ci.nodejs.org/view/All/job/nodegyp-test-pull-request/152/

[cclauss]

Awesome work here!

Can you please add a Python 3 on macOS on Travis CI test like #1846 just to make sure that is passing?

[joaocgreis]

@cclauss I think #1846 is the proper place to do it, this PR might take a while to land. I'll rebase on top of that PR and adjust accordingly when it lands!

[joaocgreis]

This is blocked by #1846, we can't land this with OSX failing.

[rvagg]

seems good to me but I'm confused on whether the macos problems block this and #1843 or if they can land independently of that?

[cclauss]

We can land this with the macOS tests still running in allow_failures mode. Our README clearly states that Python 3 is not supported which aligns with Travis CI's intent for allow_failures

[allow_failures] lets you add in experimental and preparatory builds to test against versions or configurations that you are not ready to officially support.

[joaocgreis]

#1843 has landed. In there, node-gyp only uses Python 3 if EXPERIMENTAL_NODE_GYP_PYTHON3 is defined, so it should not break anything we currently have.

This PR removes that flag, making node-gyp accept Python 3. So, if a release is shipped with this PR, node-gyp will start using Python 3 in the users computers, and any issue we have will manifest there.

Currently, we know Python 3 is failing on osx, so shipping a release with this would break node-gyp on every osx machine that has Python 3 installed.

We know that our tests pass with Python 3 on Windows and Linux, but our tests don't have a good coverage of gyp. It's likely that there are modules out there that use parts of gyp that we don't test here, and those would be broken as well. GYP3 could help a lot with this.

So, we can't land this PR until all failures that we know of are fixed.

[sam-github]

nodejs/node#29246 bumped into this.

I'm not sure what the plan is. If I'm missing the conversation elsewhere, pls point me there! I don't want to fragment an existing conversation.

node's tests can't pass on a py3-only machine until npm has a copy of node-gyp that supports py3.

Does npm even get node-gyp from this repo? :-( Sorry, you can see how late to this "party" I'm arriving.

[bnoordhuis]

Does npm even get node-gyp from this repo?

Yes. :-)

[joaocgreis]

@sam-github my comment above is still a valid summary of the current situation. Going forward, I believe the options on the table are either to use GYP3 (#1845) or fix the osx issues (@cclauss might be working on that).

This PR is ready to be rebased and land landed when we believe Python 3 support to be ready. However, this PR does not change any Python code, it only removes the block to run on Python 3.

[joaocgreis]

Rebased to make Travis run again, but it's still red. Current status: #1844 (comment) and #1844 (comment)

[cclauss]

The macOS error looks related to #1854 so I will attack it...

Fixed in Decode stdout on Python 3 #1890

[cclauss]

Please rebase again to resolve conflict and ensure macOS now passes.

[cclauss]

Once this PR lands, does NODE_GYP_FORCE_PYTHON=python3 still have to be set to use Py3?

• On a machine that has both Python 2 and Python 3 installed?
• On a machine that only has Python 3 installed?

[joaocgreis]

Once this PR lands, does NODE_GYP_FORCE_PYTHON=python3 still have to be set to use Py3?

Doesn't "have" to, but I'd rather keep it in most (if not all) of the entries in Travis, since not having it was the reason we didn't know what Python was actually being run before. This is a good way to avoid surprises.

On a machine that has both Python 2 and Python 3 installed?
On a machine that only has Python 3 installed?

Without NODE_GYP_FORCE_PYTHON the code looks for python, then python3, then python2. Whatever exists first is what's used.

[joaocgreis]

Hit an issue because of isaacs/minizlib#9 - Travis kept 1.3.1 in cache. Should be fixed now, forced update in one run and Travis updated the cache.

CI: https://ci.nodejs.org/view/All/job/nodegyp-test-pull-request/155/ ✔️

Landed in 3d1c60a

[sam-github]

Does this change which version of python is chosen? I can't tell from the diff.

I.e

• with python2, python3, and python (2.7) in /usr/bin, which will get chosen?
• with python2, python3, and python (3.6) in /usr/bin, which will get chosen?

I don't have any concerns either way, but I think we should call out in the docs/changelog if the order of choice changed, right now it just says Python 3 is accepted, not under which conditions is preferred, if ever.

[cclauss]

Without NODE_GYP_FORCE_PYTHON the code looks for python, then python3, then python2.
Whatever exists first is what's used.

I take that to mean python will be used.

[sam-github]

This makes node-gyp accept Python 3 and look for it before looking for Python 2, making Python 3 the default.

@joaocgreis Can I edit this description? I think that isn't quite accurate, it should be

This makes node-gyp accept Python 3 as compatible without needing EXPERIMENTAL_NODE_GYP_PYTHON3 in the environment. It also looks for python, python3, then python2 (in that order). Note that python is usually Python 2.

Personally, I suggest that this PR can and should land on 5.x, with the minor tweek of moving python3 to after python2, so that users will Python 2 if they have it.

cf #1892 (comment) and previous discussion