Platform requirements for Node.js 14

[richardlau]

Let's try to sort out the platform requirements for Node.js 14 ahead of the release in April. Node.js 14 is scheduled to be released in April 2020 and End-of-Life in April 2023.

I'll make a start by filling in what we're currently doing for master/Node.js 13. Please help to fill in the table below, discuss questions raised and raise another other questions for discussion. We primarily need to identify what needs changes to build infra (e.g. updating the macOS release machines) so we can try to avoid a last minute rush.

cc @nodejs/platform-aix @nodejs/platform-arm @nodejs/platform-macos @nodejs/platform-ppc @nodejs/platform-s390 @nodejs/platform-smartos @nodejs/platform-windows @nodejs/releasers @nodejs/v8-update

Based on https://github.com/nodejs/node/blob/master/BUILDING.md, https://github.com/nodejs/build/blob/master/jenkins/scripts/VersionSelectorScript.groovy we have:

What we build and test on

Only current Tier 1 and Tier 2 platforms shown.

os-arch	release machine	tested on	supported on
aix-ppc64	AIX 7.1 TL05 on PPC64BE with GCC 6	AIX 7.1 TL05	AIX >= 7.2 TL02
darwin-x64	macOS 10.11, Xcode Command Line Tools 10 with -mmacosx-version-min=10.10 macOS 15 ?	macOS 10.10 + 10.11	macOS >= 10.11
linux-arm64	CentOS 7 with devtoolset-6 / GCC 6		kernel >= 4.5, glibc >= 2.17
linux-armv7l	Cross-compiled on Ubuntu 16.04 x64 with custom GCC toolchain		kernel >= 4.14, glibc >= 2.24
linux-ppc64le	CentOS 7 with devtoolset-6 / GCC 6		>=Power 8, kernel >= 3.10.0, glibc >= 2.17
linux-s390x	RHEL 7 with devtoolset-6 / GCC 6	RHEL 7 with devtoolset-6 / GCC 6	kernel >= 3.10.0, glibc >= 2.17
linux-x64	CentOS 7 with devtoolset-6 / GCC 6		kernel >= 3.10, glibc >= 2.17
sunos-x64	SmartOS 18 with GCC 7		>= 18
win-x64 and win-x86	Windows 2012 R2 (x64) with Visual Studio 2017		>= Windows 7/2008 R2/2012 R2 Windows 8/2012 R2?

Actions

• Move cross compiling machine for linux-armv7l off Ubuntu 16.04: jenkins,ansible: arm cross-compilers as docker containers #2290
• Update BUILDING.md in core: Platform updates for Node.js 14 node#32812
• devtoolset-8 on Centos/RHEL Linux on Release machines - Use devtoolset-8 on Centos/RHEL for Node.js 14+ #2242
• Drop release binaries for smartos (but keep testing in CI) Platform requirements for Node.js 14 #2168 (comment) jenkins: drop smartos builds from release for 14+ #2283
• macOS release machine change for notarization (probably need to change the release machine for existing versions of Node.js too) - jenkins & docs: macOS 10.15 #2199
• bump minimum deployment target for macOS to 10.13 - build: update macos deployment target to 10.13 for 14.x node#32454
• Windows 7/2008 R2 are no longer supported so we should remove those from BUILDING.md - win: block running on EOL Windows versions node#31954

No changes

• Python 3 (i.e. are we ready to drop support for the out of support Python 2?) no change from current

[richardlau]

"tested on column" needs filling in but I wanted to submit this before I leave the office today.

[sam-github]

Confirmed test and release machines are same OS level for:

• aix 7.1
• centos7-ppc64
• rhel7-s390x

@richardlau do you know why we officially support AIX7.2+, but we build on AIX7.1?

I don't think we should drop Python 2 support. python3.x executables are preferred over python2.7, over python, but I don't think actually not working with Python 2 is a good idea yet, support is not onerous, complaint from users could be intense. Maybe for 15.x or 16.x.

• Replace ubuntu 16.04

We already have a full set of 18.04 OS levels in CI, do you mean remove Ubuntu 16.04?

It looks like 16.04 is LTS, supported to 2025, so no need to drop it for EOL reasons AFAICT.

On the other hand, all maintenance has a cost, I don't recall us finding issues on those machines that weren't found elsewhere. Does anyone else?

I tend towards the "we should have a compelling argument to have something in CI" not "its already there, so we need to have a compelling argument to remove it".

test-digitalocean-ubuntu1804-x64-1
test-digitalocean-ubuntu1804_docker-x64-1
test-digitalocean-ubuntu1804_docker-x64-2
test-joyent-ubuntu1804-x64-1
test-joyent-ubuntu1804_docker-x64-1
test-scaleway-ubuntu1804-armv7l-1
test-scaleway-ubuntu1804-armv7l-2
test-scaleway-ubuntu1804-armv7l-3
test-softlayer-ubuntu1804_docker-x64-1

[richardlau]

@richardlau do you know why we officially support AIX7.2+, but we build on AIX7.1?

We build on AIX 7.1 because (a) we don't have many AIX release machines (is it just the one?) and (b) Node.js 10 is supported on AIX 7.1. At the time we wrote the supported AIX versions we were still releasing and testing on AIX 6.1(!) due to lack of systems.

* Replace ubuntu 16.04

We already have a full set of 18.04 OS levels in CI, do you mean remove Ubuntu 16.04?

It looks like 16.04 is LTS, supported to 2025, so no need to drop it for EOL reasons AFAICT.

Sorry, misread the support dates. I meant the release machine for linux-armv7l but if Ubuntu 16.04 is supported for the duration of Node.js 14 let's keep it as-is.

[nschonni]

Windows 8.1 hits EOL on January 2023 https://support.microsoft.com/en-us/help/13853/windows-lifecycle-fact-sheet so it doesn't make it to the end of the LTS range. Plain Windows 8 is already EOL. Looks like 2012 is good till October 2023 https://support.microsoft.com/en-us/lifecycle/search?alpha=windows%20server%202012

[AshCripps]

Notorization requires at least xcode 10 and macOS 10.13.6

With regards to the operating system level Apple don't officaly state anywhere how long they support the os for, but I think going for latest (catalina 10.15) is the best idea to help prevent us running on old software like we do now.

We could also bump up the deployment target as is set to 10.11 (I think) atm, and using the site refack used last time the market share for 10.12 and below is fairly low -

[rvagg]

General question: are we are of any new compiler (C++) constraints that may be heading our way via V8? Are we going to struggle at any point in 14's lifetime with GCC 6? @targos you seem to be on the ball with this kind of stuff, any ideas?

Ubuntu

Ubuntu tends to be on the easier end of the support spectrum of Linux for us, mainly because of ubuntu-toolchain-r. But, 16.04 is only publicly supported till early next year. The 2024 timeframe is for ESM, which is a paid extra. I tried once to get Canonical to sponsor the project with some free ESM licenses but that never eventuated. So we probably shouldn't lock ourselves into supporting an OS that isn't getting security updates. +1 to dropping for simplication.

Re the 18.04 machines you listed @sam-github, the ones with "docker" in their name are docker hosts and don't run tests bare. So we only have 2 x64 machines in rotation and even the armv7l ones are running Debian images! 18.04 is a good default for infra-type machines for now. That should switch to 20.04 soon.

macOS

• Notarization places strict requirements on our release machines as noted
• Supporting older versions ties up resources
• We experience very few (I can remember one) issues where older versions have needed special treatment. Xcode seems to do a good job with compatibility.

I'd be in favour of raising our minimum tested to 10.13 (High Sierra) and attempting to get all our releases onto 10.15 (Catalina).

There's also -mmacosx-version-min we need to consider, common.gypi:

'MACOSX_DEPLOYMENT_TARGET': '10.10',      # -mmacosx-version-min=10.10

Let's bump that again. I'd be fine with doing it to 10.13 as well.

Linux (x64)

I'm in favour of bumping our release machines to devtoolset-8 for 14+. This is probably complicated for non-x64 but we're not raising compiler minimums so they should be fine on devtoolset-6 as they are now. But if we can take advantage of a newer compiler toolchain while maintaining our current libc compatibility then we should probably take it.

[targos]

Chromium doesn't support C++17 yet, so I think GCC 6 is still OK.

[joaocgreis]

Updated BUILDING.md for Windows in nodejs/node#31954.

The Windows release machines we have currently are good for v14.

[mhdawson]

I'm good with running some of the releases on higher versions of dev-tool set provided we still have some x86 testing using gcc6.

@sam-github, @AshCripps can you look to confirm devtoolset-8 is available for power and s390 (I'm thinking it should be). If so we should consider being consistent across the platforms that we use devtoolset on.

[sam-github]

It was available last time we looked. Here is a copy of the comment by @miladfarca

---

dev toolset does exist for for both ppc64-le and s390x.

RHEL ppc64le

All the the devtoolset-8 (gcc-8) packages for Centos are located here:
https://cbs.centos.org/repos/sclo7-devtoolset-8-rh-candidate/ppc64le/os/Packages/
They are fully compatible with Redhat. I installed and tested the above packages on a fyre machine.
To install, download all the packages using wget, install all using yum, enable using scl:

wget --recursive --no-parent <URL>
yum install *.rpm
scl enable devtoolset-8 bash 
gcc -v 

RHEL s390x

IBM has an internal account which can be used to login to the Redhat portal and view/download redhat packages for internal use only: https://ftp3.linux.ibm.com/redhat_updates.php

I located the same packages from the link above on Redhat's portal, which includes s390x architecture:

Downloading above packages outside IBM requires a paid RHEL subscription. We can then use yum to download and install them.

I have emailed Marist staff about it but have not received any replies yet, I will send them another reminder this week. To check the status myself, I logged into rhel72-s390x-2 and verified the machine isn't registered with redhat yet.

In both cases devtoolset-6, devtoolset-7 and devtoolset-8 can be used for installing gcc 6, 7 and 8 if needed.

TL;DR dev toolset is available for both architectures. ppc64-le doesn't require a Redhat subscription since Centos packages can be used instead. s390x requires a subscription.

---

Additional note: after the above was written, we got the machines donated to CI registered so they have authorization to use the subscription-only devtoolset for s390x.

[mhdawson]

@sam-github thanks for the update. Sounds like we can be consistent across the machines we build with devtoolset. We'll just want to keep enough platforms testing on gcc6 so that we have some coverage across platforms for the minimum level.

[rvagg]

gcc6 for Node >=14 would be a great candidate for a containerised build, it'd be awesome if I could get that .ci.yml stuff working because this is exactly the kind of edge case we want to be able to cover.

[rvagg]

Continuing from a discussion about ARM64 in #2242

We have CentOS 7 ARM64 machines which can get devtoolset-8 so we can switch versions for different release lines as per that PR, so 👍. But we also run Ubuntu 16.04 machines for ARM64 and I believe we've agreed that we're dropping 16.04 from the supported list for Node 14 (it'll likely run just fine thanks to our CentOS 7 + devtoolset binaries, but it's EOL not long into 14's lifetime so we don't want to be tied to it even for testing).

We could go ask packet.net for more hardware to run 18.04 machines as well, but I think at this stage I'd rather just ditch 16.04 ARM64 entirely, from all release lines and reimage those machines with 18.04 and enable them across all release lines. It'll mean a drop in coverage for older release lines, but we still have CentOS 7 which has an ancient libc and kernel at this stage, and most Ubuntu LTS users are already looking at transitioning, either to 18.04, or to 20.04 from the end of this month.

Is anyone uncomfortable with that drop in coverage?

[rvagg]

os-arch	release machine	tested on	supported on
linux-armv7l	Cross-compiled on Ubuntu 16.04 x64 with custom GCC toolchain		kernel >= 4.14, glibc >= 2.24

What do we do about this? It's probably time to bump our cross compiler machines but I can't say for sure that even with the same cross toolchains that the binaries will be stable for existing release lines. Do we then have to be doubling our cross-compile capacity to have 16.04 and 18.04 cross compilers?

I guess I or someone should investigate whether 18.04 cross binaries are going to be stable so that we can maybe upgrade for all release lines. Ugh.

[cjihrig]

We never did discuss smartos, 19.4.0 is available now and we're still chugging along on 18.

I was able to get feedback from some SmartOS people at Joyent. Upgrading to 19.4.0 in the CI would be good. However, it might make sense to stop making official releases for SmartOS. The reason is that the builds are largely unusable without bundling all of the libraries used to build the binary.

It would be great if we could keep SmartOS in the CI to prevent regressions, and instead of creating a release binary, just link from https://nodejs.org/en/download/ to https://metrics.nodejs.org/en/download/package-manager/#smartos-and-illumos for SmartOS.

[rvagg]

@cjihrig sounds reasonable, thanks for getting that feedback for us.

+1 to getting 19 into CI and +1 removing sunos binaries from >=14.

[mhdawson]

@cjihrig is the feedback that we can drop all of the SmartOS releases or stop producing them as of 14.x ?

[cjihrig]

The discussion was focused on 14.x, but the issue of unusable binaries goes back to earlier release lines. I think it would be fine to stop producing them for the older release lines as well, but as a project, is that a change we want to make in the middle of a release line?

[richardlau]

The discussion was focused on 14.x, but the issue of unusable binaries goes back to earlier release lines. I think it would be fine to stop producing them for the older release lines as well, but as a project, is that a change we want to make in the middle of a release line?

I'd be very hesitant to drop binaries for a platform in the middle of a release line. I've no idea if it would negatively affect things like nvm (cc @ljharb).

[sam-github]

+1 to dropping release builds for sunos/smartos in 14.x

+1 to dropping ubuntu16.04-arm64

I think we should be very hesitant to drop binaries for Smartos for old release lines, but I don't think we should keep producing artifacts that no one uses just because its our policy to do so. Are the download stats available somewhere I can find? It'd be interesting to know if the binaries are being downloaded.

We could drop them from a 13.x release and see if anyone complains, and if no one does, repeat that for 12.x, 10.x.

[richardlau]

There's https://nodejs.org/metrics but I'm not sure that's accurate since late last year when we put the downloads behind Cloudflare (the very obvious drop off in the graphs).

[ljharb]

Please only drop a platform in a major bump, so i can cleanly update nvm to stop checking after that.

nvm has a hook built in specifically for smart os, to be able to do the patches required. I’ll have to check and see if that applies to the binaries, or only to building.

[rvagg]

Can't (easily) get GCC 4.9 on Ubuntu 18.04, we need the multilib libraries installed to make the cross-compiler work for Node 10. So it's not going to be as simple as switching over the cross compilers to 18.04 hosts even if we could maintain compatibility. We might be forced to maintain multiple cross compilers unfortunately.

An interesting alternative might be to stick this into the docker hosts as an image. Then we could have 4 cross compilers of each type waiting to be used. They just compile, mostly from ccache, and don't run tests, so aren't a massive burden most of the time. It doesn't solve ci-release (we don't use docker hosts in the same way there), but if we switched the 2 ci cross compilers over to docker and repurpose one of them to a release cross compiler then we'd still be down one machine. I might explore that option.

[richardlau]

#2290 has landed which was the last action item left so this can now be closed. Thanks everyone.