N-API testing Windows failures

[mhdawson]

We've had failures across a number of machines over the last few days which look most like machine problems. For example: https://ci.nodejs.org/view/x%20-%20Abi%20stable%20module%20API/job/node-test-node-addon-api-new/1272/

I can't find those machines in the inventory, so not sure how to figure out. Is there a PR I did not find to add them?

Looking at the tags we were using win10 and win2012r2-vs2015 but those are not used by the current testing.

So I'm also not sure what we should be running the N-API tests on.

I tried:

win-vs2015
win-vs2017
win-vs2019
win-vcbt2015.

The win-vcbt2015 ones seem to fail consistently so I removed that.

@joaocgreis can you let me know what you think we should be testing on.

The test pulls down the release binaries for 8,x, 10.x, 12.x 13.x, master and runs the node-addon-api tests.

[richardlau]

#1996 is tracking changes to the Windows CI machines being made by @joaocgreis.

[mhdawson]

Also wondering if we expect vs2019 to work with 8.x? https://ci.nodejs.org/job/node-test-node-addon-api-new/nodes=win-vs2019/1282/console

[richardlau]

Also wondering if we expect vs2019 to work with 8.x? https://ci.nodejs.org/job/node-test-node-addon-api-new/nodes=win-vs2019/1282/console

Not out of the box. VS2019 support was added in node-gyp 5 which isn't in the version of npm that ships with the most recent Node.js 8.x.

[mhdawson]

I'm thinking the version selector script should be filtering that out, but will need to check if there is something special to the N-API job that prevents that.

[richardlau]

I'm thinking the version selector script should be filtering that out, but will need to check if there is something special to the N-API job that prevents that.

Agreed. If you look in https://ci.nodejs.org/job/node-test-node-addon-api-new/1282/console you can see win-vs2017 being filtered out by

build/jenkins/scripts/VersionSelectorScript.groovy

Line 44 in 39cacfe

[ /vs2017/, anyType, lt(10) ],

so it would seem to make sense to do the same for win-vs2019. I've raised #2085.

Node.js major version: 8
Node.js version: null
Skipping smartos18-64 for Node.js 8
Skipping centos7-ppcle for Node.js 8
Skipping smartos17-64 for Node.js 8
Skipping ubuntu1804-64 for Node.js 8
Skipping win-vs2017 for Node.js 8

[mhdawson]

@richardlau thanks. I went to look at the selector script saw 2019 was missing and was happy to see your PR.

[joaocgreis]

This is the version of VersionSelectorScript I have at the moment: https://github.com/JaneaSystems/build/blob/joaocgreis-JBC-windows-updates/jenkins/scripts/VersionSelectorScript.groovy - I plan to PR this soon when I have all the details in place.

Since the N-API job uses the test build type, as the labels are now it would only run in VS versions that can build Node, excluding a few that can build add-ons. Is this acceptable? I'm structuring the labels to have the VS version that is used to build add-ons before -COMPILED_BY- and the version that is used to build Node itself after (as in win10-vs2019-COMPILED_BY-vs2019). So, I can add a label like win10-vs2019-COMPILED_BY-ANY to be used by this job. Would this be better?

[richardlau]

@mhdawson I think this has been resolved now?

[mhdawson]

@richardlau, yup, closing