Skip to content
This repository has been archived by the owner on Aug 26, 2022. It is now read-only.

New Relic's native_metrics fails to build after Python 3 upgrade #6195

Closed
callahad opened this issue Dec 4, 2019 · 1 comment
Closed

New Relic's native_metrics fails to build after Python 3 upgrade #6195

callahad opened this issue Dec 4, 2019 · 1 comment
Labels
status: needs triage Status: Untriaged

Comments

@callahad
Copy link
Contributor

callahad commented Dec 4, 2019

Summary

New Relic's native_metrics package fails to build.

This is because our version of Node (10.16.3) ships a version of node-gyp which only works with Python 2. Updating to Node 10.17.0 (Changelog) gets us a better error message, but still fails.

Version 6.x of node-gyp supports Python 3, but the most recent npm release still depends on node-gyp 5.x. (npm/cli#287). It's possible to update npm's bundled node-gyp, and users generally seem happy with that workaround (nodejs/node-gyp#1917). However, I still see quite a few warnings

Steps To Reproduce (STR)

  • make build-base VERSION=latest

Actual behavior

Node 10.16.3:

gyp ERR! stack Error: Command failed: /usr/local/bin/python -c import sys; print "%s.%s.%s" % sys.version_info[:3];
gyp ERR! stack   File "<string>", line 1
gyp ERR! stack     import sys; print "%s.%s.%s" % sys.version_info[:3];
gyp ERR! stack                       ^
gyp ERR! stack SyntaxError: invalid syntax
Full logs 
Step 14/20 : RUN ln -s /app/package.json /tools     && ln -s /app/package-lock.json /tools     && npm ci
 ---> Running in 077bade0b542

> @newrelic/native-metrics@4.1.0 install /tools/node_modules/@newrelic/native-metrics
> node ./lib/pre-build.js install native_metrics

============================================================================
Attempting install in native-metrics module. Please note that this is an
OPTIONAL dependency, and any resultant errors in this process will not
affect the general performance of the New Relic agent, but event loop and
garbage collection metrics will not be collected.
============================================================================

> /usr/local/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js clean configure
gyp info it worked if it ends with ok
gyp info using node-gyp@3.8.0
gyp info using node@10.16.3 | linux | x64
gyp ERR! configure error 
gyp ERR! stack Error: Command failed: /usr/local/bin/python -c import sys; print "%s.%s.%s" % sys.version_info[:3];
gyp ERR! stack   File "<string>", line 1
gyp ERR! stack     import sys; print "%s.%s.%s" % sys.version_info[:3];
gyp ERR! stack                       ^
gyp ERR! stack SyntaxError: invalid syntax
gyp ERR! stack 
gyp ERR! stack     at ChildProcess.exithandler (child_process.js:294:12)
gyp ERR! stack     at ChildProcess.emit (events.js:198:13)
gyp ERR! stack     at maybeClose (internal/child_process.js:982:16)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:259:5)
gyp ERR! System Linux 4.9.184-linuxkit
gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "clean" "configure"
gyp ERR! cwd /tools/node_modules/@newrelic/native-metrics
gyp ERR! node -v v10.16.3
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok 
install successful: _newrelic_native_metrics-4_1_0-native_metrics-64-linux-x64

Node 10.17.0

gyp ERR! find Python checking if "python" can be used
gyp ERR! find Python - executable path is "/usr/local/bin/python"
gyp ERR! find Python - version is "3.8.0"
gyp ERR! find Python - version is 3.8.0 - should be >=2.6.0 <3.0.0
gyp ERR! find Python - THIS VERSION OF PYTHON IS NOT SUPPORTED
Full logs 
Step 14/20 : RUN ln -s /app/package.json /tools     && ln -s /app/package-lock.json /tools     && npm ci
 ---> Running in 81c0c71b9fb3

> @newrelic/native-metrics@4.1.0 install /tools/node_modules/@newrelic/native-metrics
> node ./lib/pre-build.js install native_metrics

============================================================================
Attempting install in native-metrics module. Please note that this is an
OPTIONAL dependency, and any resultant errors in this process will not
affect the general performance of the New Relic agent, but event loop and
garbage collection metrics will not be collected.
============================================================================

> /usr/local/bin/node /usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js clean configure
gyp ERR! find Python 
gyp ERR! find Python Python is not set from command line or npm configuration
gyp ERR! find Python Python is not set from environment variable PYTHON
gyp ERR! find Python checking if "python" can be used
gyp ERR! find Python - executable path is "/usr/local/bin/python"
gyp ERR! find Python - version is "3.8.0"
gyp ERR! find Python - version is 3.8.0 - should be >=2.6.0 <3.0.0
gyp ERR! find Python - THIS VERSION OF PYTHON IS NOT SUPPORTED
gyp ERR! find Python checking if "python2" can be used
gyp ERR! find Python - "python2" is not in PATH or produced an error
gyp ERR! find Python 
gyp ERR! find Python **********************************************************
gyp ERR! find Python You need to install the latest version of Python 2.7.
gyp ERR! find Python Node-gyp should be able to find and use Python. If not,
gyp ERR! find Python you can try one of the following options:
gyp ERR! find Python - Use the switch --python="/path/to/pythonexecutable"
gyp ERR! find Python   (accepted by both node-gyp and npm)
gyp ERR! find Python - Set the environment variable PYTHON
gyp ERR! find Python - Set the npm configuration variable python:
gyp ERR! find Python   npm config set python "/path/to/pythonexecutable"
gyp ERR! find Python For more information consult the documentation at:
gyp ERR! find Python https://github.com/nodejs/node-gyp#installation
gyp ERR! find Python **********************************************************
gyp ERR! find Python 
gyp ERR! configure error 
gyp ERR! stack Error: Could not find any Python installation to use
gyp ERR! stack     at PythonFinder.fail (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:303:47)
gyp ERR! stack     at PythonFinder.runChecks (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:132:21)
gyp ERR! stack     at PythonFinder.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:175:16)
gyp ERR! stack     at PythonFinder.execFileCallback (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/find-python.js:267:16)
gyp ERR! stack     at exithandler (child_process.js:301:5)
gyp ERR! stack     at ChildProcess.errorhandler (child_process.js:313:5)
gyp ERR! stack     at ChildProcess.emit (events.js:198:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:246:12)
gyp ERR! stack     at onErrorNT (internal/child_process.js:415:16)
gyp ERR! stack     at process._tickCallback (internal/process/next_tick.js:63:19)
gyp ERR! System Linux 4.9.184-linuxkit
gyp ERR! command "/usr/local/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "clean" "configure"
gyp ERR! cwd /tools/node_modules/@newrelic/native-metrics
gyp ERR! node -v v10.17.0
gyp ERR! node-gyp -v v5.0.3
gyp ERR! not ok 
install successful: _newrelic_native_metrics-4_1_0-native_metrics-64-linux-x64

Node 10.17.0 w/ node-gyp 6.0.1

  CXX(target) Release/obj.target/native_metrics/src/native_metrics.o
In file included from ../src/native_metrics.cpp:2:
../node_modules/nan/nan.h: In function ‘void Nan::AsyncQueueWorker(Nan::AsyncWorker*)’:
../node_modules/nan/nan.h:2298:62: warning: cast between incompatible function types from ‘void (*)(uv_work_t*)’ {aka ‘void (*)(uv_work_s*)’} to ‘uv_after_work_cb’ {aka ‘void (*)(uv_work_s*, int)’} [-Wcast-function-type]
Full logs 
Step 14/20 : RUN ln -s /app/package.json /tools     && ln -s /app/package-lock.json /tools     && npm ci
 ---> Running in ff86766ae669

> @newrelic/native-metrics@4.1.0 install /tools/node_modules/@newrelic/native-metrics
> node ./lib/pre-build.js install native_metrics

============================================================================
Attempting install in native-metrics module. Please note that this is an
OPTIONAL dependency, and any resultant errors in this process will not
affect the general performance of the New Relic agent, but event loop and
garbage collection metrics will not be collected.
============================================================================

> /usr/local/bin/node /usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/node_modules/node-gyp/bin/node-gyp.js clean configure
> /usr/local/bin/node /usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/node_modules/node-gyp/bin/node-gyp.js build -j 2 native_metrics
make: Entering directory '/tools/node_modules/@newrelic/native-metrics/build'
  CXX(target) Release/obj.target/native_metrics/src/GCBinder.o
  CXX(target) Release/obj.target/native_metrics/src/native_metrics.o
In file included from ../src/native_metrics.cpp:2:
../node_modules/nan/nan.h: In function ‘void Nan::AsyncQueueWorker(Nan::AsyncWorker*)’:
../node_modules/nan/nan.h:2298:62: warning: cast between incompatible function types from ‘void (*)(uv_work_t*)’ {aka ‘void (*)(uv_work_s*)’} to ‘uv_after_work_cb’ {aka ‘void (*)(uv_work_s*, int)’} [-Wcast-function-type]
     , reinterpret_cast<uv_after_work_cb>(AsyncExecuteComplete)
                                                              ^
In file included from ../src/GCBinder.hpp:4,
                 from ../src/GCBinder.cpp:2:
../node_modules/nan/nan.h: In function ‘void Nan::AsyncQueueWorker(Nan::AsyncWorker*)’:
../node_modules/nan/nan.h:2298:62: warning: cast between incompatible function types from ‘void (*)(uv_work_t*)’ {aka ‘void (*)(uv_work_s*)’} to ‘uv_after_work_cb’ {aka ‘void (*)(uv_work_s*, int)’} [-Wcast-function-type]
     , reinterpret_cast<uv_after_work_cb>(AsyncExecuteComplete)
                                                              ^
In file included from ../node_modules/nan/nan.h:54,
                 from ../src/native_metrics.cpp:2:
../src/native_metrics.cpp: At global scope:
/home/kuma/.cache/node-gyp/10.17.0/include/node/node.h:573:43: warning: cast between incompatible function types from ‘void (*)(Nan::ADDON_REGISTER_FUNCTION_ARGS_TYPE)’ {aka ‘void (*)(v8::Local<v8::Object>)’} to ‘node::addon_register_func’ {aka ‘void (*)(v8::Local<v8::Object>, v8::Local<v8::Value>, void*)’} [-Wcast-function-type]
       (node::addon_register_func) (regfunc),                          \
                                           ^
/home/kuma/.cache/node-gyp/10.17.0/include/node/node.h:607:3: note: in expansion of macro ‘NODE_MODULE_X’
   NODE_MODULE_X(modname, regfunc, NULL, 0)  // NOLINT (readability/null_usage)
   ^~~~~~~~~~~~~
../src/native_metrics.cpp:17:1: note: in expansion of macro ‘NODE_MODULE’
 NODE_MODULE(native_metrics, Init)
 ^~~~~~~~~~~
In file included from /home/kuma/.cache/node-gyp/10.17.0/include/node/node.h:63,
                 from ../node_modules/nan/nan.h:54,
                 from ../src/native_metrics.cpp:2:
/home/kuma/.cache/node-gyp/10.17.0/include/node/v8.h: In instantiation of ‘void v8::PersistentBase<T>::SetWeak(P*, typename v8::WeakCallbackInfo<P>::Callback, v8::WeakCallbackType) [with P = node::ObjectWrap; T = v8::Object; typename v8::WeakCallbackInfo<P>::Callback = void (*)(const v8::WeakCallbackInfo<node::ObjectWrap>&)]’:
/home/kuma/.cache/node-gyp/10.17.0/include/node/node_object_wrap.h:84:78:   required from here
/home/kuma/.cache/node-gyp/10.17.0/include/node/v8.h:9502:16: warning: cast between incompatible function types from ‘v8::WeakCallbackInfo<node::ObjectWrap>::Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<node::ObjectWrap>&)’} to ‘Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<void>&)’} [-Wcast-function-type]
                reinterpret_cast<Callback>(callback), type);
                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/kuma/.cache/node-gyp/10.17.0/include/node/node.h:63,
                 from ../node_modules/nan/nan.h:54,
                 from ../src/GCBinder.hpp:4,
                 from ../src/GCBinder.cpp:2:
/home/kuma/.cache/node-gyp/10.17.0/include/node/v8.h: In instantiation of ‘void v8::PersistentBase<T>::SetWeak(P*, typename v8::WeakCallbackInfo<P>::Callback, v8::WeakCallbackType) [with P = node::ObjectWrap; T = v8::Object; typename v8::WeakCallbackInfo<P>::Callback = void (*)(const v8::WeakCallbackInfo<node::ObjectWrap>&)]’:
/home/kuma/.cache/node-gyp/10.17.0/include/node/node_object_wrap.h:84:78:   required from here
/home/kuma/.cache/node-gyp/10.17.0/include/node/v8.h:9502:16: warning: cast between incompatible function types from ‘v8::WeakCallbackInfo<node::ObjectWrap>::Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<node::ObjectWrap>&)’} to ‘Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<void>&)’} [-Wcast-function-type]
                reinterpret_cast<Callback>(callback), type);
                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/kuma/.cache/node-gyp/10.17.0/include/node/v8.h: In instantiation of ‘void v8::PersistentBase<T>::SetWeak(P*, typename v8::WeakCallbackInfo<P>::Callback, v8::WeakCallbackType) [with P = Nan::ObjectWrap; T = v8::Object; typename v8::WeakCallbackInfo<P>::Callback = void (*)(const v8::WeakCallbackInfo<Nan::ObjectWrap>&)]’:
../node_modules/nan/nan_object_wrap.h:65:61:   required from here
/home/kuma/.cache/node-gyp/10.17.0/include/node/v8.h:9502:16: warning: cast between incompatible function types from ‘v8::WeakCallbackInfo<Nan::ObjectWrap>::Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<Nan::ObjectWrap>&)’} to ‘Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<void>&)’} [-Wcast-function-type]
/home/kuma/.cache/node-gyp/10.17.0/include/node/v8.h: In instantiation of ‘void v8::PersistentBase<T>::SetWeak(P*, typename v8::WeakCallbackInfo<P>::Callback, v8::WeakCallbackType) [with P = Nan::ObjectWrap; T = v8::Object; typename v8::WeakCallbackInfo<P>::Callback = void (*)(const v8::WeakCallbackInfo<Nan::ObjectWrap>&)]’:
../node_modules/nan/nan_object_wrap.h:65:61:   required from here
/home/kuma/.cache/node-gyp/10.17.0/include/node/v8.h:9502:16: warning: cast between incompatible function types from ‘v8::WeakCallbackInfo<Nan::ObjectWrap>::Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<Nan::ObjectWrap>&)’} to ‘Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<void>&)’} [-Wcast-function-type]
  CXX(target) Release/obj.target/native_metrics/src/LoopChecker.o
  CXX(target) Release/obj.target/native_metrics/src/RUsageMeter.o
In file included from ../src/LoopChecker.hpp:3,
                 from ../src/LoopChecker.cpp:5:
../node_modules/nan/nan.h: In function ‘void Nan::AsyncQueueWorker(Nan::AsyncWorker*)’:
../node_modules/nan/nan.h:2298:62: warning: cast between incompatible function types from ‘void (*)(uv_work_t*)’ {aka ‘void (*)(uv_work_s*)’} to ‘uv_after_work_cb’ {aka ‘void (*)(uv_work_s*, int)’} [-Wcast-function-type]
     , reinterpret_cast<uv_after_work_cb>(AsyncExecuteComplete)
                                                              ^
In file included from ../src/RUsageMeter.cpp:3:
../node_modules/nan/nan.h: In function ‘void Nan::AsyncQueueWorker(Nan::AsyncWorker*)’:
../node_modules/nan/nan.h:2298:62: warning: cast between incompatible function types from ‘void (*)(uv_work_t*)’ {aka ‘void (*)(uv_work_s*)’} to ‘uv_after_work_cb’ {aka ‘void (*)(uv_work_s*, int)’} [-Wcast-function-type]
     , reinterpret_cast<uv_after_work_cb>(AsyncExecuteComplete)
                                                              ^
In file included from /home/kuma/.cache/node-gyp/10.17.0/include/node/node.h:63,
                 from ../node_modules/nan/nan.h:54,
                 from ../src/LoopChecker.hpp:3,
                 from ../src/LoopChecker.cpp:5:
/home/kuma/.cache/node-gyp/10.17.0/include/node/v8.h: In instantiation of ‘void v8::PersistentBase<T>::SetWeak(P*, typename v8::WeakCallbackInfo<P>::Callback, v8::WeakCallbackType) [with P = node::ObjectWrap; T = v8::Object; typename v8::WeakCallbackInfo<P>::Callback = void (*)(const v8::WeakCallbackInfo<node::ObjectWrap>&)]’:
/home/kuma/.cache/node-gyp/10.17.0/include/node/node_object_wrap.h:84:78:   required from here
/home/kuma/.cache/node-gyp/10.17.0/include/node/v8.h:9502:16: warning: cast between incompatible function types from ‘v8::WeakCallbackInfo<node::ObjectWrap>::Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<node::ObjectWrap>&)’} to ‘Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<void>&)’} [-Wcast-function-type]
                reinterpret_cast<Callback>(callback), type);
                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/kuma/.cache/node-gyp/10.17.0/include/node/v8.h: In instantiation of ‘void v8::PersistentBase<T>::SetWeak(P*, typename v8::WeakCallbackInfo<P>::Callback, v8::WeakCallbackType) [with P = Nan::ObjectWrap; T = v8::Object; typename v8::WeakCallbackInfo<P>::Callback = void (*)(const v8::WeakCallbackInfo<Nan::ObjectWrap>&)]’:
../node_modules/nan/nan_object_wrap.h:65:61:   required from here
/home/kuma/.cache/node-gyp/10.17.0/include/node/v8.h:9502:16: warning: cast between incompatible function types from ‘v8::WeakCallbackInfo<Nan::ObjectWrap>::Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<Nan::ObjectWrap>&)’} to ‘Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<void>&)’} [-Wcast-function-type]
In file included from /home/kuma/.cache/node-gyp/10.17.0/include/node/node.h:63,
                 from ../node_modules/nan/nan.h:54,
                 from ../src/RUsageMeter.cpp:3:
/home/kuma/.cache/node-gyp/10.17.0/include/node/v8.h: In instantiation of ‘void v8::PersistentBase<T>::SetWeak(P*, typename v8::WeakCallbackInfo<P>::Callback, v8::WeakCallbackType) [with P = node::ObjectWrap; T = v8::Object; typename v8::WeakCallbackInfo<P>::Callback = void (*)(const v8::WeakCallbackInfo<node::ObjectWrap>&)]’:
/home/kuma/.cache/node-gyp/10.17.0/include/node/node_object_wrap.h:84:78:   required from here
/home/kuma/.cache/node-gyp/10.17.0/include/node/v8.h:9502:16: warning: cast between incompatible function types from ‘v8::WeakCallbackInfo<node::ObjectWrap>::Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<node::ObjectWrap>&)’} to ‘Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<void>&)’} [-Wcast-function-type]
                reinterpret_cast<Callback>(callback), type);
                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/kuma/.cache/node-gyp/10.17.0/include/node/v8.h: In instantiation of ‘void v8::PersistentBase<T>::SetWeak(P*, typename v8::WeakCallbackInfo<P>::Callback, v8::WeakCallbackType) [with P = Nan::ObjectWrap; T = v8::Object; typename v8::WeakCallbackInfo<P>::Callback = void (*)(const v8::WeakCallbackInfo<Nan::ObjectWrap>&)]’:
../node_modules/nan/nan_object_wrap.h:65:61:   required from here
/home/kuma/.cache/node-gyp/10.17.0/include/node/v8.h:9502:16: warning: cast between incompatible function types from ‘v8::WeakCallbackInfo<Nan::ObjectWrap>::Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<Nan::ObjectWrap>&)’} to ‘Callback’ {aka ‘void (*)(const v8::WeakCallbackInfo<void>&)’} [-Wcast-function-type]
  SOLINK_MODULE(target) Release/obj.target/native_metrics.node
  COPY Release/native_metrics.node
make: Leaving directory '/tools/node_modules/@newrelic/native-metrics/build'
install successful: _newrelic_native_metrics-4_1_0-native_metrics-64-linux-x64

Expected behavior

  • ...things build?

Additional context

According to discussion in newrelic/node-native-metrics#62, the install successful: _newrelic_native_metrics-4_1_0-native_metrics-64-linux-x64 message indicates a successful fallback to a prebuilt binary supplied by New Relic. So maybe we don't need to worry about this for now?
@callahad callahad added the status: needs triage Status: Untriaged label Dec 4, 2019
@callahad callahad mentioned this issue Dec 6, 2019
Closed
@callahad
Copy link
Contributor Author

callahad commented Dec 9, 2019

Per Slack, seems like our options are:

  1. Install Python 2, just to satisfy building native_metrics
  2. Mess with NPM under the hood to upgrade it's bundled copy of node-gyp to a version that supports Python 3
  3. Accept that we can't easily build the extension module ourselves, and automatically fall back to New Relic's prebuilt binaries if possible. It is optional, after all.

...Peter and I are both happy enough with the last option.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
status: needs triage Status: Untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@callahad @tobinmori