Release proposal: v5.0.4

.travis.yml

@@ -8,7 +8,7 @@ matrix:
       python: 2.7
     - name: "Python 2.7 on macOS"
       os: osx
-      osx_image: xcode10.2
+      osx_image: xcode11
       language: shell  # 'language: python' is not yet supported on macOS
       env: NODE_GYP_FORCE_PYTHON=python2
       before_install: HOMEBREW_NO_AUTO_UPDATE=1 brew install npm
@@ -28,6 +28,7 @@ matrix:
         PATH=/c/Python27:/c/Python27/Scripts:$PATH
         NODE_GYP_FORCE_PYTHON=/c/Python27/python.exe
       before_install: choco install python2
+
     - name: "Node.js 6 & Python 3.7 on Linux"
       python: 3.7
       env: NODE_GYP_FORCE_PYTHON=python3 EXPERIMENTAL_NODE_GYP_PYTHON3=1
@@ -44,6 +45,12 @@ matrix:
       python: 3.7
       env: NODE_GYP_FORCE_PYTHON=python3 EXPERIMENTAL_NODE_GYP_PYTHON3=1
       before_install: nvm install 12
+    - name: "Python 3.7 on macOS"
+      os: osx
+      #osx_image: xcode11
+      language: shell  # 'language: python' is not yet supported on macOS
+      env: NODE_GYP_FORCE_PYTHON=python3 EXPERIMENTAL_NODE_GYP_PYTHON3=1
+      before_install: HOMEBREW_NO_AUTO_UPDATE=1 brew install npm
     - name: "Node.js 12 & Python 3.7 on Windows"
       os: windows
       language: node_js
@@ -53,12 +60,7 @@ matrix:
         NODE_GYP_FORCE_PYTHON=/c/Python37/python.exe
         EXPERIMENTAL_NODE_GYP_PYTHON3=1
       before_install: choco install python
-  allow_failures:
-    - env: NODE_GYP_FORCE_PYTHON=python3 EXPERIMENTAL_NODE_GYP_PYTHON3=1
-    - env: >-
-        PATH=/c/Python37:/c/Python37/Scripts:$PATH
-        NODE_GYP_FORCE_PYTHON=/c/Python37/python.exe
-        EXPERIMENTAL_NODE_GYP_PYTHON3=1
+
 install:
   #- pip install -r requirements.txt
   - pip install flake8  # pytest  # add another testing frameworks later

CHANGELOG.md

@@ -1,3 +1,27 @@
+v5.0.4 2019-09-27
+=================
+
+* [[`1236869ffc`](https://github.com/nodejs/node-gyp/commit/1236869ffc)] - **gyp**: modify XcodeVersion() to convert "4.2" to "0420" and "10.0" to "1000" (Christian Clauss) [#1895](https://github.com/nodejs/node-gyp/pull/1895)
+* [[`36638afe48`](https://github.com/nodejs/node-gyp/commit/36638afe48)] - **gyp**: more decode stdout on Python 3 (cclauss) [#1894](https://github.com/nodejs/node-gyp/pull/1894)
+* [[`f753c167c5`](https://github.com/nodejs/node-gyp/commit/f753c167c5)] - **gyp**: decode stdout on Python 3 (cclauss) [#1890](https://github.com/nodejs/node-gyp/pull/1890)
+* [[`60a4083523`](https://github.com/nodejs/node-gyp/commit/60a4083523)] - **doc**: update xcode install instructions to match Node's BUILDING (Nhan Khong) [#1884](https://github.com/nodejs/node-gyp/pull/1884)
+* [[`19dbc9ac32`](https://github.com/nodejs/node-gyp/commit/19dbc9ac32)] - **deps**: update tar to 4.4.12 (Matheus Marchini) [#1889](https://github.com/nodejs/node-gyp/pull/1889)
+* [[`5f3ed92181`](https://github.com/nodejs/node-gyp/commit/5f3ed92181)] - **bin**: fix the usage instructions (Halit Ogunc) [#1888](https://github.com/nodejs/node-gyp/pull/1888)
+* [[`aab118edf1`](https://github.com/nodejs/node-gyp/commit/aab118edf1)] - **lib**: adding keep-alive header to download requests (Milad Farazmand) [#1863](https://github.com/nodejs/node-gyp/pull/1863)
+* [[`1186e89326`](https://github.com/nodejs/node-gyp/commit/1186e89326)] - **lib**: ignore non-critical os.userInfo() failures (Rod Vagg) [#1835](https://github.com/nodejs/node-gyp/pull/1835)
+* [[`785e527c3d`](https://github.com/nodejs/node-gyp/commit/785e527c3d)] - **doc**: fix missing argument for setting python path (lagorsse) [#1802](https://github.com/nodejs/node-gyp/pull/1802)
+* [[`a97615196c`](https://github.com/nodejs/node-gyp/commit/a97615196c)] - **gyp**: rm semicolons (Python != JavaScript) (MattIPv4) [#1858](https://github.com/nodejs/node-gyp/pull/1858)
+* [[`06019bac24`](https://github.com/nodejs/node-gyp/commit/06019bac24)] - **gyp**: assorted typo fixes (XhmikosR) [#1853](https://github.com/nodejs/node-gyp/pull/1853)
+* [[`3f4972c1ca`](https://github.com/nodejs/node-gyp/commit/3f4972c1ca)] - **gyp**: use "is" when comparing to None (Vladyslav Burzakovskyy) [#1860](https://github.com/nodejs/node-gyp/pull/1860)
+* [[`1cb4708073`](https://github.com/nodejs/node-gyp/commit/1cb4708073)] - **src,win**: improve unmanaged handling (Peter Sabath) [#1852](https://github.com/nodejs/node-gyp/pull/1852)
+* [[`5553cd910e`](https://github.com/nodejs/node-gyp/commit/5553cd910e)] - **gyp**: improve Windows+Cygwin compatibility (Jose Quijada) [#1817](https://github.com/nodejs/node-gyp/pull/1817)
+* [[`8bcb1fbb43`](https://github.com/nodejs/node-gyp/commit/8bcb1fbb43)] - **gyp**: Python 3 Windows fixes (João Reis) [#1843](https://github.com/nodejs/node-gyp/pull/1843)
+* [[`2e24d0a326`](https://github.com/nodejs/node-gyp/commit/2e24d0a326)] - **test**: accept Python 3 in test-find-python.js (João Reis) [#1843](https://github.com/nodejs/node-gyp/pull/1843)
+* [[`1267b4dc1c`](https://github.com/nodejs/node-gyp/commit/1267b4dc1c)] - **build**: add test run Python 3.7 on macOS (Christian Clauss) [#1843](https://github.com/nodejs/node-gyp/pull/1843)
+* [[`da1b031aa3`](https://github.com/nodejs/node-gyp/commit/da1b031aa3)] - **build**: import StringIO on Python 2 and Python 3 (Christian Clauss) [#1836](https://github.com/nodejs/node-gyp/pull/1836)
+* [[`fa0ed4aa42`](https://github.com/nodejs/node-gyp/commit/fa0ed4aa42)] - **build**: more Python 3 compat, replace compile with ast (cclauss) [#1820](https://github.com/nodejs/node-gyp/pull/1820)
+* [[`18d5c7c9d0`](https://github.com/nodejs/node-gyp/commit/18d5c7c9d0)] - **win,src**: update win\_delay\_load\_hook.cc to work with /clr (Ivan Petrovic) [#1819](https://github.com/nodejs/node-gyp/pull/1819)
+
 v5.0.3 2019-07-17
 =================
 

README.md

@@ -35,8 +35,7 @@ You will also need to install:
 
    * `python` (`v2.7` recommended, `v3.x.x` is __*not*__ supported) (already installed on macOS)
    * [Xcode](https://developer.apple.com/xcode/download/)
-     * You also need to install the `Command Line Tools` via Xcode. You can find this under the menu `Xcode -> Preferences -> Locations` (or by running `xcode-select --install` in your Terminal)
-       * This step will install `gcc` and the related toolchain containing `make`
+     * You also need to install the `XCode Command Line Tools` by running `xcode-select --install`. Alternatively, if you already have the full Xcode installed, you can find them under the menu `Xcode -> Open Developer Tool -> More Developer Tools...`. This step will install `clang`, `clang++`, and `make`.
 
 ### On Windows
 
@@ -63,7 +62,7 @@ If you have multiple Python versions installed, you can identify which Python
 version `node-gyp` uses by setting the `--python` variable:
 
 ``` bash
-$ node-gyp --python /path/to/python2.7
+$ node-gyp <command> --python /path/to/executable/python2.7
 ```
 
 If `node-gyp` is called by way of `npm`, *and* you have multiple versions of

gyp/gyp_main.py

@@ -6,10 +6,44 @@
 
 import os
 import sys
+import subprocess
+
+PY3 = bytes != str
+
+# Below IsCygwin() function copied from pylib/gyp/common.py
+def IsCygwin():
+  try:
+    out = subprocess.Popen("uname",
+            stdout=subprocess.PIPE,
+            stderr=subprocess.STDOUT)
+    stdout, stderr = out.communicate()
+    if PY3:
+      stdout = stdout.decode("utf-8")
+    return "CYGWIN" in str(stdout)
+  except Exception:
+    return False
+
+
+def UnixifyPath(path):
+  try:
+    if not IsCygwin():
+      return path
+    out = subprocess.Popen(["cygpath", "-u", path],
+            stdout=subprocess.PIPE,
+            stderr=subprocess.STDOUT)
+    stdout, stderr = out.communicate()
+    if PY3:
+      stdout = stdout.decode("utf-8")
+    return str(stdout)
+  except Exception:
+    return path
+
 
 # Make sure we're using the version of pylib in this repo, not one installed
-# elsewhere on the system.
-sys.path.insert(0, os.path.join(os.path.dirname(sys.argv[0]), 'pylib'))
+# elsewhere on the system. Also convert to Unix style path on Cygwin systems,
+# else the 'gyp' library will not be found
+path = UnixifyPath(sys.argv[0])
+sys.path.insert(0, os.path.join(os.path.dirname(path), 'pylib'))
 import gyp
 
 if __name__ == '__main__':

gyp/pylib/gyp/MSVSNew.py

@@ -45,7 +45,7 @@ def MakeGuid(name, seed='msvs_new'):
   not change when the project for a target is rebuilt.
   """
   # Calculate a MD5 signature for the seed and name.
-  d = hashlib.md5(str(seed) + str(name)).hexdigest().upper()
+  d = hashlib.md5((str(seed) + str(name)).encode('utf-8')).hexdigest().upper()
   # Convert most of the signature to GUID form (discard the rest)
   guid = ('{' + d[:8] + '-' + d[8:12] + '-' + d[12:16] + '-' + d[16:20]
           + '-' + d[20:32] + '}')

gyp/pylib/gyp/MSVSSettings_test.py

@@ -6,15 +6,19 @@
 
 """Unit tests for the MSVSSettings.py file."""
 
-import StringIO
+try:
+  from cStringIO import StringIO
+except ImportError:
+  from io import StringIO
+
 import unittest
 import gyp.MSVSSettings as MSVSSettings
 
 
 class TestSequenceFunctions(unittest.TestCase):
 
   def setUp(self):
-    self.stderr = StringIO.StringIO()
+    self.stderr = StringIO()
 
   def _ExpectedWarnings(self, expected):
     """Compares recorded lines to expected warnings."""

gyp/pylib/gyp/common.py

@@ -9,6 +9,9 @@
 import re
 import tempfile
 import sys
+import subprocess
+
+PY3 = bytes != str
 
 
 # A minimal memoizing decorator. It'll blow up if the args aren't immutable,
@@ -337,11 +340,16 @@ def WriteOnDiff(filename):
   class Writer(object):
     """Wrapper around file which only covers the target if it differs."""
     def __init__(self):
+      # On Cygwin remove the "dir" argument because `C:` prefixed paths are treated as relative,
+      # consequently ending up with current dir "/cygdrive/c/..." being prefixed to those, which was
+      # obviously a non-existent path, for example: "/cygdrive/c/<some folder>/C:\<my win style abs path>".
+      # See https://docs.python.org/2/library/tempfile.html#tempfile.mkstemp for more details
+      base_temp_dir = "" if IsCygwin() else os.path.dirname(filename)
       # Pick temporary file.
       tmp_fd, self.tmp_path = tempfile.mkstemp(
           suffix='.tmp',
           prefix=os.path.split(filename)[1] + '.gyp.',
-          dir=os.path.split(filename)[0])
+          dir=base_temp_dir)
       try:
         self.tmp_file = os.fdopen(tmp_fd, 'wb')
       except Exception:
@@ -394,6 +402,9 @@ def close(self):
         os.unlink(self.tmp_path)
         raise
 
+    def write(self, s):
+      self.tmp_file.write(s.encode('utf-8'))
+
   return Writer()
 
 
@@ -608,3 +619,16 @@ def CrossCompileRequested():
           os.environ.get('AR_target') or
           os.environ.get('CC_target') or
           os.environ.get('CXX_target'))
+
+def IsCygwin():
+  try:
+    out = subprocess.Popen("uname",
+            stdout=subprocess.PIPE,
+            stderr=subprocess.STDOUT)
+    stdout, stderr = out.communicate()
+    if PY3:
+      stdout = stdout.decode("utf-8")
+    return "CYGWIN" in str(stdout)
+  except Exception:
+    return False
+

gyp/pylib/gyp/common_test.py

@@ -59,10 +59,10 @@ def test_platform_default(self):
     self.assertFlavor('freebsd', 'freebsd9' , {})
     self.assertFlavor('freebsd', 'freebsd10', {})
     self.assertFlavor('openbsd', 'openbsd5' , {})
-    self.assertFlavor('solaris', 'sunos5'   , {});
-    self.assertFlavor('solaris', 'sunos'    , {});
-    self.assertFlavor('linux'  , 'linux2'   , {});
-    self.assertFlavor('linux'  , 'linux3'   , {});
+    self.assertFlavor('solaris', 'sunos5'   , {})
+    self.assertFlavor('solaris', 'sunos'    , {})
+    self.assertFlavor('linux'  , 'linux2'   , {})
+    self.assertFlavor('linux'  , 'linux3'   , {})
 
   def test_param(self):
     self.assertFlavor('foobar', 'linux2' , {'flavor': 'foobar'})

gyp/pylib/gyp/easy_xml.py

@@ -120,7 +120,7 @@ def WriteXmlIfChanged(content, path, encoding='utf-8', pretty=False,
 
   default_encoding = locale.getdefaultlocale()[1]
   if default_encoding.upper() != encoding.upper():
-    xml_string = xml_string.decode(default_encoding).encode(encoding)
+    xml_string = xml_string.encode(encoding)
 
   # Get the old content
   try:
@@ -132,7 +132,7 @@ def WriteXmlIfChanged(content, path, encoding='utf-8', pretty=False,
 
   # It has changed, write it
   if existing != xml_string:
-    f = open(path, 'w')
+    f = open(path, 'wb')
     f.write(xml_string)
     f.close()
 

gyp/pylib/gyp/easy_xml_test.py

@@ -8,13 +8,16 @@
 
 import gyp.easy_xml as easy_xml
 import unittest
-import StringIO
+try:
+  from cStringIO import StringIO
+except ImportError:
+  from io import StringIO
 
 
 class TestSequenceFunctions(unittest.TestCase):
 
   def setUp(self):
-    self.stderr = StringIO.StringIO()
+    self.stderr = StringIO()
 
   def test_EasyXml_simple(self):
     self.assertEqual(

gyp/pylib/gyp/generator/analyzer.py

@@ -615,7 +615,7 @@ def _supplied_target_names(self):
 
   def _supplied_target_names_no_all(self):
     """Returns the supplied test targets without 'all'."""
-    result = self._supplied_target_names();
+    result = self._supplied_target_names()
     result.discard('all')
     return result
 
@@ -668,10 +668,10 @@ def find_matching_test_target_names(self):
 
   def find_matching_compile_target_names(self):
     """Returns the set of output compile targets."""
-    assert self.is_build_impacted();
+    assert self.is_build_impacted()
     # Compile targets are found by searching up from changed targets.
     # Reset the visited status for _GetBuildTargets.
-    for target in self._name_to_target.itervalues():
+    for target in self._name_to_target.values():
       target.visited = False
 
     supplied_targets = _LookupTargets(self._supplied_target_names_no_all(),

gyp/pylib/gyp/generator/android.py

@@ -550,7 +550,7 @@ def WriteSources(self, spec, configs, extra_sources):
     # to work properly. If the file has the wrong C++ extension, then we add
     # a rule to copy that to intermediates and use the new version.
     final_generated_sources = []
-    # If a source file gets copied, we still need to add the orginal source
+    # If a source file gets copied, we still need to add the original source
     # directory as header search path, for GCC searches headers in the
     # directory that contains the source file by default.
     origin_src_dirs = []

gyp/pylib/gyp/generator/cmake.py

@@ -574,7 +574,7 @@ class CMakeNamer(object):
   """Converts Gyp target names into CMake target names.
 
   CMake requires that target names be globally unique. One way to ensure
-  this is to fully qualify the names of the targets. Unfortunatly, this
+  this is to fully qualify the names of the targets. Unfortunately, this
   ends up with all targets looking like "chrome_chrome_gyp_chrome" instead
   of just "chrome". If this generator were only interested in building, it
   would be possible to fully qualify all target names, then create
@@ -681,7 +681,7 @@ def WriteTarget(namer, qualified_target, target_dicts, build_dir, config_to_use,
   for src in srcs:
     _, ext = os.path.splitext(src)
     src_type = COMPILABLE_EXTENSIONS.get(ext, None)
-    src_norm_path = NormjoinPath(path_from_cmakelists_to_gyp, src);
+    src_norm_path = NormjoinPath(path_from_cmakelists_to_gyp, src)
 
     if src_type == 's':
       s_sources.append(src_norm_path)

gyp/pylib/gyp/generator/eclipse.py

@@ -195,7 +195,7 @@ def GetAllDefines(target_list, target_dicts, data, config_name, params,
   """Calculate the defines for a project.
 
   Returns:
-    A dict that includes explict defines declared in gyp files along with all of
+    A dict that includes explicit defines declared in gyp files along with all of
     the default defines that the compiler uses.
   """
 
@@ -272,7 +272,7 @@ def WriteMacros(out, eclipse_langs, defines):
   out.write('    <language name="holder for library settings"></language>\n')
   for lang in eclipse_langs:
     out.write('    <language name="%s">\n' % lang)
-    for key in sorted(defines.iterkeys()):
+    for key in sorted(defines):
       out.write('      <macro><name>%s</name><value>%s</value></macro>\n' %
                 (escape(key), escape(defines[key])))
     out.write('    </language>\n')