Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove /MP from default additonal options #22661

Closed
wants to merge 3 commits into from

Conversation

skelliam
Copy link
Contributor

@skelliam skelliam commented Sep 2, 2018

Forcing this option means that the user-provided option MultiProcessorCompilation in msvs_settings (from, for example, binding.gyp) is useless. It also means that one cannot use #import in their source code or they will be faced with this error when trying to build:

error C2813: #import is not supported with /MP (compiling source file...)

Please see additional discussion of this here: nodejs/node-gyp#1087
and here: nodejs/node-gyp#26

@nodejs-github-bot nodejs-github-bot added the build Issues and PRs related to build files or the CI. label Sep 2, 2018
@addaleax
Copy link
Member

addaleax commented Sep 2, 2018

@nodejs/platform-windows @nodejs/node-gyp

@refack refack added the windows Issues and PRs related to the Windows platform. label Sep 2, 2018
@refack
Copy link
Contributor

refack commented Sep 2, 2018

/CC @nodejs/build-files

This makes sense to me. It's a low value optimization, and if it has negative effects (re #import) we can remove this.

refack
refack previously approved these changes Sep 2, 2018
Copy link
Contributor

@refack refack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still LGTM

@skelliam
Copy link
Contributor Author

skelliam commented Sep 2, 2018

There is one more instance; can I add on to this commit?

@tniessen
Copy link
Member

tniessen commented Sep 2, 2018

Will this make building Node.js itself even slower on Windows?

@skelliam
Copy link
Contributor Author

skelliam commented Sep 2, 2018

@tniessen , if you look at the comment in nodejs/node-gyp#1087, it appears that Ninja already handles parallelization, and in fact it looks like there is some effort filter /MP out anyway.

@tniessen
Copy link
Member

tniessen commented Sep 2, 2018

cc @seishun who worked on this flag before.

@skelliam
Copy link
Contributor Author

skelliam commented Sep 2, 2018

Here is the documentation from MSDN regarding #import with /MP.

@refack
Copy link
Contributor

refack commented Sep 2, 2018

FYI: GYP allows for removing items from list with the ! suffix, so the following msvs_settings block removes the /MP:

{
  'targets': [
    {
      'target_name': 'binding',
      'defines': [ 'V8_DEPRECATION_WARNINGS=1' ],
      'sources': [ 'binding.cc' ],
      'msvs_settings': {
        'VCCLCompilerTool': {
          'AdditionalOptions!': [
            '/MP'
          ]
        }
      }
    }
  ]
}

@refack
Copy link
Contributor

refack commented Sep 2, 2018

Will this make building Node.js itself even slower on Windows?

Highly unlikely, since even when you use MSBuild it does multiprocess compilation by default, and we ask it to use at least two in vcbuild.bat:

node/vcbuild.bat

Lines 300 to 305 in 9f7efd5

set "msbcpu=/m:2"
if "%NUMBER_OF_PROCESSORS%"=="1" set "msbcpu=/m:1"
set "msbplatform=Win32"
if "%target_arch%"=="x64" set "msbplatform=x64"
if "%target%"=="Build" if defined no_cctest set target=node
msbuild node.sln %msbcpu% /t:%target% /p:Configuration=%config% /p:Platform=%msbplatform% /clp:NoSummary;NoItemAndPropertyList;Verbosity=minimal /nologo

P.S. /MP is probably incompatible with clcache

@skelliam
Copy link
Contributor Author

skelliam commented Sep 2, 2018

Thanks @refack; was looking all over for a solution to this, sounds like your proposal would have worked. I did try to set the contents of AdditionalOptions to an empty string, but as mentioned in the linked discussion, this is an additive option.

@refack
Copy link
Contributor

refack commented Sep 2, 2018

AdditionalOptions to an empty string, but as mentioned in the linked discussion, this is an additive option.

BTW for that GYP has the = suffix, so:

'msvs_settings': { 'VCCLCompilerTool': { 
  'AdditionalOptions=': []
} }

@targos
Copy link
Member

targos commented Sep 2, 2018

@tniessen

Will this make building Node.js itself even slower on Windows?

Building Node.js on Windows is not as slow as it used to be. There were recent improvements.

@bzoz
Copy link
Contributor

bzoz commented Sep 3, 2018

I'm -1. On my box this increases clean build time from 9 min 22 sec to 14 min 21 sec.

seishun
seishun previously requested changes Sep 3, 2018
Copy link
Contributor

@seishun seishun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-1 if it slows down compilation as @bzoz described.

@richardlau
Copy link
Member

Longer term the better solution is probably to implement nodejs/node-gyp#1118 and stop node-gyp using core's common.gypi.

Copy link
Member

@joaocgreis joaocgreis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will make compiling core much slower. I see CI hasn't yet been run for this PR, but it should be very noticeable there. If there's a way to make /MP active only for node core then this might be worth considering.

@skelliam
Copy link
Contributor Author

skelliam commented Sep 3, 2018

Can't you still have /MP with the XML option "MultiProcessorCompilation" in msvs_settings? This way the user override is at least more intuitive (IMO).

Otherwise I like the idea from #1118.

@refack refack dismissed their stale review September 3, 2018 21:27

Not until we find how to offset

@refack
Copy link
Contributor

refack commented Sep 3, 2018

I see CI hasn't yet been run for this PR, but it should be very noticeable there.

I did:
Pre on test-rackspace-win2012r2-x64-12 — 35m
Post on test-rackspace-win2012r2-x64-11 — 50m
34% increase in time. But if this is an issue of higher parallelism, I have a feeling we could offset that with a higher number in

set "msbcpu=/m:2"

@refack
Copy link
Contributor

refack commented Sep 3, 2018

Can't you still have /MP with the XML option "MultiProcessorCompilation" in msvs_settings? This way the user override is at least more intuitive (IMO).

I think that makes sense.
👍

this will replace the hardcoded /MP with an implicit /MP that is easier for users to override.
@skelliam
Copy link
Contributor Author

skelliam commented Sep 3, 2018

Added 'MultiProcessorCompilation': 'true' (implicit /MP) instead of the hardcoded /MP. Users can override this with 'MultiProcessorCompilation': 'false' in their own binding.gyp.

Copy link
Contributor

@refack refack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯 Since this is essentially a no-op.

Copy link
Member

@richardlau richardlau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving the change from /MP to 'MultiProcessorCompilation': 'true' provided build times remain unchanged.

@BridgeAR BridgeAR dismissed seishun’s stale review September 5, 2018 00:03

Outdated. PTAL

@BridgeAR BridgeAR added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Sep 5, 2018
@BridgeAR
Copy link
Member

BridgeAR commented Sep 5, 2018

@tniessen
Copy link
Member

tniessen commented Sep 9, 2018

The compilation time on Windows did not increase after the latest CI run.

@tniessen
Copy link
Member

tniessen commented Sep 9, 2018

Landed in 56d9cd4.

@tniessen tniessen closed this Sep 9, 2018
tniessen pushed a commit that referenced this pull request Sep 9, 2018
PR-URL: #22661
Reviewed-By: João Reis <reis@janeasystems.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Bartosz Sosnowski <bartosz@janeasystems.com>
targos pushed a commit that referenced this pull request Sep 10, 2018
PR-URL: #22661
Reviewed-By: João Reis <reis@janeasystems.com>
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Bartosz Sosnowski <bartosz@janeasystems.com>
@skelliam skelliam deleted the patch-2 branch September 10, 2018 18:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. build Issues and PRs related to build files or the CI. windows Issues and PRs related to the Windows platform.
Projects
None yet
Development

Successfully merging this pull request may close these issues.