Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

<atomic>: In pre-C++20 mode, the constructor should be trivial #661

Open
zeux opened this issue Mar 31, 2020 · 19 comments
Open

<atomic>: In pre-C++20 mode, the constructor should be trivial #661

zeux opened this issue Mar 31, 2020 · 19 comments
Labels
bug Something isn't working

Comments

@zeux
Copy link

zeux commented Mar 31, 2020

Starting from VS 2019 16.6 (#336), std::atomic constructor is unconditionally initializing the value:

    constexpr atomic() noexcept(is_nothrow_default_constructible_v<_Ty>) : _Base() {}

Before this version. the constructor was trivial instead:

#ifdef __clang__ // TRANSITION, VSO-406237
    constexpr atomic() noexcept(is_nothrow_default_constructible_v<_Ty>) : _Base() {}
#else // ^^^ no workaround / workaround vvv
    atomic()                                  = default;
#endif // TRANSITION, VSO-406237

As far as I understand, this is a behavior change in C++20 standard. However, I would not expect to see this behavior without specifying /std:c++latest.

This is significant because it results in some cases where std::atomic in global scope now generates dynamic initializers. Notably, in this case:

struct Foo
{
    std::atomic<int> data[2];
};

Foo foo[2];

Before this change, the foo variable was placed into BSS with no dynamic initializer. After this change, the dynamic initializer is emitted. This is problematic because if any code in static constructors accesses the atomic, it may work on the atomic before it's initialized which means that the initialization will overwrite the value of the atomic to 0.

Thus this is a breaking change - which would be fine except that it's a silent breaking change, and I'd expect to have to opt into this change by using the latest standard version instead.

@zeux
Copy link
Author

zeux commented Mar 31, 2020

Here's a full repro:

#include <atomic>
#include <stdio.h>

struct Meow
{
	std::atomic<int> data[2];
	char padding[64];
};

int bar();

// SWAP THESE TWO LINES TO OBSERVE THE BUG
Meow cats[256];
int foo = bar();

int bar()
{
	// edit: can add atomic_init(&cats[0].data[0], 0); here to eliminate UB - doesn't change behavior
	return cats[0].data[0]++;
}

int main()
{
	printf("%d\n", cats[0].data[0].load());
}

With latest MSVC (19.26.28720.3), this program prints 1 as written, and 0 if you swap the two indicated lines.

@BillyONeal
Copy link
Member

There's certainly a c1xx bug here, and maybe also a clang bug:

C:\Users\billy\Desktop>type repro.cpp
#include <assert.h>

struct FakeAtomic {
    int example;

    constexpr FakeAtomic() noexcept : example{} {}
    int operator++() { return ++example; }
    int operator++(int) { return example++; }
    int load() const { return example; }
};

struct Meow {
    FakeAtomic data[2];
    char padding[64];
};

int bar();

int foo = bar();
#ifdef WORKAROUND
Meow should_be_statically_initalized[256]{};
#else
Meow should_be_statically_initalized[256];
#endif

int bar() { return should_be_statically_initalized[0].data[0]++; }

int main() {
    assert(foo == 0);
    assert(should_be_statically_initalized[0].data[0].load() == 1);
}

C:\Users\billy\Desktop>cl /EHsc /W4 /WX .\repro.cpp
Microsoft (R) C/C++ Optimizing Compiler Version 19.25.28611 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

repro.cpp
Microsoft (R) Incremental Linker Version 14.25.28611.0
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:repro.exe
repro.obj

C:\Users\billy\Desktop>.\repro.exe
Assertion failed: should_be_statically_initalized[0].data[0].load() == 1, file .\repro.cpp, line 30

C:\Users\billy\Desktop>clang-cl /EHsc /W4 /WX .\repro.cpp

C:\Users\billy\Desktop>.\repro.exe
Assertion failed: should_be_statically_initalized[0].data[0].load() == 1, file .\repro.cpp, line 30

C:\Users\billy\Desktop>cl /DWORKAROUND /EHsc /W4 /WX .\repro.cpp
Microsoft (R) C/C++ Optimizing Compiler Version 19.25.28611 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

repro.cpp
Microsoft (R) Incremental Linker Version 14.25.28611.0
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:repro.exe
repro.obj

C:\Users\billy\Desktop>.\repro.exe
Assertion failed: should_be_statically_initalized[0].data[0].load() == 1, file .\repro.cpp, line 30

C:\Users\billy\Desktop>clang-cl /DWORKAROUND /EHsc /W4 /WX .\repro.cpp

C:\Users\billy\Desktop>.\repro.exe

C:\Users\billy\Desktop>

@BillyONeal
Copy link
Member

As I mentioned on Twitter, implementing WG21-P0883 unconditionally is very much intentional because the previous C++11-C++17 rules leaves the atomic in a broken state before someone calls atomic_init. That is, the C++17 rule said that the atomic constructor left the atomic "uninitialized", meaning there is no int there to load or ++ or whatever, before atomic_init is called. This behavior was so user hostile that we never implemented the C++11 rules; we always default-initialized the value. As far as I am aware libc++ on Clang was the only library implementation that tried to comply with the C++11 rules.

Given that the old rules were broken, the C++20 rules have a high likelihood of doing what the user expected, and the C++20 rules are actually implementable for us, we implement them unconditionally. However, we do depend on the compilers getting static initialization right, and it looks like they didn't in this case.

@BillyONeal
Copy link
Member

BillyONeal commented Mar 31, 2020

C1XX bug filed as DevCom-970302 (I don't believe we have a workaround) see workaround by @AlexGuteniev below
Clang bug filed as LLVM-45367 (Workaround by adding a {} initializer)

@BillyONeal
Copy link
Member

Upon further reflection, [basic.start.static]/2 says that either constant initialization is performed, or zero initialization, but your example wants both, so perhaps clang is correct to emit a dynamic initializer here without the extra {}s. Will keep you posted with what they say.

@AlexGuteniev
Copy link
Contributor

From what I understood so far, it is better not to mix default and zero initialization at all.

Here's workaround that works in MSVC:

struct Meow
{
	std::atomic<int> data[2];
	char padding[64] = {};
};

@BillyONeal
Copy link
Member

Richard Smith, who works on Clang, has reposted this to the C++ committee's Core Working Group:

Consider:

struct X { int n = 1; constexpr X() {} };
struct Meow { X x; char padding[64]; };
Meow should_be_statically_initalized[256];

Prior to C++20, this example required dynamic initialization: the constructor of Meow was non->constexpr due to the uninitialized member.

P1331R2 changed that, and gave Meow a constexpr constructor. But we still don't get static >initialization here, because CWG2026 changed the rules for constant initialization so that it >replaces zero-initialization instead of happening after zero-initialization.

so it looks like the workaround @AlexGuteniev posted is the correct workaround for both c1xx and clang.

@StephanTLavavej
Copy link
Member

Is this purely a pair of compiler bugs with no STL changes required? If so, should we close this as external resolved? (In any event, thanks for the report!)

@CaseyCarter CaseyCarter added the bug Something isn't working label Apr 2, 2020
@CaseyCarter
Copy link
Member

There are a couple of issues here:

  1. All specializations of std::atomic now have non-trivial default constructors. This is very much by design and not a bug.

  2. Some objects that have specializations of std::atomic as subobjects which were previously constant-initialized are now being dynamically-initialized. This is a bug in the compilers and/or a defect in the C++ Standard, but it's not clear to me which.

I'd like to see the exact nature of the bug clarified before we close this.

@Neumann-A
Copy link

@BillyONeal
Copy link
Member

@Neumann-A Thanks for running that down

@BillyONeal
Copy link
Member

BillyONeal commented Jun 24, 2020

I stand by what I said above in #661 (comment) that I think we should implement the change unconditionally, but it has now caused regressions in both LLVM and Qt, and was independently reported as DevCom-1089003 . Should we consider trying to add a special case for C++17 and earlier mode, only for lock free atomics, which default initializes the T and thus can be trivial again?

Customers calling that are still breaking the standard's rules but the standard's rules were unreasonable.

@StephanTLavavej
Copy link
Member

The purpose of Standard modes is to give customers time to adapt to source-breaking changes. While I don't think it's a great idea to provide default-initializing trivial constructors in C++14/17 mode (if those Standards didn't require triviality), this is exactly how I feel about compiling in C++14/17 mode at all.

Therefore, I am in favor of mitigating the source-breaking change by adding a special case for C++14/17 mode. Affected customers should be aware that we will never extend this to C++20 mode.

kevinAlbs pushed a commit to mongodb/mongo-cxx-driver that referenced this issue Jun 24, 2020
* Disable trivially_constructible test for atomic on MSVC++.

MSVC++ implements P0883 unconditionally, which changes the rules for std::atomic. It removes atomic's trivial constructor, and makes the default constructor value initialize the T.

Note that Mongo was not following the C++11 rules, because it used the atomic before calling atomic_init first. MSVC++ never implemented the C++11 rules and previously default initialized the T.

All versions of MSVC++ will provide constant initialization of the guarded value "current_instance". In old versions, atomic didn't implement P0883 due to bugs in the constexpr evaluator; in current versions the constexpr evaluator was fixed and atomic value initializes unconditionally. Therefore, this PR disables the check whenever MSVC++'s standard library is detected.

See microsoft/STL#661 for further discussion.

* Completely remove the assertion since it does not assert anything about current_instance.
@MikeGitb
Copy link

As a practical question for me:

Is there any combination of (released) MSVC version and standard version flag under which the syntax std::atomic<int> v{} does not initialize the atomic to zero and/or where a global v isn't put into the BSS?

@BillyONeal
Copy link
Member

@MikeGitb No. The only place this is observable is when mixing the atomic with other things where zero initialization is expected, such as:

struct aggregate {
    atomic<int> a;
    int b; // to make constexpr, add {}s or =0 here
};

aggregate g_instance;

@MikeGitb
Copy link

@BillyONeal : Thank you very much.

qtprojectorg pushed a commit to qt/qtbase that referenced this issue Jun 30, 2020
Testcase:

 struct A
 {
     A() = default;
     constexpr A(int v) :i(v) {}
     int i;
 };
 extern const A y[1] = {};

In our case, A = std::atomic<int> and y = shared_null.

With GCC, ICC, and Clang that "y" variable is value-initialized at
static initialization time, and no dynamic initialization code is
generated. However, with MSVC, because A is not an aggregate, the
default constructor isn't constexpr (it leaves A::i uninitialized) so
"y" must be dynamically initialized. That leads to Static Initialization
Order Fiasco.

This seems to be a regression in the MSVC 2019 16.6 STL:
microsoft/STL#661

The solution is simple: call the constexpr constructor.

Code is different in 6.0 so sending separately from 5.x.

Fixes: QTBUG-71548
Task-number: QTBUG-59721
Pick-to: 5.12
Change-Id: I3d4f433ff6e94fd390a9fffd161b4a7e8c1867b1
Reviewed-by: Kai Koehne <kai.koehne@qt.io>
qtprojectorg pushed a commit to qt/qtbase that referenced this issue Jun 30, 2020
Testcase:

 struct A
 {
     A() = default;
     constexpr A(int v) :i(v) {}
     int i;
 };
 extern const A y[1] = {};

In our case, A = std::atomic<int> and y = shared_null.

With GCC, ICC, and Clang that "y" variable is value-initialized at
static initialization time, and no dynamic initialization code is
generated. However, with MSVC, because A is not an aggregate, the
default constructor isn't constexpr (it leaves A::i uninitialized) so
"y" must be dynamically initialized. That leads to Static Initialization
Order Fiasco.

This seems to be a regression in the MSVC 2019 16.6 STL:
microsoft/STL#661

The solution is simple: call the constexpr constructor.

Code is different in 6.0 so sending separately from 5.x.

Fixes: QTBUG-71548
Task-number: QTBUG-59721
Change-Id: I3d4f433ff6e94fd390a9fffd161b4a7e8c1867b1
Reviewed-by: Kai Koehne <kai.koehne@qt.io>
(cherry picked from commit b619f2a)
Reviewed-by: Thiago Macieira <thiago.macieira@intel.com>
@jwatte
Copy link

jwatte commented Jul 16, 2020

Customers calling that are still breaking the standard's rules but the standard's rules were unreasonable.

From the peanut gallery: Yes, the standards rules are unreasonable.
A good remedy might be to make the old behavior exist for trivially initializable types, and only do the constructor when the contained type is non-trivial. (Someone with way too much free time could also try to do all the lobbying bullshit necessary to make the standard change to something like that)

@BillyONeal
Copy link
Member

A good remedy might be to make the old behavior exist for trivially initializable types

The "old behavior" isn't even the behavior you expected: our atomic didn't have a trivial ctor for some Ts (namely, all non is_always_lockfree Ts).

only do the constructor when the contained type is non-trivial

That would (at this point) be a breaking change and also a scheme that never happened: under both the old and the new rules there is no such thing as an atomic with a default-initialized T:

  • under the '11 rules you get value initialization by calling atomic_init
  • under the '20 rules you get value initialization by calling the constructor

(Someone with way too much free time could also try to do all the lobbying bullshit necessary to make the standard change to something like that)

I argued for default initialization rather than value initialization in the constructor and the committee said they want value init, so even were it not a breaking change I would not expect success.

blueboxd pushed a commit to blueboxd/chromium-legacy that referenced this issue Nov 18, 2021
In C++20 constructing an std::atomic is not 'trivial'. The various
reasons for that are reachable from this discussion: [1]. The change
will prevent the array from being put to BSS and be cheap when unused.

Make the array a regular uint32_t array and refer to it as an array of
std::atomic later (using reinterpret_cast). The std::atomic has the
standard layout by the spec, making it hard to imagine an implementation
of C++ that would construct the std::atomic<uint32_t> to anything
different than (uint32_t)0.

The Linux/Android memory page zero-fill is atomic and can only be done
once. Therefore the new behavior should be equivalent to the current
one. The code generated by Clang today is identical with/without this
patch.

This large chunk of zeroes in BSS still _may_ have a performance or code
size impact because of increased immediates in relative references to
.data and .bss. Resolving it may be even more controversial, hence
suggesting to do it only if there is a confirmed regression.

Disclaimer: I know this is .. bad. And I feel bad.

[1] Discussion: <atomic>: In pre-C++20 mode, the constructor should be
    trivial
    microsoft/STL#661

Bug: None
Change-Id: I06798f225bd557844072d154d47ace2d85606df5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3282196
Reviewed-by: Alex Ilin <alexilin@chromium.org>
Commit-Queue: Egor Pasko <pasko@chromium.org>
Cr-Commit-Position: refs/heads/main@{#943056}
zeng450026937 pushed a commit to zeng450026937/base that referenced this issue Nov 19, 2021
In C++20 constructing an std::atomic is not 'trivial'. The various
reasons for that are reachable from this discussion: [1]. The change
will prevent the array from being put to BSS and be cheap when unused.

Make the array a regular uint32_t array and refer to it as an array of
std::atomic later (using reinterpret_cast). The std::atomic has the
standard layout by the spec, making it hard to imagine an implementation
of C++ that would construct the std::atomic<uint32_t> to anything
different than (uint32_t)0.

The Linux/Android memory page zero-fill is atomic and can only be done
once. Therefore the new behavior should be equivalent to the current
one. The code generated by Clang today is identical with/without this
patch.

This large chunk of zeroes in BSS still _may_ have a performance or code
size impact because of increased immediates in relative references to
.data and .bss. Resolving it may be even more controversial, hence
suggesting to do it only if there is a confirmed regression.

Disclaimer: I know this is .. bad. And I feel bad.

[1] Discussion: <atomic>: In pre-C++20 mode, the constructor should be
    trivial
    microsoft/STL#661

Bug: None
Change-Id: I06798f225bd557844072d154d47ace2d85606df5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3282196
Reviewed-by: Alex Ilin <alexilin@chromium.org>
Commit-Queue: Egor Pasko <pasko@chromium.org>
Cr-Commit-Position: refs/heads/main@{#943056}
NOKEYCHECK=True
GitOrigin-RevId: 50ed626cf795bf056fbd96037b457934ab36bc40
qoor pushed a commit to qoor/basium that referenced this issue Apr 7, 2022
In C++20 constructing an std::atomic is not 'trivial'. The various
reasons for that are reachable from this discussion: [1]. The change
will prevent the array from being put to BSS and be cheap when unused.

Make the array a regular uint32_t array and refer to it as an array of
std::atomic later (using reinterpret_cast). The std::atomic has the
standard layout by the spec, making it hard to imagine an implementation
of C++ that would construct the std::atomic<uint32_t> to anything
different than (uint32_t)0.

The Linux/Android memory page zero-fill is atomic and can only be done
once. Therefore the new behavior should be equivalent to the current
one. The code generated by Clang today is identical with/without this
patch.

This large chunk of zeroes in BSS still _may_ have a performance or code
size impact because of increased immediates in relative references to
.data and .bss. Resolving it may be even more controversial, hence
suggesting to do it only if there is a confirmed regression.

Disclaimer: I know this is .. bad. And I feel bad.

[1] Discussion: <atomic>: In pre-C++20 mode, the constructor should be
    trivial
    microsoft/STL#661

Bug: None
Change-Id: I06798f225bd557844072d154d47ace2d85606df5
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/3282196
Reviewed-by: Alex Ilin <alexilin@chromium.org>
Commit-Queue: Egor Pasko <pasko@chromium.org>
Cr-Commit-Position: refs/heads/main@{#943056}
theartful added a commit to theartful/broadcast_queue that referenced this issue May 11, 2023
@frederick-vs-ja
Copy link
Contributor

CWG-2536 seems to be related.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

9 participants