[clang-format] Crash with Whitesmiths and labels in top-level block

[oold]

Bugzilla Link	50663
Resolution	FIXED
Resolved on	Jun 16, 2021 00:04
Version	12.0
OS	All
Blocks	#48661
Attachments	Zipped goto.cpp and switch.cpp files
CC	@mydeveloperday,@oold,@tstellar
Fixed by commit(s)	f7f9f94 c7d7ace

Extended Description

Formatting a file with a label within a top-level block causes clang-format to hang for an extended period of time and use a lot of memory. It then proceeds to crash:

PS C:\Users\olive\clang-format-repro> clang-format.exe --style='{BreakBeforeBraces: Whitesmiths}' goto.cpp
PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.
Stack dump:
0. Program arguments: "C:\Program Files\LLVM\bin\clang-format.exe" "--style={BreakBeforeBraces: Whitesmiths}" goto.cpp
PS C:\Users\olive\clang-format-repro> cat goto.cpp
int main() {
test:
return 0;
}

No actual backtrace is printed.

I have prepared test cases with both a simple function and a switch statement and both lead to a crash.

My suspicion is that this was caused by the patch that tried to fix case labels with Whitesmiths [1]. This bug does not occur with version 11.0 of clang-format.

[1] https://reviews.llvm.org/D94500

[mydeveloperday]

I tested this with 11 and 12 and can confirm it does indeed hang,

I tested this with my latest build and it doesn't, but I build debug so its possible it doesn't manifest itself.

[mydeveloperday]

I tested Debug and Release builds of the current tip of the main (Currently 13) and I cannot reproduce.

I checked out the 12.0.1 branches and tested both debug and release

12.0.1 does indeed still crash (Debug and Release), my guess its more a crash caused by out of memory as the process rapid grows to many GB

Just to clarify my previous comment 11 GA was fine 12 GA was not.

I'm not sure if this was somehow a known issue, I don't remember it between 12 and 13 especially for clang-format.

[mydeveloperday]

Did eventually get a stack from 12.0.1

/buildareas/clang/build_ninja/bin/clang-format.exe test1.cpp
Assertion failed: Index < Length && "Invalid index!", file C:\cygwin64\buildareas\clang\llvm-project\llvm\include\llvm/ADT/ArrayRef.h, line 248
PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.
Stack dump:
0. Program arguments: C:\cygwin64\buildareas\clang\build_ninja\bin\clang-format.exe test1.cpp
0x00007FF7B16DC89C (0x000000C600000016 0x0000000000000000 0x00007FF7B15A0C85 0x0000000000000000), HandleAbort() + 0xC bytes(s), C:\cygwin64\buildareas\clang\llvm-project
llvm\lib\Support\Windows\Signals.inc, line 408
0x00007FFF473CBC31 (0x00007FFF00000016 0x000000C617389A70 0x0000000801000006 0xFFFFFF0100000120), raise() + 0x441 bytes(s)
0x00007FFF473CD889 (0x000000C617389A70 0x0000000000000240 0x00007FFF474CDA70 0x00007FF7B1FD8020), abort() + 0x39 bytes(s)
0x00007FFF473D34B5 (0x00007FF7B1FD8020 0x00007FF7B1FDA130 0x00007FFF000000F8 0x0000000000000000), _get_wide_winmain_command_line() + 0x2515 bytes(s)
0x00007FFF473D3027 (0x00007FF7B1FD8020 0x00007FF7B1FDA130 0x00000248000000F8 0x0000024C9D4B0080), _get_wide_winmain_command_line() + 0x2087 bytes(s)
0x00007FFF473D1091 (0x00007FF7B1FD8020 0x00007FF7B1FDA130 0x0000024C000000F8 0x00007FF7B18F7E99), _get_wide_winmain_command_line() + 0xF1 bytes(s)
0x00007FFF473D3A1F (0x00007FF7B1FD8020 0x00007FF7B1FDA130 0xCCCCCCCC000000F8 0xCCCCCCCCCCCCCCCC), _wassert() + 0x2F bytes(s)
0x00007FF7B18F7E99 (0x000000C61738A0A0 0x00000000FFFFFFFF 0xCCCCCCCCCCCCCCCC 0xCCCCCCCCCCCCCCCC), llvm::ArrayRef::operator + 0x59 bytes(s), C:\cygwin64\buildarea
s\clang\llvm-project\llvm\include\llvm\ADT\ArrayRef.h, line 248 + 0x36 byte(s)
0x00007FF7B18FE3E9 (0x000000C61738A210 0x000000C61738A0A0 0xCCCCCCCCFFFFFFFF 0xCCCCCCCCCCCCCCCC), clang::format::anonymous namespace'::LevelIndentTracker::getIndent() + 0x39 bytes(s), C:\cygwin64\buildareas\clang\llvm-project\clang\lib\Format\UnwrappedLineFormatter.cpp, line 115 + 0x10 byte(s) 0x00007FF7B18FF01A (0x000000C61738A210 0x000002455CA2FC80 0xCCCCCCCCCCCCCCCC 0xCCCCCCCCCCCCCCCC), clang::format::anonymous namespace'::LevelIndentTracker::nextLine() + 0
x11A bytes(s), C:\cygwin64\buildareas\clang\llvm-project\clang\lib\Format\UnwrappedLineFormatter.cpp, line 63 + 0x4A byte(s)
0x00007FF7B18FE642 (0x000000C61738A168 0x000002455CA2FD00 0x000000C61738A210 0x000000C61738C408), clang::format::anonymous namespace'::LineJoiner::getNextMergedLine() + 0x62 bytes(s), C:\cygwin64\buildareas\clang\llvm-project\clang\lib\Format\UnwrappedLineFormatter.cpp, line 178 0x00007FF7B18EEBB0 (0x000000C61738B300 0x000000C61738C408 0x000000C61738A400 0x000000C600000000), clang::format::UnwrappedLineFormatter::format() + 0x550 bytes(s), C:\cyg win64\buildareas\clang\llvm-project\clang\lib\Format\UnwrappedLineFormatter.cpp, line 1148 + 0x1A byte(s) 0x00007FF7B1874229 (0x000000C61738C780 0x000000C61738C4E8 0x000000C61738C4B8 0x000000C61738C408), clang::format::anonymous namespace'::Formatter::analyze() + 0x379 bytes
(s), C:\cygwin64\buildareas\clang\llvm-project\clang\lib\Format\Format.cpp, line 1584 + 0x105 byte(s)
0x00007FF7B18D1B04 (0x000000C61738C780 0x000000C61738D8C8 0x000000C61738D3A0 0x000000C61738E188), clang::format::TokenAnalyzer::process() + 0x4F4 bytes(s), C:\cygwin64\bu
ildareas\clang\llvm-project\clang\lib\Format\TokenAnalyzer.cpp, line 93
0x00007FF7B186B24C (0x000000C61738D760 0x000000C61738D8C8 0x000002455CA36F00 0xCCCCCCCCCCCCCCCC), <lambda_be157eee42e494a9ed92b4bd992e61c3>::operator()() + 0x7C bytes(s),
C:\cygwin64\buildareas\clang\llvm-project\clang\lib\Format\Format.cpp, line 2729 + 0x3C byte(s)
0x00007FF7B185038C (0x000000C61738D8C8 0x000000C61738D760 0x000002455CA36F00 0xCCCCCCCCCCCCCCCC), std::invoke<<lambda_be157eee42e494a9ed92b4bd992e61c3> &,clang::format::E
nvironment const &>() + 0x3C bytes(s), C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29910\include\type_traits, line 1596 + 0x14 byte
(s)
0x00007FF7B183F04C (0x000000C61738D8C8 0x000000C61738D760 0x000002455CA36F00 0xCCCCCCCCCCCCCCCC), std::_Invoker_ret<std::pair<clang::tooling::Replacements,unsigned int>,0

::_Call<<lambda_be157eee42e494a9ed92b4bd992e61c3> &,clang::format::Environment const &>() + 0x3C bytes(s), C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise
\VC\Tools\MSVC\14.28.29910\include\functional, line 753 + 0x14 byte(s)
0x00007FF7B186EB6B (0x000000C61738D758 0x000000C61738D8C8 0x000002455CA36F00 0xCCCCCCCCCCCCCCCC), std::_Func_impl_no_alloc<<lambda_be157eee42e494a9ed92b4bd992e61c3>,std::
pair<clang::tooling::Replacements,unsigned int>,clang::format::Environment const &>::_Do_call() + 0x4B bytes(s), C:\Program Files (x86)\Microsoft Visual Studio\2019\Enter
prise\VC\Tools\MSVC\14.28.29910\include\functional, line 938 + 0x23 byte(s)
0x00007FF7B186B5C9 (0x000000C61738D758 0x000000C61738D8C8 0x000002455CA36F00 0x000000C61738DA40), std::_Func_class<std::pair<clang::tooling::Replacements,unsigned int>,cl
ang::format::Environment const &>::operator()() + 0x79 bytes(s), C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29910\include\function
al, line 986 + 0x2C byte(s)
0x00007FF7B18316A9 (0x000000C61738DB50 0x000000C61738DD78 0x000000C61738DB70 0x000000C61738DB80), clang::format::internal::reformat() + 0x499 bytes(s), C:\cygwin64\builda
reas\clang\llvm-project\clang\lib\Format\Format.cpp, line 2746
0x00007FF7B182F5F0 (0x000000C61738E1A8 0x000000C61738DD78 0x000000C61738F370 0x000000C61738F380), clang::format::reformat() + 0xD0 bytes(s), C:\cygwin64\buildareas\clang
llvm-project\clang\lib\Format\Format.cpp, line 2769 + 0x9C byte(s)
0x00007FF7B15F96A6 (0x000000C61738F6B0 0x000002455CA2BA10 0x000000C61738F690 0x0000000000000000), clang::format::format() + 0xC16 bytes(s), C:\cygwin64\buildareas\clang\l
lvm-project\clang\tools\clang-format\ClangFormat.cpp, line 420
0x00007FF7B15FA7C1 (0x0000000000000002 0x000002455CA21860 0x0000000000000000 0x00007FF7B1ABB8ED), main() + 0x421 bytes(s), C:\cygwin64\buildareas\clang\llvm-project\clang
\tools\clang-format\ClangFormat.cpp, line 535 + 0x56 byte(s)
0x00007FF7B1ABC3E9 (0x00007FF7B1FD1000 0x00007FF7B1FD16F0 0x0000000000000000 0x0000000000000000), invoke_main() + 0x39 bytes(s), D:\a01_work\26\s\src\vctools\crt\vcstart
up\src\startup\exe_common.inl, line 79
0x00007FF7B1ABC2CE (0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000), __scrt_common_main_seh() + 0x12E bytes(s), D:\a01_work\26\s\src\vctools
\crt\vcstartup\src\startup\exe_common.inl, line 288 + 0x5 byte(s)
0x00007FF7B1ABC18E (0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000), __scrt_common_main() + 0xE bytes(s), D:\a01_work\26\s\src\vctools\crt\v
cstartup\src\startup\exe_common.inl, line 331
0x00007FF7B1ABC47E (0x000000C61697C000 0x0000000000000000 0x0000000000000000 0x0000000000000000), mainCRTStartup() + 0xE bytes(s), D:\a01_work\26\s\src\vctools\crt\vcsta
rtup\src\startup\exe_main.cpp, line 17
0x00007FFF90687034 (0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000), BaseThreadInitThunk() + 0x14 bytes(s)
0x00007FFF91942651 (0x0000000000000000 0x0000000000000000 0x0000000000000000 0x0000000000000000), RtlUserThreadStart() + 0x21 bytes(s)
make: *** [Makefile:24: all] Error 3

[mydeveloperday]

I applied https://reviews.llvm.org/D94500 to the 12.0.1 branch and this fixes your issue.

Can we consider this issue fixed? for LLVM 13.0

I would prefer that @​timwoj fixed this in the branch if that's what needs to be done before 13 is released

[oold]

Would be nice to see an early fix since this is obviously broken and shouldn't crash the formatter. As long as this issue remains, I will not be able to adopt clang-format for my code.

[tstellar]

The fix does not apply cleanly, could someone backport this and push a branch to their local github fork?

[mydeveloperday]

Tom am I allowed to simply commit/push it directly into the branch?

do I need to go back round the Phabricator review cycle? if the changes to get into the branch are minimal?

[mydeveloperday]

I should have mentioned, I don't use a fork.

[tstellar]

Tom am I allowed to simply commit/push it directly into the branch?

Yes, if you can backport the patch cleanly, go ahead and push it. Just make sure to use git cherry-pick -x so that the commit in main is referenced in the commit log and make sure the lit tests pass.

[mydeveloperday]

Ok its pushed to the 12.x branch (cherry picked -x), for the life of me I hope I've done it correctly, the unit tests all pass.

c7d7ace

If I've messed anything up, I apologize in advance, but I guess we all have to do something scary the first time

Thanks for your help.

[mydeveloperday]

Is the a branch build bot?

[tstellar]

Is the a branch build bot?

Yes, there are branch buildbots. They report status to github, so if you click on the commit in the web interface, you should be able to see the status.

[tstellar]

Merged: c7d7ace

[mydeveloperday]

Thanks Tom for helping me out (I got my green tick in the end, took a while...)

Thank you for everything you do.