Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[TableGen] Print memory stats in detailed record emitter #106990

Merged
merged 1 commit into from
Sep 4, 2024
Merged

[TableGen] Print memory stats in detailed record emitter #106990

merged 1 commit into from
Sep 4, 2024

Conversation

jurahul
Copy link
Contributor

@jurahul jurahul commented Sep 2, 2024

Print memory allocation and related statistics when dumping detailed record information.

@jurahul jurahul requested a review from River707 September 2, 2024 18:26
@jurahul jurahul marked this pull request as ready for review September 2, 2024 18:26
@jurahul jurahul requested a review from arsenm September 2, 2024 18:26
@jurahul
Copy link
Contributor Author

jurahul commented Sep 2, 2024

The motivation was #106986 where I wanted to see how much reduction in memory footprint the change results in.

arsenm
arsenm approved these changes Sep 4, 2024
View reviewed changes
llvm/lib/TableGen/Record.cpp Outdated Show resolved Hide resolved
@jurahul
[TableGen] Print memory stats in detailed record emitter
4ffc61e 
Print memory allocation and related statistics when dumping detailed
record information.
@jurahul jurahul force-pushed the alloc_status_in_detailed_records_emitter branch from 58451fb to 4ffc61e Compare September 4, 2024 12:25
@jurahul jurahul merged commit 30f1cfb into llvm:main Sep 4, 2024
8 checks passed
@jurahul jurahul deleted the alloc_status_in_detailed_records_emitter branch September 4, 2024 14:04
@llvm-ci
Copy link
Collaborator

llvm-ci commented Sep 4, 2024

LLVM Buildbot has detected a new failure on builder sanitizer-x86_64-linux running on sanitizer-buildbot1 while building llvm at step 2 "annotate".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/66/builds/3472

Here is the relevant piece of the build log for the reference 
Step 2 (annotate) failure: 'python ../sanitizer_buildbot/sanitizers/zorg/buildbot/builders/sanitizers/buildbot_selector.py' (failure)
...
[379/384] Generating Fuzzer-x86_64-Test
[380/384] Generating MSAN_INST_TEST_OBJECTS.msan_test.cpp.x86_64-with-call.o
[381/384] Generating Msan-x86_64-with-call-Test
[382/384] Generating MSAN_INST_TEST_OBJECTS.msan_test.cpp.x86_64.o
[383/384] Generating Msan-x86_64-Test
[383/384] Running compiler_rt regression tests
llvm-lit: /home/b/sanitizer-x86_64-linux/build/llvm-project/llvm/utils/lit/lit/main.py:72: note: The test suite configuration requested an individual test timeout of 0 seconds but a timeout of 900 seconds was requested on the command line. Forcing timeout to be 900 seconds.
-- Testing: 10229 tests, 88 workers --
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
TIMEOUT: SanitizerCommon-hwasan-x86_64-Linux :: Posix/fork_threaded.c (10229 of 10229)
******************** TEST 'SanitizerCommon-hwasan-x86_64-Linux :: Posix/fork_threaded.c' FAILED ********************
Exit Code: 137
Timeout: Reached timeout of 900 seconds

Command Output (stderr):
--
RUN: at line 1: /home/b/sanitizer-x86_64-linux/build/build_default/./bin/clang  -gline-tables-only -fsanitize=hwaddress -fuse-ld=lld -fsanitize-hwaddress-experimental-aliasing  -m64 -funwind-tables  -I/home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/test -ldl -O0 /home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/test/sanitizer_common/TestCases/Posix/fork_threaded.c -o /home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp && env HWASAN_OPTIONS=die_after_fork=0  /home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp
+ /home/b/sanitizer-x86_64-linux/build/build_default/./bin/clang -gline-tables-only -fsanitize=hwaddress -fuse-ld=lld -fsanitize-hwaddress-experimental-aliasing -m64 -funwind-tables -I/home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/test -ldl -O0 /home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/test/sanitizer_common/TestCases/Posix/fork_threaded.c -o /home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp
+ env HWASAN_OPTIONS=die_after_fork=0 /home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp
==94351==ERROR: HWAddressSanitizer: invalid-free on address 0x680200000020 at pc 0x5e505309cbe7 on thread T0
tags: 01/04 (ptr/mem)
/home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.script: line 1: 94276 Killed                  env HWASAN_OPTIONS=die_after_fork=0 /home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp

--

********************
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90.. 
Slowest Tests:
--------------------------------------------------------------------------
900.07s: SanitizerCommon-hwasan-x86_64-Linux :: Posix/fork_threaded.c
142.93s: libFuzzer-x86_64-default-Linux :: out-of-process-fuzz.test
139.84s: libFuzzer-x86_64-libcxx-Linux :: out-of-process-fuzz.test
133.94s: libFuzzer-x86_64-static-libcxx-Linux :: out-of-process-fuzz.test
51.88s: libFuzzer-i386-libcxx-Linux :: value-profile-switch.test
44.96s: libFuzzer-i386-libcxx-Linux :: fork.test
42.94s: libFuzzer-i386-default-Linux :: value-profile-switch.test
42.01s: libFuzzer-x86_64-static-libcxx-Linux :: fork_corpus_groups.test
41.42s: libFuzzer-i386-default-Linux :: fork_corpus_groups.test
41.38s: libFuzzer-i386-static-libcxx-Linux :: value-profile-switch.test
41.35s: libFuzzer-i386-libcxx-Linux :: fork_corpus_groups.test
40.65s: libFuzzer-i386-static-libcxx-Linux :: fork.test
40.64s: libFuzzer-i386-default-Linux :: fork.test
40.43s: libFuzzer-i386-libcxx-Linux :: disable-leaks.test
40.25s: libFuzzer-x86_64-default-Linux :: fork_corpus_groups.test
39.90s: libFuzzer-i386-static-libcxx-Linux :: fork_corpus_groups.test
39.48s: libFuzzer-x86_64-libcxx-Linux :: fork_corpus_groups.test
39.33s: libFuzzer-x86_64-libcxx-Linux :: fork.test
39.04s: libFuzzer-x86_64-static-libcxx-Linux :: fork.test
39.02s: libFuzzer-x86_64-default-Linux :: fork.test
Step 9 (test compiler-rt symbolizer) failure: test compiler-rt symbolizer (failure)
...
[379/384] Generating Fuzzer-x86_64-Test
[380/384] Generating MSAN_INST_TEST_OBJECTS.msan_test.cpp.x86_64-with-call.o
[381/384] Generating Msan-x86_64-with-call-Test
[382/384] Generating MSAN_INST_TEST_OBJECTS.msan_test.cpp.x86_64.o
[383/384] Generating Msan-x86_64-Test
[383/384] Running compiler_rt regression tests
llvm-lit: /home/b/sanitizer-x86_64-linux/build/llvm-project/llvm/utils/lit/lit/main.py:72: note: The test suite configuration requested an individual test timeout of 0 seconds but a timeout of 900 seconds was requested on the command line. Forcing timeout to be 900 seconds.
-- Testing: 10229 tests, 88 workers --
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90..
TIMEOUT: SanitizerCommon-hwasan-x86_64-Linux :: Posix/fork_threaded.c (10229 of 10229)
******************** TEST 'SanitizerCommon-hwasan-x86_64-Linux :: Posix/fork_threaded.c' FAILED ********************
Exit Code: 137
Timeout: Reached timeout of 900 seconds

Command Output (stderr):
--
RUN: at line 1: /home/b/sanitizer-x86_64-linux/build/build_default/./bin/clang  -gline-tables-only -fsanitize=hwaddress -fuse-ld=lld -fsanitize-hwaddress-experimental-aliasing  -m64 -funwind-tables  -I/home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/test -ldl -O0 /home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/test/sanitizer_common/TestCases/Posix/fork_threaded.c -o /home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp && env HWASAN_OPTIONS=die_after_fork=0  /home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp
+ /home/b/sanitizer-x86_64-linux/build/build_default/./bin/clang -gline-tables-only -fsanitize=hwaddress -fuse-ld=lld -fsanitize-hwaddress-experimental-aliasing -m64 -funwind-tables -I/home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/test -ldl -O0 /home/b/sanitizer-x86_64-linux/build/llvm-project/compiler-rt/test/sanitizer_common/TestCases/Posix/fork_threaded.c -o /home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp
+ env HWASAN_OPTIONS=die_after_fork=0 /home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp
==94351==ERROR: HWAddressSanitizer: invalid-free on address 0x680200000020 at pc 0x5e505309cbe7 on thread T0
tags: 01/04 (ptr/mem)
/home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.script: line 1: 94276 Killed                  env HWASAN_OPTIONS=die_after_fork=0 /home/b/sanitizer-x86_64-linux/build/build_default/runtimes/runtimes-bins/compiler-rt/test/sanitizer_common/hwasan-x86_64-Linux/Posix/Output/fork_threaded.c.tmp

--

********************
Testing:  0.. 10.. 20.. 30.. 40.. 50.. 60.. 70.. 80.. 90.. 
Slowest Tests:
--------------------------------------------------------------------------
900.07s: SanitizerCommon-hwasan-x86_64-Linux :: Posix/fork_threaded.c
142.93s: libFuzzer-x86_64-default-Linux :: out-of-process-fuzz.test
139.84s: libFuzzer-x86_64-libcxx-Linux :: out-of-process-fuzz.test
133.94s: libFuzzer-x86_64-static-libcxx-Linux :: out-of-process-fuzz.test
51.88s: libFuzzer-i386-libcxx-Linux :: value-profile-switch.test
44.96s: libFuzzer-i386-libcxx-Linux :: fork.test
42.94s: libFuzzer-i386-default-Linux :: value-profile-switch.test
42.01s: libFuzzer-x86_64-static-libcxx-Linux :: fork_corpus_groups.test
41.42s: libFuzzer-i386-default-Linux :: fork_corpus_groups.test
41.38s: libFuzzer-i386-static-libcxx-Linux :: value-profile-switch.test
41.35s: libFuzzer-i386-libcxx-Linux :: fork_corpus_groups.test
40.65s: libFuzzer-i386-static-libcxx-Linux :: fork.test
40.64s: libFuzzer-i386-default-Linux :: fork.test
40.43s: libFuzzer-i386-libcxx-Linux :: disable-leaks.test
40.25s: libFuzzer-x86_64-default-Linux :: fork_corpus_groups.test
39.90s: libFuzzer-i386-static-libcxx-Linux :: fork_corpus_groups.test
39.48s: libFuzzer-x86_64-libcxx-Linux :: fork_corpus_groups.test
39.33s: libFuzzer-x86_64-libcxx-Linux :: fork.test
39.04s: libFuzzer-x86_64-static-libcxx-Linux :: fork.test
39.02s: libFuzzer-x86_64-default-Linux :: fork.test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@arsenm arsenm arsenm approved these changes

@River707 River707 Awaiting requested review from River707

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jurahul @llvm-ci @arsenm