Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optionally enable LLVM profile continuous mode #15166

Closed
wants to merge 1 commit into from
Closed

Optionally enable LLVM profile continuous mode #15166

wants to merge 1 commit into from

Conversation

fmeum
Copy link
Collaborator

@fmeum fmeum commented Apr 3, 2022

LLVM's profile instrumentation offers a continuous mode in which
counters are continuously synced to a file rather than being dumped
once in an exit hook. This is useful for coverage runs that include
binaries that exit abnormally (e.g. in failure tests), but may require
additional compiler flags and can negatively impact runtime performance
and memory usage.

Enabling continuous mode requires adding the "%c" modifier to the value
of the LLVM_PROFILE_FILE environment variable. With this commit, the
collect_coverage.sh script adds the modifier if the test environment has
the variable LLVM_PROFILE_CONTINUOUS_MODE set. This allows both all and
individual tests to use continuous mode by setting the variable, either
via --test_env or the env attribute.

@fmeum
Optionally enable LLVM profile continuous mode
f3d2fcd 
LLVM's profile instrumentation offers a continuous mode in which
counters are continuously synced to a file rather than being dumped
once in an exit hook. This is useful for coverage runs that include
binaries that exit abnormally (e.g. in failure tests), but may require
additional compiler flags and can negatively impact runtime performance
and memory usage.

Enabling continuous mode requires adding the "%c" modifier to the value
of the LLVM_PROFILE_FILE environment variable. With this commit, the
collect_coverage.sh script adds the modifier if the test environment has
the variable LLVM_PROFILE_CONTINUOUS_MODE set. This allows both all and
individual tests to use continuous mode by setting the variable, either
via --test_env or the env attribute.
@fmeum fmeum requested a review from lberki as a code owner April 3, 2022 14:30
@fmeum
Copy link
Collaborator Author

fmeum commented Apr 3, 2022

@c-mita This is a minimal change required to make LLVM profile's continuous mode usable with Bazel at all (everything else can be configured via existing knobs). Since I need this mode to properly generate complete coverage reports for Jazzer, I am very interested in getting at least this change into Bazel.

If you would appreciate a better integration, I could submit a follow-up PR that adds the required compiler flags as toolchain features together with an experimental flag, similar to the existing --experimental_use_llvm_covmap.

@sgowroji sgowroji added the team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. label Apr 5, 2022
@meteorcloudy meteorcloudy added team-Rules-CPP Issues for C++ rules and removed team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. labels Apr 12, 2022
@fmeum
Copy link
Collaborator Author

fmeum commented Apr 19, 2022

@c-mita Sorry for the high PR load, but I assume this one is also yours to review

fmeum added a commit to CodeIntelligenceTesting/jazzer that referenced this pull request Apr 20, 2022
@fmeum
Generate end-to-end coverage reports for tests
7600aa8 
Coverage reports can be generated via:

  bazel run //bazel/coverage

Due to bazelbuild/bazel#15166, coverage
collection does not yet encompass tests that end with a native sanitizer
report.
fmeum added a commit to CodeIntelligenceTesting/jazzer that referenced this pull request Apr 20, 2022
@fmeum
Generate end-to-end coverage reports for tests
dc772e9 
Coverage reports can be generated via:

  bazel run //bazel/coverage

Due to bazelbuild/bazel#15166, coverage
collection does not yet encompass tests that end with a native sanitizer
report.

Temporarily updates .bazelversion to an unreleased version of Bazel that
includes required coverage fixes.
fmeum added a commit to CodeIntelligenceTesting/jazzer that referenced this pull request Apr 20, 2022
@fmeum
Generate end-to-end coverage reports for tests
346f69d 
Coverage reports can be generated via:

  bazel run //bazel/coverage

Due to bazelbuild/bazel#15166, coverage
collection does not yet encompass tests that end with a native sanitizer
report.

Temporarily updates .bazelversion to an unreleased version of Bazel that
includes required coverage fixes.
@sgowroji sgowroji added the awaiting-review PR is awaiting review from an assigned reviewer label Apr 21, 2022
@fmeum
Copy link
Collaborator Author

fmeum commented May 4, 2022

@c-mita Friendly ping

@sgowroji sgowroji requested a review from c-mita May 5, 2022 11:09
fmeum added a commit to CodeIntelligenceTesting/jazzer that referenced this pull request May 20, 2022
@fmeum
Generate end-to-end coverage reports for tests
862f22a 
Coverage reports can be generated via:

  bazel run //bazel/coverage

Due to bazelbuild/bazel#15166, coverage
collection does not yet encompass tests that end with a native sanitizer
report.

Temporarily updates .bazelversion to an unreleased version of Bazel that
includes required coverage fixes.
fmeum added a commit to CodeIntelligenceTesting/jazzer that referenced this pull request May 20, 2022
@fmeum
Generate end-to-end coverage reports for tests
bfb066d 
Coverage reports can be generated via:

  bazel run //bazel/coverage

Due to bazelbuild/bazel#15166, coverage
collection does not yet encompass tests that end with a native sanitizer
report.

Temporarily updates .bazelversion to an unreleased version of Bazel that
includes required coverage fixes.
fmeum added a commit to CodeIntelligenceTesting/jazzer that referenced this pull request May 20, 2022
@fmeum
Generate end-to-end coverage reports for tests
d6c0adf 
Coverage reports can be generated via:

  bazel run //bazel/coverage

Due to bazelbuild/bazel#15166, coverage
collection does not yet encompass tests that end with a native sanitizer
report.

Temporarily updates .bazelversion to an unreleased version of Bazel that
includes required coverage fixes.
fmeum added a commit to CodeIntelligenceTesting/jazzer that referenced this pull request May 20, 2022
@fmeum
Generate end-to-end coverage reports for tests
3260114 
Coverage reports can be generated via:

  bazel run //bazel/coverage

Due to bazelbuild/bazel#15166, coverage
collection does not yet encompass tests that end with a native sanitizer
report.

Temporarily updates .bazelversion to an unreleased version of Bazel that
includes required coverage fixes.
fmeum added a commit to CodeIntelligenceTesting/jazzer that referenced this pull request May 20, 2022
@fmeum
Generate end-to-end coverage reports for tests
224339c 
Coverage reports can be generated via:

  bazel run //bazel/coverage

Due to bazelbuild/bazel#15166, coverage
collection does not yet encompass tests that end with a native sanitizer
report.

Temporarily updates .bazelversion to an unreleased version of Bazel that
includes required coverage fixes.
@fmeum
Copy link
Collaborator Author

fmeum commented Jun 9, 2022

@c-mita Friendly ping, could you take a look?

c-mita
c-mita reviewed Jun 9, 2022
View reviewed changes
Copy link
Member

@c-mita c-mita left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine. @oquenchil - would it be worth getting this integrated with a flag?

@fmeum fmeum deleted the llvm-continuous-profile branch June 30, 2022 11:15
@fmeum
Copy link
Collaborator Author

fmeum commented Jun 30, 2022

@bazel-io flag

@bazel-io bazel-io added the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label Jun 30, 2022
@ckolli5
Copy link

ckolli5 commented Jun 30, 2022

@bazel-io fork 5.3.0

@bazel-io bazel-io removed the potential release blocker Flagged by community members using "@bazel-io flag". Should be added to a release blocker milestone label Jun 30, 2022
@bazel-io bazel-io mentioned this pull request Jun 30, 2022
Closed
This was referenced Jun 30, 2022
Closed
Merged
ckolli5 added a commit that referenced this pull request Jun 30, 2022
@ckolli5 @fmeum
Optionally enable LLVM profile continuous mode (#15782)
aedca1d 
LLVM's profile instrumentation offers a continuous mode in which
counters are continuously synced to a file rather than being dumped
once in an exit hook. This is useful for coverage runs that include
binaries that exit abnormally (e.g. in failure tests), but may require
additional compiler flags and can negatively impact runtime performance
and memory usage.

Enabling continuous mode requires adding the "%c" modifier to the value
of the LLVM_PROFILE_FILE environment variable. With this commit, the
collect_coverage.sh script adds the modifier if the test environment has
the variable LLVM_PROFILE_CONTINUOUS_MODE set. This allows both all and
individual tests to use continuous mode by setting the variable, either
via --test_env or the env attribute.

Closes #15166.

PiperOrigin-RevId: 454558752
Change-Id: Id03d00d0644742e336feab41ea479d7d8527f6c4

Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
fmeum added a commit to fmeum/bazel that referenced this pull request Jul 2, 2022
@ckolli5 @fmeum
Optionally enable LLVM profile continuous mode (bazelbuild#15782)
0eed7c2 
LLVM's profile instrumentation offers a continuous mode in which
counters are continuously synced to a file rather than being dumped
once in an exit hook. This is useful for coverage runs that include
binaries that exit abnormally (e.g. in failure tests), but may require
additional compiler flags and can negatively impact runtime performance
and memory usage.

Enabling continuous mode requires adding the "%c" modifier to the value
of the LLVM_PROFILE_FILE environment variable. With this commit, the
collect_coverage.sh script adds the modifier if the test environment has
the variable LLVM_PROFILE_CONTINUOUS_MODE set. This allows both all and
individual tests to use continuous mode by setting the variable, either
via --test_env or the env attribute.

Closes bazelbuild#15166.

PiperOrigin-RevId: 454558752
Change-Id: Id03d00d0644742e336feab41ea479d7d8527f6c4

Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
@ShreeM01 ShreeM01 removed the awaiting-review PR is awaiting review from an assigned reviewer label Sep 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@c-mita c-mita c-mita left review comments

@lberki lberki Awaiting requested review from lberki lberki is a code owner

Assignees
No one assigned
Labels
team-Rules-CPP Issues for C++ rules
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@fmeum @ckolli5 @c-mita @meteorcloudy @bazel-io @sgowroji @ShreeM01