Skip to content

Commit

Permalink
[CHERRY-PICK] BaseTools/Plugin: Clarify code coverage failure message (
Browse files Browse the repository at this point in the history
…#375)

## Description

HostBasedUnitTestRunner.py is a build plugin responsible for locating
and executing host-based unit tests.

Recently, commit 6bb00aa introduced support for the plugin to
generate code coverage reports via lcov and OpenCppCoverage.

The plugin has discovered unit tests by searching for executables
with "Test" in the name for a while. However, the test coverage
change makes assumptions about test presence when crafting the
OpenCppCoverage command that ultimately fails with an ambiguous error
message if no host-based unit tests are discovered (see "ERROR").

```
SECTION - Run Host based Unit Tests
SUBSECTION - Testing for architecture: X64
ERROR - UnitTest Coverage: Failed to generate cobertura format xml in
        single package.
PROGRESS - --->Test Success: Host Unit Test Compiler Plugin NOOPT
```

This change preempts that message with a check in the plugin to
determine if any host-based tests were discovered. If not, a message
is printed with more guidance about how the developer should proceed
to either (1) fix their tests so code coverage is generated as
expected or (2) prevent the error message.

New message:

```
SECTION - Run Host based Unit Tests
SUBSECTION - Testing for architecture: X64
WARNING - UnitTest Coverage:
  No unit tests discovered. Test coverage will not be generated.

  Prevent this message by:
  1. Adding host-based unit tests to this package
  2. Ensuring tests have the word "Test" in their name
  3. Disabling HostUnitTestCompilerPlugin in the package CI YAML file
PROGRESS - --->Test Success: Host Unit Test Compiler Plugin NOOPT
```

Cc: Rebecca Cran <rebecca@bsdio.com>
Cc: Liming Gao <gaoliming@byosoft.com.cn>
Cc: Bob Feng <bob.c.feng@intel.com>
Cc: Yuwei Chen <yuwei.chen@intel.com>
Cc: Michael D Kinney <michael.d.kinney@intel.com>
Cc: Sean Brogan <sean.brogan@microsoft.com>
Signed-off-by: Michael Kubacki <michael.kubacki@microsoft.com>
Reviewed-by: Liming Gao <gaoliming@byosoft.com.cn>
Reviewed-by: Rebecca Cran <rebecca@bsdio.com>
(cherry picked from commit 3163f34)

- [ ] Impacts functionality?
- **Functionality** - Does the change ultimately impact how firmware
functions?
- Examples: Add a new library, publish a new PPI, update an algorithm,
...
- [ ] Impacts security?
- **Security** - Does the change have a direct security impact on an
application,
    flow, or firmware?
  - Examples: Crypto algorithm change, buffer overflow fix, parameter
    validation improvement, ...
- [ ] Breaking change?
- **Breaking change** - Will anyone consuming this change experience a
break
    in build or boot behavior?
- Examples: Add a new library class, move a module to a different repo,
call
    a function in a new library class in a pre-existing module, ...
- [ ] Includes tests?
  - **Tests** - Does the change include any explicit test code?
  - Examples: Unit tests, integration tests, robot tests, ...
- [x] Includes documentation?
- **Documentation** - Does the change contain explicit documentation
additions
    outside direct code modifications (and comments)?
- Examples: Update readme file, add feature readme file, link to
documentation
    on an a separate Web page, ...

## How This Was Tested

Local and server CI.

## Integration Instructions

N/A - No major change in functionality
  • Loading branch information
makubacki authored and kenlautner committed May 9, 2023
1 parent 768fbda commit dbd0095
Showing 1 changed file with 13 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
from edk2toolext.environment import shell_environment
from edk2toollib.utility_functions import RunCmd
from edk2toollib.utility_functions import GetHostInfo
from textwrap import dedent


class HostBasedUnitTestRunner(IUefiBuildPlugin):
Expand Down Expand Up @@ -84,6 +85,18 @@ def do_post_build(self, thebuilder):
else:
raise NotImplementedError("Unsupported Operating System")

if not testList:
logging.warning(dedent("""
UnitTest Coverage:
No unit tests discovered. Test coverage will not be generated.
Prevent this message by:
1. Adding host-based unit tests to this package
2. Ensuring tests have the word "Test" in their name
3. Disabling HostUnitTestCompilerPlugin in the package CI YAML file
""").strip())
return 0

for test in testList:
# Configure output name if test uses cmocka.
shell_env.set_shell_var(
Expand Down

0 comments on commit dbd0095

Please sign in to comment.