Skip to content

Commit

Permalink
SQ-10: test with SonarScanner 6.1.0.4477
Browse files Browse the repository at this point in the history
SonarScanner 6.x is a major refactoring of SonarSource. The parameters, default values and especially the output in the LOG file has changed. Because our CI/CD test evaluates the texts in the LOG file, the Pyhton scripts had to be adapted accordingly.

There was a braking change in scanner parameter sonar.host.url:
- For SonarScanner CLI from v6.0, .NET from v7.0, and NPM from v4.0:  https://sonarcloud.io
- For older versions of the SonarScanner CLI, .NET, and NPM:  http://localhost:9000

got the message below:

```
06:50:43.753 INFO  Scanner configuration file: /home/runner/work/sonar-cxx/sonar-cxx/test/sonar-scanner-6.1.0.4477/conf/sonar-scanner.properties
06:50:43.756 INFO  Project root configuration file: /home/runner/work/sonar-cxx/sonar-cxx/integration-tests/testdata/boosttest_project/sonar-project.properties
06:50:43.769 INFO  SonarScanner CLI 6.1.0.4477
06:50:43.772 INFO  Java 17.0.12 Eclipse Adoptium (64-bit)
06:50:43.772 INFO  Linux 6.5.0-1025-azure amd64
06:50:43.779 DEBUG Scanner max available memory: 3 GB
06:50:43.797 DEBUG uname -m returned 'x86_64'
06:50:43.798 DEBUG Create: /home/runner/.sonar/cache
06:50:43.799 INFO  User cache: /home/runner/.sonar/cache
06:50:43.799 DEBUG Create: /home/runner/.sonar/cache/_tmp
06:50:44.107 DEBUG Loaded [413] system trusted certificates
06:50:44.253 INFO  JRE provisioning: os[linux], arch[x86_64]
06:50:44.279 DEBUG --> GET https://api.sonarcloud.io/analysis/jres?os=linux&arch=x86_64
06:50:45.445 DEBUG <-- 401 https://api.sonarcloud.io/analysis/jres?os=linux&arch=x86_64 (1165ms, 83-byte body)
06:50:45.445 INFO  EXECUTION FAILURE
06:50:45.446 INFO  Total time: 1.694s
06:50:45.447 ERROR Error during SonarScanner CLI execution
java.lang.IllegalStateException: Error status returned by url [https://api.sonarcloud.io/analysis/jres?os=linux&arch=x86_64]: 401
	at org.sonarsource.scanner.lib.internal.http.ServerConnection.callUrl(ServerConnection.java:182)
	at org.sonarsource.scanner.lib.internal.http.ServerConnection.callApi(ServerConnection.java:145)
	at org.sonarsource.scanner.lib.internal.http.ServerConnection.callRestApi(ServerConnection.java:123)
	at org.sonarsource.scanner.lib.internal.JavaRunnerFactory.getJreMetadata(JavaRunnerFactory.java:159)
	at org.sonarsource.scanner.lib.internal.JavaRunnerFactory.getJreFromServer(JavaRunnerFactory.java:138)
	at org.sonarsource.scanner.lib.internal.JavaRunnerFactory.createRunner(JavaRunnerFactory.java:85)
	at org.sonarsource.scanner.lib.internal.ScannerEngineLauncherFactory.createLauncher(ScannerEngineLauncherFactory.java:53)
	at org.sonarsource.scanner.lib.ScannerEngineBootstrapper.bootstrap(ScannerEngineBootstrapper.java:118)
	at org.sonarsource.scanner.cli.Main.analyze(Main.java:75)
	at org.sonarsource.scanner.cli.Main.main(Main.java:63)
```
- handle: WARN The properties 'sonar.login' and 'sonar.password' are deprecated and will be removed in the future. Please pass a token with the 'sonar.token' property instead.
. use WEB API to create token and store it in environement variable 'SONAR_TOKEN'
- fix tests: colon removed in LOG file after ERROR, INFO, WARN
- get_url_from_log: INFO is now without colon
- fix warnings in integration tests: scanner using different formats/texts
- test with SQ 9 & SQ 10
- handle: /api/rules/create Parameter 'severity' is deprecated since 10.4 and will be removed in a future version.
- handle: /api/qualityprofiles/activate_rule Parameter 'severity' is deprecated since 10.2 and will be removed in a future version.
- Code scanning: deprecation of CodeQL Action v2
- improve EOL handling on different OS of integration tests
  • Loading branch information
guwirth committed Aug 21, 2024
1 parent 776c530 commit 622107d
Show file tree
Hide file tree
Showing 14 changed files with 73 additions and 89 deletions.
8 changes: 4 additions & 4 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,15 +46,15 @@ jobs:

# setup Java
- name: Set up JDK Java ${{ matrix.java }} | ${{ matrix.distribution }} | ${{ matrix.os }}
uses: actions/setup-java@v3
uses: actions/setup-java@v4
with:
java-version: ${{ matrix.java }}
distribution: ${{ matrix.distribution }}
cache: maven

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
Expand All @@ -65,7 +65,7 @@ jobs:
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v2
uses: github/codeql-action/autobuild@v3

# ℹ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl
Expand All @@ -79,4 +79,4 @@ jobs:
# make release

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
uses: github/codeql-action/analyze@v3
8 changes: 4 additions & 4 deletions .github/workflows/cxx-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -308,8 +308,8 @@ jobs:
os: [ubuntu-latest]
java: [ '17' ]
distribution: [ 'temurin' ]
sonarqube: [ '10.6.0.92116' ]
sonarscanner: [ '5.0.1.3006' ]
sonarqube: [ '9.9.6.92038', '10.6.0.92116' ]
sonarscanner: [ '6.1.0.4477' ]

runs-on: ${{ matrix.os }}
needs: [build-linux, verify-rules]
Expand Down Expand Up @@ -449,8 +449,8 @@ jobs:
os: [windows-latest]
java: [ '17' ]
distribution: [ 'temurin' ]
sonarqube: [ '10.6.0.92116' ]
sonarscanner: [ '5.0.1.3006' ]
sonarqube: [ '9.9.6.92038', '10.6.0.92116' ]
sonarscanner: [ '6.1.0.4477' ]

runs-on: ${{ matrix.os }}
# needs build-linux because of JAR artifacts
Expand Down
12 changes: 3 additions & 9 deletions integration-tests/features/boosttest.feature
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,7 @@ Feature: Providing test execution measures
When I run sonar-scanner with "-X -Dsonar.cxx.xslt.1.inputs=btest_test_simple-test_suite.xml -Dsonar.cxx.xunit.reportPaths=btest_test_simple-test_suite.after_xslt"
Then the analysis finishes successfully
And the analysis in server has completed
And the analysis log contains no error/warning messages except those matching:
"""
.*WARN.*Unable to get a valid mac address, will use a dummy address
"""
And the analysis log contains no error/warning messages
And the following metrics have following values:
| metric | value |
| tests | 1 |
Expand All @@ -55,10 +52,7 @@ Feature: Providing test execution measures
When I run sonar-scanner with "-X -Dsonar.cxx.xslt.1.inputs=btest_test_nested-test_suite.xml -Dsonar.cxx.xunit.reportPaths=btest_test_nested-test_suite.after_xslt"
Then the analysis finishes successfully
And the analysis in server has completed
And the analysis log contains no error/warning messages except those matching:
"""
.*WARN.*Unable to get a valid mac address, will use a dummy address
"""
And the analysis log contains no error/warning messages
And the following metrics have following values:
| metric | value |
| tests | 4 |
Expand All @@ -78,8 +72,8 @@ Feature: Providing test execution measures
And the analysis in server has completed
And the analysis log contains no error/warning messages except those matching:
"""
.*WARN.*Unable to get a valid mac address, will use a dummy address
.*WARN.*cannot find the sources for '.*'
.*WARN.*Preprocessor:.*
"""
And the following metrics have following values:
| metric | value |
Expand Down
5 changes: 1 addition & 4 deletions integration-tests/features/clangtidy.feature
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,7 @@ Feature: Importing Clang-Tidy reports
Then the analysis finishes successfully
And the analysis in server has completed
And the server log (if locatable) contains no error/warning messages
And the analysis log contains no error/warning messages except those matching:
"""
.*WARN.*Unable to get a valid mac address, will use a dummy address
"""
And the analysis log contains no error/warning messages
And the number of violations fed is <violations>
Examples:
| reportpaths | violations |
Expand Down
17 changes: 7 additions & 10 deletions integration-tests/features/common.py
Original file line number Diff line number Diff line change
Expand Up @@ -125,26 +125,23 @@ def analyse_log(logpath, toignore=None):
return badlines, errors, warnings

def get_url_from_log(lines):
url = ''
for line in lines:
if 'INFO: More about the report processing at' in line:
url = line.split('INFO: More about the report processing at')[1].strip()

if 'INFO - More about the report processing at' in line:
url = line.split('INFO - More about the report processing at')[1].strip()
if 'More about the report processing at' in line:
return line.split('at ')[1].strip()

return url
return ''

def analyse_log_lines(lines, toignore=None):
badlines = []
errors = warnings = 0
toingore_re = None if toignore is None else re.compile(toignore)
for line in lines:
line = line.strip()
if is_sonar_error(line, toingore_re):
badlines.append(line)
badlines.append(line + '\n')
errors += 1
elif is_sonar_warning(line, toingore_re):
badlines.append(line)
badlines.append(line + '\n')
warnings += 1

return badlines, errors, warnings
Expand All @@ -156,5 +153,5 @@ def is_sonar_warning(line, toignore_re):
return (SONAR_WARN_RE.match(line) and not SONAR_WARN_TO_IGNORE_RE.match(line) and (toignore_re is None or not toignore_re.match(line)))

def build_regexp(multiline_str):
lines = [line for line in multiline_str.split('\n') if line != '']
lines = [line.strip() for line in multiline_str.split('\n') if line != '']
return re.compile('|'.join(lines))
11 changes: 4 additions & 7 deletions integration-tests/features/coverage.feature
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ Feature: Importing coverage data
And the analysis in server has completed
And the analysis log contains no error/warning messages except those matching:
"""
.*WARN.*Unable to get a valid mac address, will use a dummy address
.*WARN.*cannot find the sources for '#include <iostream>'
.*WARN.*Preprocessor:.*
"""
And the following metrics have following values:
| metric | value |
Expand All @@ -27,10 +27,7 @@ Feature: Importing coverage data
When I run sonar-scanner with "-X"
Then the analysis finishes successfully
And the analysis in server has completed
And the analysis log contains no error/warning messages except those matching:
"""
.*WARN.*Unable to get a valid mac address, will use a dummy address
"""
And the analysis log contains no error/warning messages
And the following metrics have following values:
| metric | value |
| coverage | 94.4 |
Expand All @@ -51,9 +48,9 @@ Feature: Importing coverage data
And the analysis in server has completed
And the analysis log contains no error/warning messages except those matching:
"""
.*WARN.*Unable to get a valid mac address, will use a dummy address
.*WARN.*cannot find the sources for '#include <iostream>'
.*WARN.*Cannot find a report for '.*'
.*WARN.*Property 'sonar.cxx.cobertura.reportPaths': cannot find any files.*
.*WARN.*Preprocessor:.*
"""
And the following metrics have following values:
| metric | value |
Expand Down
13 changes: 5 additions & 8 deletions integration-tests/features/cppcheck.feature
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Feature: Importing Cppcheck reports
And the server log (if locatable) contains no error/warning messages
But the analysis log contains a line matching
"""
WARN: The 'Cppcheck V2' report is empty.*skipping
WARN The 'Cppcheck V2' report is empty.*skipping
"""
And the number of violations fed is 0

Expand All @@ -35,7 +35,7 @@ Feature: Importing Cppcheck reports
And the server log (if locatable) contains no error/warning messages
But the analysis log contains a line matching
"""
WARN: Cannot find the file.*skipping
WARN Cannot find the file 'component1\.cc'.*skipping
"""
And the number of violations fed is 0

Expand Down Expand Up @@ -70,7 +70,7 @@ Feature: Importing Cppcheck reports
And the server log (if locatable) contains no error/warning messages
But the analysis log contains a line matching
"""
WARN: The 'Cppcheck V2' report is invalid.*skipping
WARN The 'Cppcheck V2' report is invalid.*skipping
"""
And the number of violations fed is <violations>
Examples:
Expand All @@ -91,10 +91,7 @@ Feature: Importing Cppcheck reports
Then the analysis finishes successfully
And the analysis in server has completed
And the server log (if locatable) contains no error/warning messages
And the analysis log contains no error/warning messages except those matching:
"""
.*WARN.*Unable to get a valid mac address, will use a dummy address
"""
And the analysis log contains no error/warning messages
And the number of violations fed is <violations>
Examples:
| reportpaths | violations |
Expand All @@ -116,7 +113,7 @@ Feature: Importing Cppcheck reports
And the server log (if locatable) contains no error/warning messages
And the analysis log contains no error/warning messages except those matching:
"""
.*WARN.*Unable to get a valid mac address, will use a dummy address
.*WARN.*Preprocessor:.*
"""
And the number of violations fed is <violations>
Examples:
Expand Down
17 changes: 14 additions & 3 deletions integration-tests/features/environment.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
from shutil import move
from tempfile import mkstemp
from common import analyse_log, get_sonar_log_file, cleanup_logs, print_logs
from webapi import web_api_get
from webapi import web_api_get, web_api_set


BASEDIR = os.path.dirname(os.path.realpath(__file__))
Expand All @@ -50,15 +50,15 @@ def before_all(context):
global SONAR_STARTED

print('\n\n' + 80 * '-', flush=True)
print('starting SonarQube ...', flush=True)
print('setup SonarQube ...', flush=True)
print(80 * '-', flush=True)

print('\nSonarQube already running? ', flush=True)
if is_webui_up():
print('\n\tusing already running SonarQube\n\n', flush=True)
return

print('\nSetting up the test environment', flush=True)
print('\nSetting up the test environment ...', flush=True)

sonarhome = os.environ.get('SONARHOME', None)
if sonarhome is None:
Expand All @@ -85,6 +85,17 @@ def before_all(context):
SONAR_STARTED = True
check_logs(sonarhome)

try:
print(f"\nCreate 'SONAR_TOKEN' for SonarScanner ...\n", flush=True)
url = ('/api/user_tokens/generate')
payload = {'login': 'admin', 'name': 'SonarScanner', 'type': 'GLOBAL_ANALYSIS_TOKEN'}
response = web_api_set(url, payload)
token = response.json()['token']
os.environ['SONAR_TOKEN'] = token
except:
print(f"\tCannot create 'SONAR_TOKEN' for SonarScanner.\n", flush=True)
sys.exit(1)

print('\n\n' + 80 * '-', flush=True)
print('starting tests ...', flush=True)
print(80 * '-', flush=True)
Expand Down
12 changes: 6 additions & 6 deletions integration-tests/features/googletest.feature
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,9 @@ Feature: Providing test execution measures
And the analysis in server has completed
And the analysis log contains no error/warning messages except those matching:
"""
.*WARN.*Unable to get a valid mac address, will use a dummy address
.*WARN.*cannot find the sources for '#include <gtest/gtest\.h>'
.*WARN.*cannot find the sources for '#include <unistd\.h>'
.*WARN.*Preprocessor:.*
"""
And the following metrics have following values:
| metric | value |
Expand All @@ -55,7 +55,7 @@ Feature: Providing test execution measures
Then the analysis breaks
And the analysis log contains a line matching:
"""
ERROR: Invalid xUnit report.*stop analysis
ERROR Invalid xUnit report.*stop analysis
"""


Expand All @@ -70,9 +70,9 @@ Feature: Providing test execution measures
And the analysis in server has completed
And the analysis log contains no error/warning messages except those matching:
"""
.*WARN.*Unable to get a valid mac address, will use a dummy address
.*WARN.*cannot find the sources for '#include <gtest/gtest\.h>'
.*WARN.*cannot find the sources for '#include <unistd\.h>'
.*WARN.*Preprocessor:.*
"""
And the following metrics have following values:
| metric | value |
Expand All @@ -95,11 +95,11 @@ Feature: Providing test execution measures
And the analysis in server has completed
And the analysis log contains no error/warning messages except those matching:
"""
.*WARN.*Unable to get a valid mac address, will use a dummy address
.*WARN.*cannot find the sources for '#include <gtest/gtest\.h>'
.*WARN.*cannot find the sources for '#include <unistd\.h>'
.*WARN.*The report.*seems to be empty, ignoring\.
.*WARN.*Cannot find a report for '.*'
.*WARN .*The xUnit report.*seems to be empty, ignoring\.
.*WARN.*Property 'sonar\.cxx\.xunit\.reportPaths': cannot find any files.*
.*WARN.*Preprocessor:.*
"""
And the following metrics have following values:
| metric | value |
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/features/json-db.feature
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Feature: JSON Compilation Database support
And the analysis in server has completed
And the analysis log contains no error/warning messages except those matching:
"""
.*WARN.*Unable to get a valid mac address, will use a dummy address
.*ERROR.*preprocessor:.*
"""
And the following metrics have following values:
| metric | value |
Expand Down
15 changes: 3 additions & 12 deletions integration-tests/features/regex.feature
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,7 @@ Feature: Regex
When I run sonar-scanner with "-X -Dsonar.exclusions=**/*-BOM-*.cc"
Then the analysis finishes successfully
And the analysis in server has completed
And the analysis log contains no error/warning messages except those matching:
"""
.*WARN.*Unable to get a valid mac address, will use a dummy address
"""
And the analysis log contains no error/warning messages
And the following metrics have following values:
| metric | value |
| ncloc | 3 |
Expand All @@ -26,10 +23,7 @@ Feature: Regex
When I run sonar-scanner with "-X -Dsonar.inclusions=**/utf8-BOM-*.cc"
Then the analysis finishes successfully
And the analysis in server has completed
And the analysis log contains no error/warning messages except those matching:
"""
.*WARN.*Unable to get a valid mac address, will use a dummy address
"""
And the analysis log contains no error/warning messages
And the following metrics have following values:
| metric | value |
| ncloc | 3 |
Expand All @@ -43,10 +37,7 @@ Feature: Regex
When I run sonar-scanner with "-X -Dsonar.inclusions=**/utf16-BOM-*.cc"
Then the analysis finishes successfully
And the analysis in server has completed
And the analysis log contains no error/warning messages except those matching:
"""
.*WARN.*Unable to get a valid mac address, will use a dummy address
"""
And the analysis log contains no error/warning messages
And the following metrics have following values:
| metric | value |
| ncloc | 3 |
Expand Down
Loading

0 comments on commit 622107d

Please sign in to comment.