Added CodeCov coverage report support

[connorhu]

Pull Request

Related issue

Fixes #284

What does this PR do?

• Integrates CodeCov

PR checklist

Please check if your PR fulfills the following requirements:

• Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
• Have you read the contributing guidelines?
• Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!

[norkunas]

For consistency use single quotes

[codecov]

Welcome to Codecov 🎉

Once merged to your default branch, Codecov will compare your coverage reports and display the results in this comment.

Thanks for integrating Codecov - We've got you covered ☂️

[connorhu]

We need to lock phpunit version because coverage report is broken with PHP 7.4, Symfony 5.4 and phpunit 8. With this constellation, code coverage did not work. One solution is to lock the phpunit version, another is to create a .ci/github/phpunit/phpunit.xml.dist file and lock the version only in that file. Both solutions have their advantages and disadvantages.

PHP Fatal error:  Uncaught TypeError: Argument 1 passed to SebastianBergmann\CodeCoverage\CodeCoverage::__construct() must be an instance of SebastianBergmann\CodeCoverage\Driver\Driver, null given, called in /home/runner/work/meilisearch-symfony/meilisearch-symfony/vendor/bin/.phpunit/phpunit-8.5-0/src/TextUI/TestRunner.php on line 526 and defined in /home/runner/work/meilisearch-symfony/meilisearch-symfony/vendor/phpunit/php-code-coverage/src/CodeCoverage.php:122
Stack trace:
#0 /home/runner/work/meilisearch-symfony/meilisearch-symfony/vendor/bin/.phpunit/phpunit-8.5-0/src/TextUI/TestRunner.php(526): SebastianBergmann\CodeCoverage\CodeCoverage->__construct()
#1 /home/runner/work/meilisearch-symfony/meilisearch-symfony/vendor/bin/.phpunit/phpunit-8.5-0/src/TextUI/Command.php(235): PHPUnit\TextUI\TestRunner->doRun()
#2 /home/runner/work/meilisearch-symfony/meilisearch-symfony/vendor/bin/.phpunit/phpunit-8.5-0/src/TextUI/Command.php(194): PHPUnit\TextUI\Command->run()
#3 /home/runner/work/meilisearch-symfony/meilisearch-symfony in /home/runner/work/meilisearch-symfony/meilisearch-symfony/vendor/phpunit/php-code-coverage/src/CodeCoverage.php on line 122
Script SYMFONY_DEPRECATIONS_HELPER='ignoreFile=./tests/baseline-ignore' ./vendor/bin/simple-phpunit --colors=always --verbose handling the test:unit event returned with error code 255

[norkunas]

I don't mind locking it as long as tests runs and are green :)

[norkunas]

The CI breaks now because bors expects these jobs:

status = [
    'integration-tests (PHP 7.4) (Symfony 5.4.*)',
    'integration-tests (PHP 8.0) (Symfony 6.0.*)',
    'integration-tests (PHP 8.1) (Symfony 6.1.*)',
    'integration-tests (PHP 8.2) (Symfony 6.2.*)',
    'integration-tests (PHP 8.3) (Symfony 6.3.*)',
    'integration-tests (PHP 8.3) (Symfony 6.4.*)',
    'integration-tests (PHP 8.3) (Symfony 7.0.*)',
]

So I think either restore using with .* and then just strip it before passing to codecov to avoid limitations as you said (will be less changes) or update the bors.toml

[connorhu]

Both solutions are good. If editing bors.toml causes further problems, I'll put an wildcard in the name of the job instead.

[norkunas]

The less changes are better :)

[norkunas]

Error: No coverage reports found. Please make sure you're generating reports successfully.
Warning: Codecov: 
                      Failed to properly upload report: The process '/home/runner/work/_actions/codecov/codecov-action/v4/dist/codecov' failed with exit code 1

[connorhu]

Now it looks like it will find it:

[2024-02-14T08:05:45.455Z] ['info'] => Found 5 possible coverage files:
  phpunit-7.4-5.4-coverage/coverage.xml
  phpunit-8.0-6.0-coverage/coverage.xml
  phpunit-8.1-6.1-coverage/coverage.xml
  phpunit-8.2-6.2-coverage/coverage.xml
  phpunit-8.3-7.0-coverage/coverage.xml

[norkunas]

Why it's not working with v4? Adding a new feature should use last versions 🤔

[connorhu]

Why it's not working with v4? Adding a new feature should use last versions 🤔

The problem was caused by the fact that I only transferred to v4 on download, not on upload. v3 uploaded, v4 downloaded.

[connorhu]

Shiny 86% reported. Nice.

[norkunas]

I still suggest to revert changes about wildcard, because in repo settings jobs are made as required with previous names, and I don't have permissions to do that

[norkunas]

Good job 🎉

After #323 CI will be green (I can't review/merge my own PR's)
Before merging this we should probably wait for @brunoocasali approval.

[connorhu]

I signed up for that PR. If it went in I'll rebase this and we'll green it.

[connorhu]

About this php-cs-fixer problem: I think you should hardcode the php-cs-fixer version everywhere, and apply the fixer changes with version changes. Elsewhere I just ran into that two versions interpreting something completely differently (incorrectly) and it took me some time to find out what the source of the problem was.

[norkunas]

That's just because @Symfony ruleset rules has changed, it doesn't change too much, so therefore I'm not sure about locking cs fixer

[norkunas]

@connorhu you can rebase now :)

[norkunas]

bors merge

[norkunas]

bors merge

[norkunas]

bors cancel

[norkunas]

@connorhu could you squash your commits?

[connorhu]

@norkunas of course. done

[norkunas]

Thank you @connorhu