How To Test

Testing

Important: Before starting your tests clear the cache for the test enviornment!

bin/console cache:clear --env=test

For testing with docker, there is a testing container available.

---

For our project we use three different Test suites. PhpSpec, PhpUnit and Behat (Tutorial). ---

• PhpUnit
  • Unit tests. Files are in "tests/phpUnit/..."
  • bin/phpunit tests
• PhpSpec
  • Specification tests. Files are in "tests/PhpSpec/specs/Catrobat/..."
  • bin/phpspec run
• Behat
  • Our main test suite. Files are in "tests/behat/features/..."
  • if tests fail:
    -> Firefox screenshot for each failed Scenario in "tests/testreports/screens/..."
    -> errors.json file for response errors in "tests/testreports/behat/"
  • For Behat Headless Chrome must run in the background!
    google-chrome-stable --headless --remote-debugging-address=0.0.0.0 --remote-debugging-port=9222 bin/behat
  • bin/behat

---

Tests that fail right now:

PhpUnit	PhpSpec	Behat
0	0	8 (Admin)

Note: The first behat test using chrome headless might fail after clearing the cache. Just try to rerun the failed tests again.

Note: When running behat tests; make sure to define the suite when you are checking a specific feature. The test will be way much faster, since not every hook from every suite must be executed. Just from that one suite.

bin/behat -s admin tests/behat/features/admin/xx.feature

instead of

bin/behat tests/behat/features/admin/xx.feature

Note: Some behat tests might just fail because of timing issues. At the end just rerun the failed tests with

bin/behat --rerun

This will just execute the failed tests again.

---

Test your Code Sytle

After successfully testing your code please apply the Style check tests.

• PHP: The PHP coding standard can be tested with PHPCheckStyle:

  bin/console phpcheckstyle
  

  (please adjust it to your own installation!) The HTML report can be found in /Catroweb-Symfony/style-report/

---

Additional Configurations:

In the project, you can find the two files phpspec.yml.dist and behat.yml.dist. Copy this files and rename it to phpspec.yml and behat.yml. These two new files will be only local and only available for you, so you can modify them as much as you want. To use standard setting just leave this files untouched.

---

FAQ

• At your first Behat run you might get fatal errors. Just reset sqlite permissions

  chmod o+w+x tests/behat/sqlite/ -R
  

• If your changes in feature files seem to change nothing you can try to clear the cache

  bin/console cache:clear -e test