Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

E2E testing enhancements #2408

Merged
merged 7 commits into from
Jun 26, 2017
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -90,6 +90,12 @@ and then run `npm start` or `npm run build`.

*Note: if you are using yarn, we suggest that you use `yarn install --no-lockfile` instead of the bare `yarn` or `yarn install` because we [intentionally](https://github.com/facebookincubator/create-react-app/pull/2014#issuecomment-300811661) do not ignore or add yarn.lock to our repo.*

## Contributing to E2E (end to end) tests

**TL;DR** use the command yarn e2e:docker to run unit and e2e tests.

More detailed information are in the dedicated [README](/packages/react-scripts/fixtures/kitchensink/README.md).

## Cutting a Release

1. Tag all merged pull requests that go into the release with the relevant milestone. Each merged PR should also be labeled with one of the [labels](https://github.com/facebookincubator/create-react-app/labels) named `tag: ...` to indicate what kind of change it is.
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
@@ -5,6 +5,7 @@
"changelog": "lerna-changelog",
"create-react-app": "tasks/cra.sh",
"e2e": "tasks/e2e-simple.sh",
"e2e:docker": "tasks/local-test.sh",
"postinstall": "lerna bootstrap && cd packages/react-error-overlay/ && npm run build:prod",
"publish": "tasks/release.sh",
"start": "node packages/react-scripts/scripts/start.js",
52 changes: 52 additions & 0 deletions packages/react-scripts/fixtures/kitchensink/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# Contributing to Create React App's E2E tests

This is an end to end kitchensink test suite, but has multiple usages in it.

## Running the test suite

Tests are automatically run by the CI tools.
In order to run them locally, without having to manually install and configure everything, the `yarn e2e:docker` CLI command can be used.

This is a simple script that runs a **Docker** container, where the node version, git branch to clone, test suite, and whether to run it with `yarn` or `npm` can be chosen.
Simply run `yarn e2e:docker -- --help` to get additional info.

## Writing tests

Each time a new feature is added, it is advised to add at least one test covering it.

Features are categorized by their scope:

- *env*, all those which deal with environment variables (e.g. `NODE_PATH`)

- *syntax*, all those which showcase a single EcmaScript syntax feature that is expected to be transpiled by **Babel**

- *webpack*, all those which make use of webpack settings, loaders or plugins

### Using it as Unit Tests

In it's most basic for this serve as a collection of unit tests on a single functionality.

Unit tests are written in a `src/features/**/*.test.js` file located in the same folder as the feature they test, and usually consist of a simple `ReactDOM.render` call.

These tests are run by **jest** and the environment is `test`, so that it resembles how a **Create React App** application is tested.

### Using it as Integration Tests

This suite tests how the single features as before behave while development and in production.
A local HTTP server is started, then every single feature is loaded, one by one, to be tested.

Test are written in `integration/{env|syntax|webpack}.test.js`, depending on their scope.

For every test case added there is just a little chore to do:

- a `case` statement must be added in `src/App.js`, which simply perform a dynamic `import()` of the feature

- add a test case in the appropriate integration test file, which calls and awaits `initDOM` with the previous `SwitchCase` string

An usual flow for the test itself is something similar to:

- add an `id` attribute in a target HTML tag in the feature itself

- since `initDOM` returns a `Document` element, the previous `id` attribute is used to target the feature's DOM and `expect` accordingly

These tests are run by **mocha** (why not **jest**? See [this issue](https://github.com/facebook/jest/issues/2288)) and the environments used are both `development` and `production`.
113 changes: 113 additions & 0 deletions tasks/local-test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
#!/usr/bin/env bash

function print_help {
echo "Usage: ${0} [OPTIONS]"
echo ""
echo "OPTIONS:"
echo " --node-version <version> the node version to use while testing [6]"
echo " --git-branch <branch> the git branch to checkout for testing [the current one]"
echo " --test-suite <suite> which test suite to use ('simple', installs', 'kitchensink', 'all') ['all']"
echo " --yarn if present, use yarn as the package manager"
echo " --interactive gain a bash shell after the test run"
echo " --help print this message and exit"
echo ""
}

cd $(dirname $0)

node_version=6
current_git_branch=`git rev-parse --abbrev-ref HEAD`
git_branch=${current_git_branch}
use_yarn=no
test_suite=all
interactive=false

while [ "$1" != "" ]; do
case $1 in
"--node-version")
shift
node_version=$1
;;
"--git-branch")
shift
git_branch=$1
;;
"--yarn")
use_yarn=yes
;;
"--test-suite")
shift
test_suite=$1
;;
"--interactive")
interactive=true
;;
"--help")
print_help
exit 0
;;
esac
shift
done

test_command="./tasks/e2e-simple.sh && ./tasks/e2e-kitchensink.sh && ./tasks/e2e-installs.sh"
case ${test_suite} in
"all")
;;
"simple")
test_command="./tasks/e2e-simple.sh"
;;
"kitchensink")
test_command="./tasks/e2e-kitchensink.sh"
;;
"installs")
test_command="./tasks/e2e-installs.sh"
;;
*)
;;
esac

read -r -d '' apply_changes <<- CMD
cd /var/create-react-app
git config --global user.name "Create React App"
git config --global user.email "cra@email.com"
git stash save -u
git stash show -p > patch
git diff 4b825dc642cb6eb9a060e54bf8d69288fbee4904 stash^3 >> patch
git stash pop
cd -
mv /var/create-react-app/patch .
git apply patch
rm patch
CMD

if [ ${git_branch} != ${current_git_branch} ]; then
apply_changes=''
fi

read -r -d '' command <<- CMD
echo "prefix=~/.npm" > ~/.npmrc
mkdir ~/.npm
export PATH=\$PATH:~/.npm/bin
set -x
git clone /var/create-react-app create-react-app --branch ${git_branch}
cd create-react-app
${apply_changes}
node --version
npm --version
set +x
${test_command} && echo -e "\n\e[1;32m✔ Job passed\e[0m" || echo -e "\n\e[1;31m✘ Job failes\e[0m"
$([[ ${interactive} == 'true' ]] && echo 'bash')
CMD

docker run \
--env CI=true \
--env NPM_CONFIG_QUIET=true \
--env USE_YARN=${use_yarn} \
--tty \
--user node \
--volume ${PWD}/..:/var/create-react-app \
--workdir /home/node \
$([[ ${interactive} == 'true' ]] && echo '--interactive') \
node:${node_version} \
bash -c "${command}"