Skip to content

Commit

Permalink
Merge pull request #143 from storybookjs/next
Browse files Browse the repository at this point in the history 
Release 0.4.0
  • Loading branch information
@yannbf
yannbf authored Jul 12, 2022
2 parents 358b13d + d271743 commit f0a044b
Show file tree
Hide file tree
Showing 10 changed files with 802 additions and 339 deletions.
68 changes: 49 additions & 19 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Read the announcement: [Interaction Testing with Storybook](https://storybook.js
- [CLI Options](#cli-options)
- [Configuration](#configuration)
- [Running against a deployed Storybook](#running-against-a-deployed-storybook)
- [Stories.json mode](#storiesjson-mode)
- [Index.json mode](#indexjson-mode)
- [Running in CI](#running-in-ci)
- [1. Running against deployed Storybooks on Github Actions deployment](#1-running-against-deployed-storybooks-on-github-actions-deployment)
- [2. Running against locally built Storybooks in CI](#2-running-against-locally-built-storybooks-in-ci)
Expand Down Expand Up @@ -116,10 +116,11 @@ Usage: test-storybook [options]
| Options | Description |
| ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| `--help` | Output usage information <br/>`test-storybook --help` |
| `-s`, `--stories-json` | Run in stories json mode. Automatically detected (requires a compatible Storybook) <br/>`test-storybook --stories-json` |
| `--no-stories-json` | Disables stories json mode <br/>`test-storybook --no-stories-json` |
| `-i`, `--index-json` | Run in index json mode. Automatically detected (requires a compatible Storybook) <br/>`test-storybook --index-json` |
| `--no-index-json` | Disables index json mode <br/>`test-storybook --no-index-json` |
| `-c`, `--config-dir [dir-name]` | Directory where to load Storybook configurations from <br/>`test-storybook -c .storybook` |
| `--watch` | Run in watch mode <br/>`test-storybook --watch` |
| `--watch` | Watch files for changes and rerun tests related to changed files.<br/>`test-storybook --watch` |
| `--watchAll` | Watch files for changes and rerun all tests when something changes.<br/>`test-storybook --watchAll` |
| `--coverage` | Indicates that test coverage information should be collected and reported in the output <br/>`test-storybook --coverage` |
| `--url` | Define the URL to run tests in. Useful for custom Storybook URLs <br/>`test-storybook --url http://the-storybook-url-here.com` |
| `--browsers` | Define browsers to run tests in. One or multiple of: chromium, firefox, webkit <br/>`test-storybook --browsers firefox chromium` |
Expand All @@ -132,7 +133,7 @@ Usage: test-storybook [options]

## Configuration

The test runner is based on [Jest](https://jestjs.io/) and will accept the [CLI options](https://jestjs.io/docs/cli) that Jest does, like `--watch`, `--maxWorkers`, etc.
The test runner is based on [Jest](https://jestjs.io/) and will accept the [CLI options](https://jestjs.io/docs/cli) that Jest does, like `--watch`, `--watchAll`, `--maxWorkers`, etc.

The test runner works out of the box, but if you want better control over its configuration, you can run `test-storybook --eject` to create a local `test-runner-jest.config.js` file in the root folder of your project, which will be used by the test runner.

Expand All @@ -153,23 +154,38 @@ Or by using the `--url` flag:
yarn test-storybook --url https://the-storybook-url-here.com
```

### Stories.json mode
### Index.json mode

By default, the test runner transforms your story files into tests. It also supports a secondary "stories.json mode" which runs directly against your Storybook's `stories.json`, a static index of all the stories.
By default, the test runner transforms your story files into tests. It also supports a secondary "index.json mode" which runs directly against your Storybook's index data, which dependending on your Storybook version is located in a `stories.json` or `index.json`, a static index of all the stories.

This is particularly useful for running against a deployed storybook because `stories.json` is guaranteed to be in sync with the Storybook you are testing. In the default, story file-based mode, your local story files may be out of sync--or you might not even have access to the source code. Furthermore, it is not possible to run the test-runner directly against `.mdx` stories, and stories.json mode must be used.
This is particularly useful for running against a deployed storybook because `index.json` is guaranteed to be in sync with the Storybook you are testing. In the default, story file-based mode, your local story files may be out of sync – or you might not even have access to the source code. Furthermore, it is not possible to run the test-runner directly against `.mdx` stories, and `index.json` mode must be used.

To run in stories.json mode, first make sure your Storybook has a v3 `stories.json` file. You can navigate to:
<!-- TODO: switch details to 6.4 once Storybook 7.0 becomes default -->

<details>
<summary>If you're using Storybook 7.0</summary>

To run in `index.json` mode, first make sure your Storybook has a v4 `index.json` file. You can find it when navigating to:

```
https://your-storybook-url-here.com/stories.json
https://your-storybook-url-here.com/index.json
```

It should be a JSON file and the first key should be `"v": 3` followed by a key called `"stories"` containing a map of story IDs to JSON objects.
It should be a JSON file and the first key should be `"v": 4` followed by a key called `"entries"` containing a map of story IDs to JSON objects.

In Storybok 7.0, `index.json` is enabled by default, unless you are using the `storiesOf()` syntax, in which case it is not supported.

If your Storybook does not have a `stories.json` file, you can generate one provided:
</details>

On Storybook 6.4 and 6.5, to run in `index.json` mode, first make sure your Storybook has a file called `stories.json` that has `"v": 3`, available at:

- You are running SB6.4 or above
```
https://your-storybook-url-here.com/stories.json
```

If your Storybook does not have a `stories.json` file, you can generate one, provided:

- You are running Storybook 6.4 or above
- You are not using `storiesOf` stories

To enable `stories.json` in your Storybook, set the `buildStoriesJson` feature flag in `.storybook/main.js`:
Expand All @@ -180,21 +196,21 @@ module.exports = {
};
```

Once you have a valid `stories.json` file, your Storybook will be compatible with the "stories.json mode".
Once you have a valid `stories.json` file, your Storybook will be compatible with the "index.json mode".

By default, the test runner will detect whether your Storybook URL is local or remote, and if it is remote, it will run in "stories.json mode" automatically. To disable it, you can pass the `--no-stories-json` flag:
By default, the test runner will detect whether your Storybook URL is local or remote, and if it is remote, it will run in "index.json mode" automatically. To disable it, you can pass the `--no-index-json` flag:

```bash
yarn test-storybook --no-stories-json
yarn test-storybook --no-index-json
```

If you are running tests against a local Storybook but for some reason want to run in "stories.json mode", you can pass the `--stories-json` flag:
If you are running tests against a local Storybook but for some reason want to run in "index.json mode", you can pass the `--index-json` flag:

```bash
yarn test-storybook --stories-json
yarn test-storybook --index-json
```

> **NOTE:** stories.json mode is not compatible with watch mode.
> **NOTE:** index.json mode is not compatible with watch mode.
## Running in CI

Expand Down Expand Up @@ -378,6 +394,20 @@ module.exports = {
};
```

When running with `--stories-json`, tests get generated in a temporary folder and snapshots get stored alongside. You will need to `--eject` and configure a custom [`snapshotResolver`](https://jestjs.io/docs/configuration#snapshotresolver-string) to store them elsewhere, e.g. in your working directory:

```js
const path = require('path');
module.exports = {
resolveSnapshotPath: (testPath, snapshotExtension) =>
path.join(process.cwd(), '__snapshots__', path.basename(testPath) + snapshotExtension),
resolveTestPath: (snapshotFilePath, snapshotExtension) =>
path.join(process.env.TEST_ROOT, path.basename(snapshotFilePath, snapshotExtension)),
testPathForConsistencyCheck: path.join(process.env.TEST_ROOT, 'example.test.js'),
};
```

### Image snapshot recipe

Here's a slightly different recipe for image snapshot testing:
Expand Down
96 changes: 67 additions & 29 deletions bin/test-storybook.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,11 @@ const log = (message) => console.log(`[test-storybook] ${message}`);
const error = (message) => console.error(`[test-storybook] ${message}`);

// Clean up tmp files globally in case of control-c
let storiesJsonTmpDir;
let indexTmpDir;
const cleanup = () => {
if (storiesJsonTmpDir) {
log(`Cleaning up ${storiesJsonTmpDir}`);
fs.rmSync(storiesJsonTmpDir, { recursive: true, force: true });
if (indexTmpDir) {
log(`Cleaning up ${indexTmpDir}`);
fs.rmSync(indexTmpDir, { recursive: true, force: true });
}
};

Expand Down Expand Up @@ -67,7 +67,7 @@ async function reportCoverage() {
// --check-coverage if we want to break if coverage reaches certain threshold
// .nycrc will be respected for thresholds etc. https://www.npmjs.com/package/nyc#coverage-thresholds
execSync(`npx nyc report --reporter=text -t ${coverageFolder} --report-dir ${coverageFolder}`, { stdio: 'inherit' })
}
}

const onProcessEnd = () => {
cleanup();
Expand All @@ -87,6 +87,9 @@ function sanitizeURL(url) {
// remove iframe.html if present
finalURL = finalURL.replace(/iframe.html\s*$/, '');

// remove index.html if present
finalURL = finalURL.replace(/index.html\s*$/, '');

// add forward slash at the end if not there
if (finalURL.slice(-1) !== '/') {
finalURL = finalURL + '/';
Expand All @@ -97,19 +100,21 @@ function sanitizeURL(url) {

const checkForIncompatibilities = () => {
try {
const jestVersion = require('jest/package.json').version
const jestVersion = require('jest/package.json').version;
if (semver.gte(jestVersion, '28.0.0')) {
error(dedent`We detected that your project is using Jest 28.0.0 or higher, which is currently incompatible with the test runner.
You can find more info at: https://github.com/storybookjs/test-runner#errors-with-jest-28
`)
process.exit(1)
`);
process.exit(1);
}
} catch (err) {
error('We detected that Jest is not installed in your project. Please install it and run test-storybook again.')
process.exit(1)
error(
'We detected that Jest is not installed in your project. Please install Jest@27 and run test-storybook again.'
);
process.exit(1);
}
}
};

async function executeJestPlaywright(args) {
const jest = require('jest');
Expand Down Expand Up @@ -142,25 +147,56 @@ async function checkStorybook(url) {
}
}

async function fetchStoriesJson(url) {
async function getIndexJson(url) {
const indexJsonUrl = new URL('index.json', url).toString();
const storiesJsonUrl = new URL('stories.json', url).toString();

const [indexRes, storiesRes] = await Promise.all([
fetch(indexJsonUrl),
fetch(storiesJsonUrl)
]);

if (indexRes.ok) {
try {
const json = await indexRes.text();
return JSON.parse(json);
} catch (err) { }
}

if(storiesRes.ok) {
try {
const json = await storiesRes.text();
return JSON.parse(json);
} catch (err) { }
}

throw new Error(dedent`
Failed to fetch index data from the project.
Make sure that either of these URLs are available with valid data in your Storybook:
${
// TODO: switch order once index.json becomes more common than stories.json
storiesJsonUrl
}
${indexJsonUrl}
More info: https://github.com/storybookjs/test-runner/blob/main/README.md#indexjson-mode
`);
}

async function getIndexTempDir(url) {
let tmpDir;
try {
const res = await fetch(storiesJsonUrl);
const json = await res.text();
const titleIdToTest = transformPlaywrightJson(json);
const indexJson = await getIndexJson(url)
const titleIdToTest = transformPlaywrightJson(indexJson);

tmpDir = tempy.directory();
Object.entries(titleIdToTest).forEach(([titleId, test]) => {
const tmpFile = path.join(tmpDir, `${titleId}.test.js`);
fs.writeFileSync(tmpFile, test);
});
} catch (err) {
console.error(`Failed to fetch stories.json from ${storiesJsonUrl}`);
console.error(
'More info: https://github.com/storybookjs/test-runner/blob/main/README.md#storiesjson-mode\n'
);
console.error(err);
error(err);
process.exit(1);
}
return tmpDir;
Expand Down Expand Up @@ -195,10 +231,12 @@ const main = async () => {
}

// set this flag to skip reporting coverage in watch mode
isWatchMode = jestOptions.watch;
isWatchMode = jestOptions.watch || jestOptions.watchAll;

const rawTargetURL = process.env.TARGET_URL || runnerOptions.url || 'http://localhost:6006';
await checkStorybook(rawTargetURL);

const targetURL = sanitizeURL(process.env.TARGET_URL || runnerOptions.url);
await checkStorybook(targetURL);
const targetURL = sanitizeURL(rawTargetURL)

process.env.TARGET_URL = targetURL;

Expand All @@ -217,16 +255,16 @@ const main = async () => {
const { hostname } = new URL(targetURL);

const isLocalStorybookIp = await isLocalhostIp(hostname, true);
const shouldRunStoriesJson = runnerOptions.storiesJson !== false && !isLocalStorybookIp;
if (shouldRunStoriesJson) {
const shouldRunIndexJson = runnerOptions.indexJson !== false && !isLocalStorybookIp;
if (shouldRunIndexJson) {
log(
'Detected a remote Storybook URL, running in stories json mode. To disable this, run the command again with --no-stories-json'
'Detected a remote Storybook URL, running in index json mode. To disable this, run the command again with --no-index-json\n'
);
}

if (runnerOptions.storiesJson || shouldRunStoriesJson) {
storiesJsonTmpDir = await fetchStoriesJson(targetURL);
process.env.TEST_ROOT = storiesJsonTmpDir;
if (runnerOptions.indexJson || shouldRunIndexJson) {
indexTmpDir = await getIndexTempDir(targetURL);
process.env.TEST_ROOT = indexTmpDir;
process.env.TEST_MATCH = '**/*.test.js';
}

Expand Down
7 changes: 7 additions & 0 deletions package.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@
"test-storybook": "./bin/test-storybook.js"
},
"devDependencies": {
"@auto-it/released": "^10.37.1",
"@babel/cli": "^7.12.1",
"@babel/core": "^7.12.3",
"@babel/preset-env": "^7.12.1",
Expand Down Expand Up @@ -118,5 +119,11 @@
"@storybook/csf-tools": "^6.5.0",
"@storybook/store": "^6.5.0",
"jest": "^26.6.3 || ^27.0.0"
},
"auto": {
"plugins": [
"npm",
"released"
]
}
}
Loading

0 comments on commit f0a044b

Please sign in to comment.