Starting storybook takes very long

[kroeder]

Bug or support request summary

I'm not sure if this is just the nature of a big storybook (we got 46 stories so far 😃 ) but starting it takes more and more time (about 2,5 min, currently)

Also hot-reloading takes about 3-5s (which is "fine" but not smooth compared to the initial experience of an empty project)

I'd like to investigate this further and, if possible, improve this but I don't really know where to start.
One thing I already noticed is, that the build stops at a certain % for quite a while; it takes about 60-70% of the whole build time

First try

 33% building modules 193/219 modules 26 active …ress-column\address-column.component.ts
 33% building modules 193/220 modules 27 active …@storybook\ui\dist\modules\ui\routes.js
 33% building modules 194/221 modules 27 active …ok\angular\dist\client\preview\index.js
 33% building modules 195/221 modules 26 active …ok\angular\dist\client\preview\index.js
 33% building modules 195/222 modules 27 active …@storybook\react-komposer\dist\index.js

Second try

 31% building modules 181/210 modules 29 active …lient\node_modules\es5-shim\es5-shim.js
 32% building modules 184/212 modules 28 active …ybook\addon-storysource\dist\preview.js
 32% building modules 185/212 modules 27 active …ybook\addon-storysource\dist\preview.js
 32% building modules 187/213 modules 26 active …@storybook\ui\dist\modules\ui\routes.js
 32% building modules 189/217 modules 28 active …ient\node_modules\fbjs\lib\invariant.js

Third try

 31% building modules 182/212 modules 30 active …js\library\modules\es6.object.create.js
 32% building modules 184/213 modules 29 active …odules\json-stringify-safe\stringify.js
 32% building modules 185/213 modules 28 active …odules\json-stringify-safe\stringify.js
 32% building modules 186/213 modules 27 active …odules\json-stringify-safe\stringify.js
 32% building modules 187/214 modules 27 active …@storybook\ui\dist\modules\ui\routes.js
 32% building modules 189/214 modules 25 active …@storybook\ui\dist\modules\ui\routes.js

Have you noticed this yourself for larger storybook projects?
Do you know any tools or options I can use to identify the root of this problem?

We are getting about 10-15 more stories the next couple months I guess. I hope at least the hot reload time does not scale further with the size of our project

Please specify which version of Storybook and optionally any affected addons that you're running

 "@storybook/addon-actions": "^4.0.0-alpha.14",
    "@storybook/addon-knobs": "^4.0.0-alpha.14",
    "@storybook/addon-links": "^4.0.0-alpha.14",
    "@storybook/addon-notes": "^4.0.0-alpha.14",
    "@storybook/addon-options": "^4.0.0-alpha.14",
    "@storybook/addon-storysource": "^4.0.0-alpha.14",
    "@storybook/addons": "^4.0.0-alpha.14",
    "@storybook/angular": "^4.0.0-alpha.14",

[igor-dv]

You can probably add a profile prop in the custom .storybook/webpack.config.js.

Also, can you share it?

[kroeder]

Unfortunately, it's not a public project. I've started to generate some webpack stats files and playing around with different webpack configurations.

Webpack compiler configuration

Test-Method	Compile Time (in min)
Full-Control-Mode + Default	4.4, 5.4
Full-Control-Mode + Default + parallelism: 4	4.33
Full-Control-Mode + Default + parallelism: 1	2.35
--- same test ---	2.52
--- same test ---	2.61
--- same test ---	2.60

So, somehow parallelism runs faster at 1 compared to any other input value (tested 2 as well)
It's also the most stable one (does not vary between 4.4 and 5.4)

Story-Loader

I also noticed that the compiler always stops right after

69% building modules 1298/1299 modules 1 active …t-client\projects sync /\.stories\.ts$/ <----------
 70% building modules 1299/1299 modules 0 active
 69% building modules 1299/1300 modules 1 active …ib\table\table\stories\table.stories.ts

Which is triggered by this I guess?

const req = require.context('../projects/', true, /\.stories\.ts$/);

function loadStories() {
    req.keys().forEach((filename) => req(filename))
}

configure(loadStories, module);

Test-Method	Compile Time (in min)
Loading no stories	0.23
Loading 1 story	2.03

Loading just 1 story was quite interesting
My config now looks like this

function loadStories() {
    // req.keys().forEach((filename) => req(filename))
    require('../projects/client-ui/src/lib/components/layouts/master-detail/stories/master-detail.stories');
}

configure(loadStories, module);

I forced to just require 1 single story but the compiler stucks again right at the first story he tries to pack

 69% building modules 1493/1494 modules 1 active …\_esm5\internal\observable\fromEvent.js
 70% building modules 1498/1498 modules 0 active
 70% building modules 1503/1503 modules 0 active
 69% building modules 1503/1504 modules 1 active …stories\master-detail-demo.component.ts

It seems it has nothing to do with how many stories I have but with webpack starting to pack the first story. After that it does not stop again.

Is this in any way helpful? I can try to isolate this in a fresh Angular-CLI 6.1 Project with ng generate library test-lib and adding a single story to it. Might be tomorrow until I can post the results of this test, though.

webpack.config.ts

module.exports = (baseConfig, env, defaultConfig) => {

    const tsFile = path.resolve(__dirname, '../tsconfig.json'); // it's important that this is absolute for the resolve.alias values!
    const tsConfig = JSON.parse(fs.readFileSync(tsFile).toString('utf8'));
    const root = path.join(path.dirname(tsFile), tsConfig.compilerOptions.baseUrl);
    if (!defaultConfig.resolve.alias) {
        defaultConfig.resolve.alias = {};
    }

    for (const alias in tsConfig.compilerOptions.paths) {
        if (alias.includes('*')) {
            // we have mappings for /* in the tsconfig, but we don't need these in webpack
            continue;
        }
        const targetPath = tsConfig.compilerOptions.paths[alias][0];
        defaultConfig.resolve.alias[alias] = path.join(root, targetPath);
    }

    defaultConfig.parallelism = 1;

    return defaultConfig;
};

config.js

import { setOptions } from '@storybook/addon-options';
import { configure, addDecorator  } from '@storybook/angular';
import { withNotes } from '@storybook/addon-notes';

addDecorator(withNotes);

setOptions({
    /**
     * name to display in the top left corner
     * @type {String}
     */
    name: 'Storybook',
    /**
     * URL for name in top left corner to link to
     * @type {String}
     */
    url: '#',
    /**
     * open story component as full screen
     * @type {Boolean}
     */
    goFullScreen: false,
    /**
     * display panel that shows a list of stories
     * @type {Boolean}
     */
    showStoriesPanel: true,
    /**
     * display panel that shows addon configurations
     * @type {Boolean}
     */
    showAddonPanel: true,
    /**
     * display floating search box to search through stories
     * @type {Boolean}
     */
    showSearchBox: false,
    /**
     * open addon panel as a vertical panel on the right
     * @type {Boolean}
     */
    addonPanelInRight: false,
    /**
     * sorts stories
     * @type {Boolean}
     */
    sortStoriesByKind: false,
    /**
     * regex for finding the hierarchy separator
     * @example:
     *   null - turn off hierarchy
     *   /\// - split by `/`
     *   /\./ - split by `.`
     *   /\/|\./ - split by `/` or `.`
     * @type {Regex}
     */
    hierarchySeparator: null,
    /**
     * regex for finding the hierarchy root separator
     * @example:
     *   null - turn off mulitple hierarchy roots
     *   /\|/ - split by `|`
     * @type {Regex}
     */
    hierarchyRootSeparator: null,
    /**
     * sidebar tree animations
     * @type {Boolean}
     */
    sidebarAnimations: true
});

const req = require.context('../projects/', true, /\.stories\.ts$/);

function loadStories() {
    // req.keys().forEach((filename) => req(filename))
    require('../projects/client-ui/src/lib/components/layouts/master-detail/stories/master-detail.stories');
}

configure(loadStories, module);

[igor-dv]

Very nice research.
You wrote that you are using @storybook/addon-storysource, but I don't see it's configured.

[kroeder]

It isn't configured (thanks for the hint) because it does not fit to our way of writing stories in angular (see #3566 (comment))

I tried to deactivate all addons and stopped loading readme.md files (also removed the notes parameter)

Nothing improved the build time

[kroeder]

Which tsconfig is used to build storybook? The root tsconfig of a project?

[igor-dv]

Storybook itself is not in TS. For angular you have this - https://github.com/storybooks/storybook/blob/master/app/angular/tsconfig.json

[kroeder]

Thanks! I found out that I can place a custom tsconfig in the .storybook configuration folder
I'm currently trying to make a minimal reproduction repo but this is really a tough one.

I tried to start with an empty project and add more and more stuff but this does not produce this issue. I'm now going the other way around and remove more and more stuff from our project until the issue is gone. Hopefuly, the result of my work makes it possible to offer you a github repo or at least a way to reproduce this.

Super cool would be some sort of parameter in the CLI to analyze the webpack build npm run storybook --debug

I assume using https://github.com/stephencookdev/speed-measure-webpack-plugin/ is only possible in full-control mode without extending the storybook base config? Is this even possible in an angular project with angular-cli?

[igor-dv]

Super cool would be some sort of parameter in the CLI to analyze the webpack build npm run storybook --debug

Not sure, I got you what it will do

Is this even possible in an angular project with angular-cli?

Yeah, you can add whatever you want to webpack in the extended/full control of any app

[andrei-ilyukovich]

@kroeder @igor-dv I also faced with significant performance issues during starting up storybook. It works fast in case of having one or two simple stories, but adding about 10 leads to start up time about 3-4 mins.
Also I've noticed that addon-knobs for Angular works in a very strange manner that impact story loading performance. Every time I change some knob, the whole story is reinitialized like during story first loading (I assume that based on initializing of modules needed by story, changing value in knob causes modules to be recreated)

[kroeder]

Can you reproduce this in a Repo? I was not able to..

[andrei-ilyukovich]

Yes, sure. You can open browser console for this page https://storybooks-angular.netlify.com/?selectedKind=Addon%7CKnobs&selectedStory=All%20knobs&full=0&addons=1&stories=1&panelRight=0&addonPanel=storybooks%2Fstorybook-addon-knobs&background=%2300aced
There should be "constructor" log message from AllKnobsComponent. Then try to open another story then get back to "All knobs" again - you'll see double "constructor" message. Also changing any value in knob panel forces AllKnobsComponent to be recreated

[andrei-ilyukovich]

I didn't find such behavior for stories which doesn't have knobs

[andrei-ilyukovich]

Probably performance issue is related somehow to typescript files processing. https://storybook.js.org/configurations/typescript-config/
Guide tells that we need to add "awesome-typescript-loader" to work with typescript, but angular-cli example doesn't have it configured. So do we need it or not? If not then what loader is used by storybook under the hood to process ts files?

[igor-dv]

If awesome-typescript-loader solves the performance, we can make it work, Storybook doesn't use angular-cli's ts rules.

[andrei-ilyukovich]

So could you please explain how angular-cli example is working without any typescript configuration? Does default webpack config contain typescript loader definition?

[kroeder]

I did some more research regarding my initial issue

I said "Loading 1 story takes as long as loading every story (about 40-45)"
That's not entirely correct.

Loading 1 story in a project with 40-45 stories takes as long as loading every of these 40-45 stories

Note that these times were recorded on a different machine than the one I used for my research above

Start time: 1,5min
Hot reload time: 5.2s

function loadStories() {
    // req.keys().forEach((filename) => req(filename))
    require("../projects/client-ui/src/lib/components/buttons/button-primary/stories/button-primary.stories")
}

configure(loadStories, module);

Removing every component + story except 1

Start time: 0,5min
Hot reload time: 1s

I prepared a repo with the 1 story scenario but I guess this is of no interest if it runs so much faster than one with many components, which I can't make public..

In my next steps I'll play around with a seperat tsconfig file for storybook. It seems that starting storybook includes every file that is specified in the root tsconfig.json. I noticed that by removing all of the components. Compile errors occurred in non-story relevant code

Edit: Might work if I add 40 dummy stories

[igor-dv]

@kroeder, it makes sense the problem is somewhere in ts configuration. Maybe you can just copy-past an existing component 100 times to simulate the problem ?

@andrei-ilyukovich with the full control mode of the webpack.config, you can do something like this (pseudo):

module.exports = (basicConfig, mode, defaultConfig) => {
   basicConfig.module.rule = removeTsRule(basicConfig.module.rule);

   basicConfig.module.rule.push(createAwesomeTsRule())
   
   return basicConfig;
};

[andrei-ilyukovich]

@kroeder @igor-dv It look like I found solution to reduce building time significantly. The reason why it takes so long in current config is just ts-loader configuration, which is used internally by storybook. We need to skip type checking with setting flag transpileOnly: true for ts-loader in webpack config. Ts-loader docs tell us how it is possible to still have faster type checking with "fork-ts-checker-webpack-plugin". These configuration updates reduces building time from 4 mins to 11 sec. Here is how webpack.config should look like:

const path = require("path");
const ForkTsCheckerWebpackPlugin = require("fork-ts-checker-webpack-plugin");

module.exports = (baseConfig, env, config) => {
// Find rule for processing ts files and then add transpileOnly option to skip type checking
    let rule = config.module.rules.find((rule) => rule.use[0].loader.includes("ts-loader"));
    rule.use[0].options.transpileOnly = true;
// Add plugin to have separate type checking
    config.plugins.push(
        new ForkTsCheckerWebpackPlugin({
            tslint: path.resolve(__dirname, "../tslint.json"),
            tsconfig: path.resolve(__dirname, "tsconfig.json")
        })
    );
    return config;
};

Another option is to replace ts-loader with awesome-typescript-loader. With such config build time is about 25 sec, which is good but worse than first solution.

const path = require("path");

module.exports = (baseConfig, env, config) => {
// Find rule for processing ts files and then set new loader
    let rule = config.module.rules.find((rule) => rule.use[0].loader.includes("ts-loader"));
    rule.use[0].loader = require.resolve("awesome-typescript-loader");
// "awesome-typescript-loader" has a bit different option name to set tsconfig.json (ts-loader has configFile option)
    rule.use[0].options.configFileName = path.resolve(__dirname, "tsconfig.json");

    return config;
};

[igor-dv]

😱 omg, it's a huge difference. We probably need to enable transpileOnly by default for angular...
I don't know though if we want to have a ForkTsCheckerWebpackPlugin set by default. It means we will also need to discover the user's tslint.json.

BTW, there is an option to define a custom tsconfig.json in the .storybook dir.

[andrei-ilyukovich]

Basically tslint is just an option, you could configure ForkTsCheckerWebpackPlugin to not make ts linting.
But anyway you could use awesome-typescript-loader which also have fast type checking out-of-the box.

Yes, I know about custom tsconfig.json, in examples above I have to set it explicitly, because by default it isn't picked up by plugin

[kroeder]

Can confirm.
Using ForkTsCheckerWebpackPlugin and defaultConfig.parallelism = 1; it took
Build Time: 50s~

Without defaultConfig.parallelism = 1; (which was only used, because ts-loader ran faster with it)
Build Time: 38s~

Might be quicker but I have do deal with a couple new typescript errors like

typings.d.ts(7,5):
TS1038: A 'declare' modifier cannot be used in an already ambient context.

or

@types/storybook__addon-notes/index.d.ts(9,32):
TS2307: Cannot find module '@storybook/react'.

[kroeder]

After fixing everything it takes around 30-35s - Sadly, my hot-reload still takes around 5s but that's ok for now

More good news: It seems this approach moves the build-time bottleneck somewhere else.
30-35s build time on a Citrix VM
25s~ build time on a Gaming PC

Before:
2:30min build time on a Citrix VM
1:30min build time on a Local Developer Machine (which is obviously weaker than my gaming PC 😂 )

Sorry for throwing around with numbers the whole time but I guess build time is exactly about that 😃

So: @igor-dv is this something you would consider add to the @storybook/angular defaultConfig or should we use a custom webpack config? Regarding the tslint.json discovery.. In most cases it should be at the same place as the angular.json. If not you could add an example of how to configure a custom tsconfig.json and custom tslint.json location

Maybe too naive?

[andrei-ilyukovich]

@kroeder you could try to make your build time even better by using other hints from ts-loader team.
For example it worth using HappyPack. I didn't try it as far as I am fine with ~13 sec for my setup with just transpileOnly

[igor-dv]

I think we need to include transpileOnly to the default config,

Would somebody like to PR this?

[kroeder]

transpileOnly sounds good but there should be any form of type checking per default
Just setting it to true feels wrong to me

What about your concerns regarding tslint.json? See my comment above

Regarding the tslint.json discovery.. In most cases it should be at the same place as the angular.json. If not you could add an example of how to configure a custom tsconfig.json and custom tslint.json location

[igor-dv]

Makes sense

[kroeder]

On the other hand: nowadays you might use storybook in Angular 6 library projects which have multiple projects / tslint.json

Per default, angular-cli creates a library project with its own tslint.json containing

    "extends": "../../tslint.json",
    "rules": {
        "directive-selector": [
            true,
            "attribute",
            "lib",
            "camelCase"
        ],
        "component-selector": [
            true,
            "element",
            "lib",
            "kebab-case"
        ]
    }
}

File-structure

./.storybook
./projects (with multiple library projects, created by angular-cli)
--- Library A/tslint.json
--- Library B/tslint.json
src (demo app, created by angular-cli)
--- /tslint.json
./tslint.json

I haven't seen support for this in Fork TS Checker Webpack Plugin (and hardly can imagine how this should work). Only one tslint can be used

Has the old ts-loader implementation used tslint? If not, it would not remove any features but just increasing the build speed instead if we disable the usage of tslint.

E.g.

const path = require("path");
const ForkTsCheckerWebpackPlugin = require("fork-ts-checker-webpack-plugin");

module.exports = (baseConfig, env, config) => {

    // ...

    config.plugins.push(
        new ForkTsCheckerWebpackPlugin({
            // tslint: path.resolve(__dirname, "../tslint.json"), <--- not defining it makes it 'undefined' per default;
            tsconfig: path.resolve(__dirname, "tsconfig.json")
        })
    );

    // ...
};

Not defining tslint makes it 'undefined'
https://github.com/Realytics/fork-ts-checker-webpack-plugin#options

Not sure if it does not includes any linting, though. Must be tested

My proposal would be:

• Use transpileOnly: true
• Don't add lint-checking as a feature, if it did not exists before in order to minimize the complexity angular 6 brings in with their multi-project feature
• Use ForkTsChecker, because its fast (but only if it does work without tslint)
• Use awesome-ts-loader if ForkTsChecker does not fit into the use-case for whatever-reason
• Use root-tsconfig if no tsconfig was defined in the .storybook directory (behave like storybook itself does behave with a custom tsconfig)

Update: ForkTsChecker does work without tslint
Update 2: Using ForkTsChecker with *.d.ts files inside libraries and using noImplicitAny: true somehow does make a lot of trouble in my project. It seems like the build process just does not include these *.d.ts files even when they're inside include or files of tsconfig.json - This might be a good additional test since noImplicitAny: true is nothing uncommon. I'm not sure if this is just an issue in my project, though.

[nicholasmaddren]

Hi everyone, I'm using @Storybook/react 4.1.6 with the recommended config in the Storybook docs:

const path = require('path');
module.exports = (baseConfig, env, config) => {
  config.module.rules.push({
    test: /\.(ts|tsx)$/,
    use: [{
      loader: require.resolve('awesome-typescript-loader')
    }, {
      loader: require.resolve('react-docgen-typescript-loader')
    }]
  });
  config.resolve.extensions.push('.ts', '.tsx');
  return config;
};

It currently takes 4 minutes for Storybook to compile and start. I've tried to implement the examples mentioned above to resolve this however I keep getting errors such as:

ValidationError: react-docgen-typescript-loader

  Options Validation Error

  options['transpileOnly']  is an invalid additional property

Any idea how I can speed up my start time whilst also using react-docgen-typescript-loader?

Thanks

[ghost]

@nicholasmaddren Did you find any performance optimisations?

[kroeder]

@igor-dv @ndelangen do you know anything about that issue?

[paulcpk]

@kroeder I had the same problem. Removing react-docgen-typescript-loader from ./storybook/webpack.config.js improved my performance ~5-fold (both on build and hot-reload). If you can live without having your typescript props displayed in your addons section (see the description: https://github.com/strothj/react-docgen-typescript-loader) this seems like a quick an easy fix.

[RoystonS]

If you haven't already, it's worth ensuring you've provided an explicit tsconfig.json file to your loaders, especially the docgen loader, e.g.

      {
        loader: require.resolve('react-docgen-typescript-loader'),
        options: {
          tsconfigPath: path.resolve(__dirname, '../stories/tsconfig.json'),
        },
      },

This took my generation down from 2 minutes to 9 seconds.

Without the tsconfig.json file, the loaders can easily end up scanning an entire file system tree, which takes ages.

[shilman]

cc @strothj @clintandrewhall

[flxwu]

I've observed that babel-loader for me starts up a lot faster than atl - maybe it would make sense to change that in the sb documentation?

[maninacan]

I'm having a similar issue where starting SB pretty consistently takes about 10 minutes. It will be going fine and then it hangs on some arbitrary file (it's a different file every time) and then it finishes out fine. Meaning it only hangs once but it takes 10 minutes. I'm on Storybook 5, but I'm not using TypeScript. It was working fine with a few simple components but there are a couple of components I added that are now causing it to hang. The components in question work fine in production.

[shilman]

@flxwu Mind submitting a PR for a docs change?

[shilman]

@maninacan Wanna jump on our discord? Maybe somebody from the community can help you debug it? https://discordapp.com/invite/UUt2PJb

[felipenmoura]

This is really annoying and frustrating, and this is still happening to me!
I pinpointed it to this line:

@import './test.scss';

If I comment this line, it works fast and well. If I uncomment it, it stops at around 36%.

Edited:
It seems to be even worst than that!
If I comment that line and start storybook, it builds really fast and start ok. Then I uncomment that line and everything rebuilds normally. I use the variables in it, and everything works perfectly.
It looks like this is a problem only during the startup of storybook!

[felipenmoura]

Hey I might have found something here!

I wanted storybook to load all files in any __stories__ directories. These stories could end with .stories.js or not, so I was using:

module.exports = {
  stories: [
    "../src/stories/**/*.stories.mdx",
    "../src/stories/**/*.@(js|jsx|ts|tsx)",
    "../src/components/**/__stories__/index.stories.@(js|jsx|ts|tsx)",
    "../src/components/**/__stories__/index.@(js|jsx|ts|tsx)"
  ],
  addons: [ /* ... */],
  // ...
}

This was making the start up time to take minutes (I think, I never waited it to end, actually).
In the end, it turns out that commenting the last line of that list fixed it!

module.exports = {
  stories: [
    "../src/stories/**/*.stories.mdx",
    "../src/stories/**/*.@(js|jsx|ts|tsx)",
    "../src/components/**/__stories__/index.stories.@(js|jsx|ts|tsx)",
    // "../src/components/**/__stories__/index.@(js|jsx|ts|tsx)"
  ],
  addons: [ /* ... */],
  // ...
}

I'm not sure what was really causing it, but storybook could be doing some kind of deadlock there, or some level of loop.
It looks like I will have to be satisfied with using __stories__/index.stories.js after all.

I hope this might help someone who faces the same situation.
Should it become an issue, maybe?

[shilman]

Interesting @felipenmoura . So the idea is that when you have multiple globs that resolve to the same story, some race condition is triggered that blows up the startup time?

[ndelangen]

I'm hoping that in the future, we don't rely on webpack's require.context at all anymore, and we resolve the globs (and can de-dup them) in node before starting webpack.

Sounds that might resolve that issue @shilman ?

[shilman]

@ndelangen that it would. it might also help with #12771 ?

[felipenmoura]

yup, exactly @shilman , this is not a critical problem, but it took me quite a while to figure it out :p
I asked if this could become an issue because this is relatively intuitive to use it like that, and the problem was rather unexpected, you know.

[felipenmoura]

Oh my...we are facing this problem all over again!
But this time, it's happening only on a windows machine! Any ideas?!