[Bug]: pnpm + vite Mono-Repository Test Rig — Cases of No JSDocs, No new file HMR, Recursion, Breaking

[D1no]

Edit 2023-03-18: Consolidated the scope of this ticket to issue cases. See the original at the end.

Describe the bug

The following provides a list of issue cases of using storybook with pnpm and vite. To do so, I have looked at a number of closed source small to mid scale integrated mono-repository architectures to highlight storybook functionality issues as with as minimal content as possible. I hope this test rig helps you to achieve a "just work 🤺" state within larger projects, as it would help me to integrate storybook / chromatic into such.

In this first version I've only concentrated on sourcing and rendering of stories files. In the future, building, interactions like testing and reporting should probably also be checked. As well as automated via playwright and github actions.

Storybook Version: ^7.0.0-rc.4
PNPM Version: ^7.29.3
Node Version: 18 LTS
Builder: vite
Renderer: react 18

Test Rig: Storybook in a Typical PNPM Monorepo

This repository mimics a typical integrated monorepo architecture where individual views or "micro front-ends" consume one or more centralized UI libraries. Build and test tooling (like storybook) live outside of that relationship to ensure weak coupling.

 MonoRepo/                        PNPM Workspace
     │
     ├─ ...
     │                            Outside components
     ├─ tools/                    are referenced via
     │  └── storybook ◄──┬────┐   symlinked packages
     │                   │    │   in node_modules.
     ├─ ui/              │    │
     │  ├── Component────┘    │   Storybook Issues:
     │  ├── Component         │
     │  └── ...    │          │   · TSDocs to Arg-Docs
     │             │          │   · "New File" HMR
     └─ views/     ▼          │   · Recursions / Broken
        ├── SampleView────────┘   · ...
        └── ...

How to run Storybook Issues Inspection

Clone D1no/reproduction-storybook-symlinks-pnpm

git clone git@github.com:D1no/reproduction-storybook-symlinks-pnpm.git && cd reproduction-storybook-symlinks-pnpm.git

and install

pnpm install

For install with empty cache, run

pnpm clean-install

then for the case run the respective pnpm script...

these are defined in main.ts under tools/.storybook/main.ts

Expectation

After running pnpx storybook@next init --builder vite, official storybook features should function without any additional configuration of loaders, builders or other down stream dependencies installed by storybook.

❗️ Preview images are from Version 7.0.0-rc.4 and NOT continuously updated to the latest version.

---

Case 1: Components inside storybook folder

pnpm run storybook:case_1

Components that are inside of the storybook root folder (../stories) will:

• For real files (tools/stories/Component)
  • Render successfully in storybook
  • Have JSDocs converted to story docs
  • Have HMR show new stories files (while running, co-located another.stories-hmr.tsx to .stories.tsx)
• For symbolic links (tools/stories/ComponentAsSymlink)
  • Render successfully in storybook
  • Have JSDocs converted to story docs
  • Have HMR show new stories files (while running, co-located another.stories-hmr.tsx to .stories.tsx)

---

Case 2-1: Components outside storybook folder

pnpm run storybook:case_2-1

Components that are outside of the storybook root folder but referenced with a relative path (../../ui/) will:

• Render successfully in storybook
• Have JSDocs converted to story docs
• Have HMR show new stories files (while running, co-located another.stories-hmr.tsx to .stories.tsx)

---

Case 2-2: Single capitalized component folder outside storybook

pnpm run storybook:case_2-2

A single component that is outside of the storybook root folder but referenced with a relative path (../../ui/Component) to its capitalized folder, will:

❌ ERROR: WARN 🚨 Unable to index ./../ui/Component/index.stories.tsx

tools storybook: WARN 🚨 Unable to index ./../ui/Component/index.stories.tsx:
tools storybook: WARN   Error: Invalid kind '', must include alphanumeric characters
tools storybook: WARN     at m (/Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/node_modules/.pnpm/@storybook+csf@0.0.2-next.10/node_modules/@storybook/csf/dist/index.js:3:3033)
tools storybook: WARN     at N (/Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/node_modules/.pnpm/@storybook+csf@0.0.2-next.10/node_modules/@storybook/csf/dist/index.js:3:3126)
tools storybook: WARN     at /Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/node_modules/.pnpm/@storybook+csf-tools@7.0.0-rc.4/node_modules/@storybook/csf-tools/dist/index.js:18:343
tools storybook: WARN     at Array.reduce (<anonymous>)
tools storybook: WARN     at CsfFile.parse (/Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/node_modules/.pnpm/@storybook+csf-tools@7.0.0-rc.4/node_modules/@storybook/csf-tools/dist/index.js:18:230)
tools storybook: WARN     at Object.indexer (/Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/node_modules/.pnpm/@storybook+core-server@7.0.0-rc.4/node_modules/@storybook/core-server/dist/presets/common-preset.js:8:2055)
tools storybook: WARN     at async StoryIndexGenerator.extractStories (/Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/node_modules/.pnpm/@storybook+core-server@7.0.0-rc.4/node_modules/@storybook/core-server/dist/index.js:13:3554)
tools storybook: WARN     at async /Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/node_modules/.pnpm/@storybook+core-server@7.0.0-rc.4/node_modules/@storybook/core-server/dist/index.js:13:1869
tools storybook: WARN     at async Promise.all (index 0)
tools storybook: WARN     at async Promise.all (index 0)

• Render successfully in storybook
• Have JSDocs converted to story docs
• Have HMR show new stories files (while running, co-located another.stories-hmr.tsx to .stories.tsx)

---

Case 3: Components added to storybook from pnpm workspace

pnpm run storybook:case_3

Components that are outside of the storybook root folder but referenced via pnpm workspaces, therefore located in ../node_modules/@repo/ as a symlink will:

❌ ERROR: importers[path] is not a function

importers[path] is not a function
TypeError: importers[path] is not a function
    at StoryStore.importFn (http://localhost:6006/virtual:/@storybook/builder-vite/storybook-stories.js:6:31)
    at StoryStore.loadCSFFileByStoryId (http://localhost:6006/sb-preview/runtime.mjs:40:8376)
    at StoryStore.loadStory (http://localhost:6006/sb-preview/runtime.mjs:40:9563)
    at async http://localhost:6006/sb-preview/runtime.mjs:74:9003
    at async StoryRender.runPhase (http://localhost:6006/sb-preview/runtime.mjs:74:8764)
    at async StoryRender.prepare (http://localhost:6006/sb-preview/runtime.mjs:74:8922)
    at async PreviewWeb.renderSelection (http://localhost:6006/sb-preview/runtime.mjs:94:3120)

• Render successfully in storybook
• Have JSDocs converted to story docs
• Have HMR show new stories files (while running, co-located another.stories-hmr.tsx to .stories.tsx)
• Detects symlink recursion or provides easy way to exclude them

---

Case 4: Components via pnpm with globby workaround in main.ts to avoid node_modules recursion

pnpm run storybook:case_4
# or
pnpm run storybook

Components that are outside of the storybook root folder but referenced via pnpm workspaces, ingested via a negative glob to avoid recursion as ../node_modules/@repo/**!(node_modules)/**!(node_modules)/*.stories.@(js|jsx|ts|tsx) will:

• Render successfully in storybook
• Have JSDocs converted to story docs
• Have HMR show new stories files (while running, co-located another.stories-hmr.tsx to .stories.tsx)
• Detects symlink recursion or provides easy way to exclude them

System

Via pnpx sb@next info on 2023-03-18

Environment Info:

  System:
    OS: macOS 13.2.1
    CPU: (10) arm64 Apple M1 Max
  Binaries:
    Node: 18.14.1 - ~/.nvs/default/bin/node
    npm: 9.3.1 - ~/.nvs/default/bin/npm
  Browsers:
    Chrome: 111.0.5563.64
    Firefox: 97.0.1
    Safari: 16.3
  npmPackages:
    @storybook/addon-essentials: ^7.0.0-rc.4 => 7.0.0-rc.4
    @storybook/addon-interactions: ^7.0.0-rc.4 => 7.0.0-rc.4
    @storybook/addon-links: ^7.0.0-rc.4 => 7.0.0-rc.4
    @storybook/blocks: ^7.0.0-rc.4 => 7.0.0-rc.4
    @storybook/react: ^7.0.0-rc.4 => 7.0.0-rc.4
    @storybook/react-vite: ^7.0.0-rc.4 => 7.0.0-rc.4
    @storybook/testing-library: ^0.0.14-next.1 => 0.0.14-next.1

Original Issue

[Bug]: No JSDoc generation when story outside root or symlinked (MonoRepo Blocker) ts + vite + ^storybook@7.0.0-rc.1

Show

Describe the bug

Spend a couple hours trying to get storybooks typescript JSDoc to storybook ArgDocs working, but to no avail. As typical with MonoRepos, the UI library sits outside of the testing / build toolchain.

Components render, but there is no ArgDocs generation...

• when stories are included via "../../" in .storybook/main.ts
  or
• when stories are included via a symlink (pnpm workspaces mount project libraries into node_modules/*lib*/**/*stories*) — but symlinks anywhere wont work for docgen (see reproduction)

This means docgen doesn't work here:

.
└── MonoRepo/           // git root
    ├── tools/
    │   ├── ...
    │   └── storybook   // <- Storybook root
    │                    
    └── ui/             // React Components
        ├── Component
        ├── Component
        └── ...

To Reproduce

You find a reproduction as the following git repo:
https://github.com/D1no/reproduction-storybook-symlinks-pnpm

The same component is included as (1) local copy inside stories/, (2) as reference in .storybook/main.ts and as a (3) symbolic link into stories/.

✅ All three types render successfully in storybook.

❌ Only variant (1) has JSDoc converted to story docs (!)

Exact same component and story, but as you see = different location = no JSDocGen.

The Component index.tsx

import React from "react";

interface ComponentProps {
  /**
   * Set the text of the component. This is a JSDoc of the text property.
   */
  text?: string;
}

/**
 * This is the description of the Component as JSDoc.
 */
const Component = ({ text }: ComponentProps) => {
  return <div>This component text says: {text}</div>;
};

export default Component;

The Component Story index.stories.tsx

import type { Meta, StoryObj } from "@storybook/react";

import { default as LocalComponent } from "./index";

// More on how to set up stories at: https://storybook.js.org/docs/7.0/react/writing-stories/introduction
const meta = {
  title: "stories/Component (local)",
  component: LocalComponent,
  tags: ["autodocs"],
} satisfies Meta<typeof LocalComponent>;

export default meta;
type Story = StoryObj<typeof meta>;

// More on writing stories with args: https://storybook.js.org/docs/7.0/react/writing-stories/args
export const Component: Story = {
  args: {
    text: "I am a local component inside of tools/stories/Component. And I have docs from the JSDoc comments.",
  },
};

System

Environment Info:

  System:
    OS: macOS 13.2.1
    CPU: (10) arm64 Apple M1 Max
  Binaries:
    Node: 18.14.1 - ~/.nvs/default/bin/node
    npm: 9.3.1 - ~/.nvs/default/bin/npm
  Browsers:
    Chrome: 110.0.5481.177
    Firefox: 97.0.1
    Safari: 16.3
  npmPackages:
    @storybook/addon-essentials: ^7.0.0-beta.61 => 7.0.0-beta.61
    @storybook/addon-interactions: ^7.0.0-beta.61 => 7.0.0-beta.61
    @storybook/addon-links: ^7.0.0-beta.61 => 7.0.0-beta.61
    @storybook/blocks: ^7.0.0-alpha.8 => 7.0.0-alpha.8
    @storybook/react: ^7.0.0-beta.61 => 7.0.0-beta.61
    @storybook/react-vite: ^7.0.0-beta.61 => 7.0.0-beta.61
    @storybook/testing-library: ^0.0.14-next.1 => 0.0.14-next.1

Additional context

I have the feeling, the problem lies in @joshwooding/vite-plugin-react-docgen-typescript 👀 ping @joshwooding

_{I really love storybook and have mad respect of what you guys are dealing with. For me, to get storybook into more larger projects, it would be important to have reliable mono repository functionality. In the past I recall from engineers issues with SB, be it with nx (or here, pnpm), that some features seem to always not quite work... Not sure what the SB test suit is, but it would be nice to know that common mono repo tools (pnpm worspaces, nx, turborepo, lerna, etc) are — in a basic set-up — in your test suit for features and add-ons. As single root projects seem to be a thing of the past (which is a great thing for test driven development and team scalability). Anyway: Tip to the hat; I hope this repro is helpful!}

[D1no]

Updated Reproduction to storybook@7.0.0-beta.62

Edit:

Please see inbound issue links below for consecutive version updates

[tmeasday]

I'm not 100% sure about Vite but my best guess is the plugin you mentioned isn't configured to run on files outside of the CWD. @D1no -- you could experiment with setting options on the plugin in a viteFinal to see if there's a workaround.

[D1no]

@tmeasday yeah since the Vite configuration is coming from storybook dependencies, and the reproduction simply creates a Vite based storybook set-up:

Manually overwriting / tinkering with how storybook instruments the bundler would be a hack - but not proper functionality.

[tmeasday]

I get it, I am just interested in what the source of the problem @D1no

[D1no]

@tmeasday I sadly don't understand enough about vite 😕 (which is actually the benefit, that you don't need to). I have the gut feeling that this has something to do with this dependency @joshwooding/vite-plugin-react-docgen-typescript by @joshwooding

[D1no]

Edit: Created Issue

@tmeasday I've just tried to update the reproduction from 7.0.0-rc.1 to 7.0.0-rc.3 in hope that it helps you. But now the vite bundler seems to be missing all together 😅 oh boy

cd tools && pnpx storybook@next upgrade --prerelease --skip-check

➜  tools git:(master) ✗ pnpm run storybook                                   

> tools@ storybook /Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools
> storybook dev -p 6006

@storybook/cli v7.0.0-rc.3

ERR! Error: Cannot find module '@storybook/builder-vite'
ERR! Require stack:
ERR! - /Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools/node_modules/.pnpm/@storybook+core-server@7.0.0-rc.3/node_modules/@storybook/core-server/dist/index.js
ERR! - /Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools/node_modules/.pnpm/@storybook+cli@7.0.0-rc.3/node_modules/@storybook/cli/dist/generate.js
ERR! - /Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools/node_modules/.pnpm/@storybook+cli@7.0.0-rc.3/node_modules/@storybook/cli/bin/index.js
ERR! - /Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools/node_modules/.pnpm/storybook@7.0.0-rc.3/node_modules/storybook/index.js
ERR!     at Module._resolveFilename (node:internal/modules/cjs/loader:1075:15)
ERR!     at Function.resolve (node:internal/modules/cjs/helpers:116:19)
ERR!     at getPreviewBuilder (/Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools/node_modules/.pnpm/@storybook+core-server@7.0.0-rc.3/node_modules/@storybook/core-server/dist/index.js:10:1789)
ERR!     at buildDevStandalone (/Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools/node_modules/.pnpm/@storybook+core-server@7.0.0-rc.3/node_modules/@storybook/core-server/dist/index.js:40:2022)
ERR!     at async withTelemetry (/Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools/node_modules/.pnpm/@storybook+core-server@7.0.0-rc.3/node_modules/@storybook/core-server/dist/index.js:27:3469)
ERR!     at async dev (/Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools/node_modules/.pnpm/@storybook+cli@7.0.0-rc.3/node_modules/@storybook/cli/dist/generate.js:446:400)
ERR!     at async Command.<anonymous> (/Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools/node_modules/.pnpm/@storybook+cli@7.0.0-rc.3/node_modules/@storybook/cli/dist/generate.js:448:327)
ERR!  Error: Cannot find module '@storybook/builder-vite'
ERR! Require stack:
ERR! - /Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools/node_modules/.pnpm/@storybook+core-server@7.0.0-rc.3/node_modules/@storybook/core-server/dist/index.js
ERR! - /Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools/node_modules/.pnpm/@storybook+cli@7.0.0-rc.3/node_modules/@storybook/cli/dist/generate.js
ERR! - /Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools/node_modules/.pnpm/@storybook+cli@7.0.0-rc.3/node_modules/@storybook/cli/bin/index.js
ERR! - /Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools/node_modules/.pnpm/storybook@7.0.0-rc.3/node_modules/storybook/index.js
ERR!     at Module._resolveFilename (node:internal/modules/cjs/loader:1075:15)
ERR!     at Function.resolve (node:internal/modules/cjs/helpers:116:19)
ERR!     at getPreviewBuilder (/Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools/node_modules/.pnpm/@storybook+core-server@7.0.0-rc.3/node_modules/@storybook/core-server/dist/index.js:10:1789)
ERR!     at buildDevStandalone (/Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools/node_modules/.pnpm/@storybook+core-server@7.0.0-rc.3/node_modules/@storybook/core-server/dist/index.js:40:2022)
ERR!     at async withTelemetry (/Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools/node_modules/.pnpm/@storybook+core-server@7.0.0-rc.3/node_modules/@storybook/core-server/dist/index.js:27:3469)
ERR!     at async dev (/Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools/node_modules/.pnpm/@storybook+cli@7.0.0-rc.3/node_modules/@storybook/cli/dist/generate.js:446:400)
ERR!     at async Command.<anonymous> (/Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools/node_modules/.pnpm/@storybook+cli@7.0.0-rc.3/node_modules/@storybook/cli/dist/generate.js:448:327) {
ERR!   code: 'MODULE_NOT_FOUND',
ERR!   requireStack: [
ERR!     '/Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools/node_modules/.pnpm/@storybook+core-server@7.0.0-rc.3/node_modules/@storybook/core-server/dist/index.js',
ERR!     '/Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools/node_modules/.pnpm/@storybook+cli@7.0.0-rc.3/node_modules/@storybook/cli/dist/generate.js',
ERR!     '/Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools/node_modules/.pnpm/@storybook+cli@7.0.0-rc.3/node_modules/@storybook/cli/bin/index.js',
ERR!     '/Users/worker/development/_REPRODUCTIONS/reproduction-storybook-symlinks-pnpm/tools/node_modules/.pnpm/storybook@7.0.0-rc.3/node_modules/storybook/index.js'
ERR!   ]
ERR! }

WARN Broken build, fix the error above.
WARN You may need to refresh the browser.

✔ Would you like to help improve Storybook by sending anonymous crash reports? … yes
 ELIFECYCLE  Command failed with exit code 1.

[IanVS]

We need to do a better job documenting this, obviously, but there is a setting that you can use to control which files are included for processing by react-docgen-typescript:

//.storybook/main.ts

export default {
	//... rest of config
	typescript: {
		reactDocgenTypescriptOptions: {
			include: ['**/**.tsx', '../ui/**/**.tsx'],
		},
	},
}

This doesn't fix the symlinks. Normally I would suggest setting https://vitejs.dev/config/shared-options.html#resolve-preservesymlinks to true, but that causes other things to break in your repo, likely because of pnpm.

[IanVS]

@jonniebigodes have you thought about having a section of the docs dedicated to monorepos and the things that need to be done in some situations to get things like the above working?

[D1no]

@IanVS I have extended the ticket to a test rig. I hope that helps.

[jonniebigodes]

@IanVS to answer your question, yes, I've considered it, but before committing to it I wanted to collect some information beforehand from our community sources (e.g., here in the issue tracker, and Discord) and see what are the major pain points and how to address them to avoid having the information scattered all around. If you could assist me with this, I'd be thankful for it. We could probably collaborate on it after the 7.0 launch. What do you think? Sounds like a reasonable plan?

[shilman]

Can you try out the following canary in your project and let me know whether it resolves any of your issues (or generates new ones)? We're looking to switch the default docgen from react-docgen-typescript to react-docgen, which is much faster and may also fix some long-standing bugs. Many thanks!

Note that the change is currently only for Vite projects. Instructions in the "how to test" section: 👉 #23825

[AMSTKO]

We need to do a better job documenting this, obviously, but there is a setting that you can use to control which files are included for processing by react-docgen-typescript:

//.storybook/main.ts

export default {
	//... rest of config
	typescript: {
		reactDocgenTypescriptOptions: {
			include: ['**/**.tsx', '../ui/**/**.tsx'],
		},
	},
}

Haha thanks you @IanVS my god I'm looking for this since a long time