CLI: Support version specifiers in init, upgrade and sandbox

[ndelangen]

Closes #25521
Closes #25571
Closes #25572

Tracking: #25293

What I did

Init

• Change the mechanism for picking the version to install for packages when adding packages via the package-manager. When a package is in the monorepo, a different heuristic is used. The version in versions.ts is used, and added, unless that version happens to be the latest version, in which case, no modifier is added.
• The welcome message to sb init has been changed to be more noticeable, as it can now contain a warning to the user, that they are running an old version of the CLI. (The CLI will still proceed to run the actual init!)
• It will also print a message when the user is running a prerelease version (not latest).

upgrade

This PR changes the upgrade command from always upgrading to latest or next versions, to upgrading to the version that the CLI is started as. This adds support for the scenario where users want to upgrade to a specific version that might not be the latest, by doing eg. npx storybook@8.0.1 upgrade.

It also throws errors if users attempt to "upgrade" to a lower or equal version than the one already installed. A warning is also shown if users are upgrading to a non-latest version. This is a common mistake when running npx storybook upgrade without the version specifier, but it is a valid use-case, hence the warning.

Another major change here is that previously we would upgrade (almost) all packages containing "storybook" in their name, while now we only upgrade the packages in this monorepo, because they are the only ones we truly know how to upgrade.

Sandbox

• ensure sandbox cli command uses latest version of template with the current version of the CLI
• when the version match, download the after-storybook dir
• otherwise, download the before-storybook dir, and run init (if requested)

Checklist for Contributors

Testing

The changes in this PR are covered in the following automated tests:

• stories
• unit tests
• integration tests
• end-to-end tests

Manual testing

• install in both scripts and code
• run nr task compile -s compile --no-link
• edit run-registry.ts, comment out lines 130-137
• run nr task publish --no-link -s publish
• run nr task run-registry --no-link -s run-registry (this process will stay running)
• open browser at http://localhost:6001, check if verdaccio is running and has all packages
• run npm config edit and add registry=http://localhost:6001, remove all other registries (you can comment them out).
• save the file, and close it
• create a new empty folder somewhere outside the monorepo, (I use ~/projects/storybook/repros/cli-versioning).
• run npx storybook@latest sandbox, answer yes to the "should download" question
• pick any template, I use react-vite/default-ts
• Verification 1: expect to get a fully setup sandbox, the after-storybook dir should have been used.
• Kill the run-registry process
• Change the versions in every package.json in the monorepo to the next patch version, e.g. 1.0.0 -> 1.0.1
• Change the versions.ts file manually to reflect the new versions.
• run nr task compile -s compile --no-link
• run nr task publish --no-link -s publish
• run nr task run-registry --no-link -s run-registry (this process will stay running)
• open browser at http://localhost:6001, check if verdaccio is running and has all packages (should now have 2 versions, where the latest is the one you just published)
• cd into the sandbox folder (for me that would be ~/projects/storybook/repros/cli-versioning/react-vite/default-ts)
• run npx storybook@latest upgrade
• Verification 2: the upgrade should work.
• cd back to the base repro folder (for me that would be ~/projects/storybook/repros/cli-versioning)
• run npx storybook@{{version before the patch}} sandbox, answer yes to the "should download" question
  at the time of writing main was on "version": "7.6.8", so after bumping it was on "version": "7.6.9".
  so the command would be npx storybook@7.6.8 sandbox
• pick any template, I use react-vite/default-ts, it will complain the folder name is taken, give it another name, I used react-vite/default-ts2.
• Verification 3: expect to get a fully setup sandbox, the before-storybook dir should have been used, and the init step should be executed by the sandbox script.
  The package.json of the sandbox should have the exact version of the CLI that was used to create it, no ^-modifier.

You're done, we've tested

• the sandbox command as latest.
• the sandbox command as a specific version, which tests the init command.
• the upgrade command to latest.

We did not test:

• the upgrade command to a specific version. We could test that, but we'd need to get a third version of the CLI published.
• the init command directly. We could test that, but we'd need to manually generate a project storybook can init on top of.
• failure case of upgrading to a version that's lower than the current version. We could test that, if we wanted.
• the sandbox command with a specific version, but skipping init.

When you're done:

• run npm config edit and remove registry=http://localhost:6001 (restore it to how it was before)

Documentation

• Add or update documentation reflecting your changes
• If you are deprecating/removing a feature, make sure to update
  MIGRATION.MD

Checklist for Maintainers

• When this PR is ready for testing, make sure to add ci:normal, ci:merged or ci:daily GH label to it to run a specific set of sandboxes. The particular set of sandboxes can be found in code/lib/cli/src/sandbox-templates.ts

• Make sure this PR contains one of the labels below:

  Available labels

  • bug: Internal changes that fixes incorrect behavior.
  • maintenance: User-facing maintenance tasks.
  • dependencies: Upgrading (sometimes downgrading) dependencies.
  • build: Internal-facing build tooling & test updates. Will not show up in release changelog.
  • cleanup: Minor cleanup style change. Will not show up in release changelog.
  • documentation: Documentation only changes. Will not show up in release changelog.
  • feature request: Introducing a new feature.
  • BREAKING CHANGE: Changes that break compatibility in some way with current major version.
  • other: Changes that don't fit in the above categories.

🦋 Canary release

This PR does not have a canary release associated. You can request a canary release of this pull request by mentioning the @storybookjs/core team here.

core team members can create a canary release here or locally with gh workflow run --repo storybookjs/storybook canary-release-pr.yml --field pr=<PR_NUMBER>

[socket-security]

Removed dependencies detected. Learn more about Socket for GitHub ↗︎

🚮 Removed packages: @storybook/client-logger@7.5.0, @storybook/theming@7.5.0

[shilman]

why the formatting change? is this due to the prettier upgrade, or a configuration issue on your machine?

[JReinhold]

Caused by the prettier upgrade removing the root config, Kasper brought it back again here: 4997513 (#25601)

[shilman]

why? and why here?

[ndelangen]

This change will not affect users.

This ensures we'll use the workspace version, even if other versions are referenced, somehow.

[ndelangen]

As you can see in the lockfile changes, the result is we don't install the 7.5.0 version anymore, due to this change.

Which:

• saves time during install
• reduced the risk of bundling in the wrong version
• reduced the risk of type conflicts / failures to compile

[ndelangen]

why now? I had merge conflicts, and this is how I was able to resolve + I noticed during the merge conflicts that we were installing the 7.5.0 version of some packages, which seemed very wrong, so I fixed it.

Again: AFAICS, this change cannot affect users.

[ndelangen]

@shilman I think this is the right thing to do, because we should not install 2 version of @storybook/theming in the monorepo.

It IS though if I remove this line.

And this is because of:

├─ @storybook/design-system@npm:7.15.15
│  └─ @storybook/theming@workspace:lib/theming (via npm:^7.0.2)