Use chromatic.config.json rather than setting options in main.js

[tmeasday]

Depends on chromaui/chromatic-cli#814

Now you can only set configPath in main.js (the location of the configuration, which defaults to chromatic.config.json).

To QA

1. Remove the chromatic.config.json, unset the main.js configPath option
2. Check you can onboard
3. Check it still works when your restart SB.
4. Reset chromatic.config.json to only having the projectToken set (simulating someone who's configured Chromatic already)
5. Do steps 2-3 again
6. Make chromatic.config.json have invalid JSON, check for meaningful error.
7. Set the configPath to something different (test.config.json), check for meaningful error when you start without the file existing
8. Create test.config.json with just projectToken
9. Do steps 2-3 again.

📦 Published PR as canary version: 0.0.70--canary.94.0c85a5f.0

✨ Test out this PR locally via:

npm install @chromaui/addon-visual-tests@0.0.70--canary.94.0c85a5f.0
# or 
yarn add @chromaui/addon-visual-tests@0.0.70--canary.94.0c85a5f.0

[linear]

AP-3623 Add CLI support for a chromatic.js config file that allows multiple environments and configuration outside of .storybook/

Currently users must configure their chromatic cli inside of storybook configuration and through cli flags. This may be confusing as users might require different settings for local builds vs ci builds. We may be able to improve this by allowing users to define their configurations in a separate, type-safe config file.

This should be an additional form of configuration, and allow backwards compatibility for existing setups using cli flags and ci configuration. It may need to allow both approaches to be mixed.

Multiple Environments:

With the addon-visual-tests having more users run builds outside of CI, configuration for the cli may need to vary based on the environment. Supporting multiple environments, through environment-suffixed names or programatic configuration would help optimize configuration for local builds.

Related Slack thread: https://chromaticqa.slack.com/archives/C051TQR6QLC/p1693198584200749

[weeksling]

LGTM.
In testing, on step 4, when it doesn't have a projectId but does have a projectToken (presumably from existing chromatic config) it removes the projectToken as soon as storybook starts. Is that intentional? It makes sense that we update it after configuration anyways, but wanted to confirm.

[jonniebigodes]

@tmeasday I'm still running some tests and share the findings tomorrow morning my time.

[tmeasday]

@weeksling

it removes the projectToken as soon as storybook starts.

That seems.. odd. Can you explain how I can reproduce that?

[weeksling]

@weeksling

it removes the projectToken as soon as storybook starts.

That seems.. odd. Can you explain how I can reproduce that?

@tmeasday change the chromatic.config.json to only have the projectToken (remove the projectId) and then start Storybook. It changes the file to be an empty object.

It looks like the first time it loads the configuration triggers the on change handler from my test

[tmeasday]

Fixed it, thanks @weeksling

[jonniebigodes]

@tmeasday so far, this is shaping up nicely; based on the QA instructions you left, it seems that one of the significant issues was already mentioned by @weeksling, and it seems fixed. However, during my testing, I noticed some things that require attention from the CLI pr.

Here's a bit of an overview of what I uncovered and the steps taken:

• Set up a small reproduction and added the canary version that was published (i.e., 0.0.66--canary.6774b46.0)

• Added some components and stories to test this

• Followed the instructions provided all good

• Adjusted chromatic.config.json to include an issue in one of the options, turning it into:

     {
        "projectToken": "example-token",
         "projectId": "example-project",
         "buildScriptName": "deploy-storybook"
     }

• Adjusted the script from build-storybook to deploy-storybook

• Started Storybook, triggered a build, and automatically reported the following error:

 Error: ✖ Build script not found
 The CLI didn't find a script called "build-storybook" in your package.json.
 Make sure you set the --build-script-name option to the value of the script name that builds your Storybook.
  at JZ (/Users/my-user/testing-file-config/node_modules/chromatic/dist/chunk-3ZZOUQMD.js:202:3145)
  at H8t (/Users/my-user/testing-file-config/node_modules/chromatic/dist/chunk-3ZZOUQMD.js:1360:1319)
  at async Promise.all (index 0)
  at async Cgi (/Users/my-user/testing-file-config/node_modules/chromatic/dist/chunk-3ZZOUQMD.js:1396:1226)
  at async Object.UWi (/Users/my-user/testing-file-config/node_modules/chromatic/dist/chunk-3ZZOUQMD.js:1396:462)
  at async Object.<anonymous> (/Users/my-user/testing-file-config/node_modules/@chromaui/addon-visual-tests/dist/index.js:8:418)

• Ran an additional test with the zip flag to verify the allowed types and if they were detected by adjusting the config file as follows:

    {
       projectToken": "example-token",
       "projectId": "example-project",
       "zip": "something"
    }

• Restarted Storybook and triggered yet another build, and surprisingly it didn't complain or throw an error when the type is actually a boolean value.

[tmeasday]

Hey @jonniebigodes that's just because I forgot to add zip as an allowed configuration option! I guess it doesn't error on unknown keys, I might add that.