feat(monorepo): setup monorepo skeleton and break js to 2 packages

[subzero10]

Status

READY

Description

Monorepo features

• Typescript with Project References
  • tsconfig.json for the IDE (./src + ./test)
  • tsconfig.build.json to build only ./src
• Conventional Commits with husky and commitlint (global)
• lerna version/publish with conventional commits and changelog
• Eslint (global)

todo

• typescript - project references
• esModuleInterop - not working because in test folder - consider adding tsconfig in test folder with include
• commitlint - husky
• eslint
• gitignore
• tsd
• js - package.json
• replace shipjs with lerna version
• create draft PR
• js - integration tests
• js - npm run build - remove warnings
• ci - tests
• ci - release new version
• test lerna publish with test org
• README - core
• README - js
• README - monorepo
• ci - scheduled deployment
• fix ts build on save
• example projects
  • express
  • almond - (uglify did not work because of es6)
  • aws-lambda
  • aws-lambda-typescript (ts source not found - to check with old version)
  • browserify
  • chrome-extension
  • fastify
  • global
  • hapi
  • requirejs
  • restify
  • sails
  • webpack
• when monorepo is ready:
  • enable post-publish script in js package,
  • modify lerna publish workflow to checkout from master and set correct NPM_AUTH_TOKEN
  • remove --no-git-tag-version from lerna publish

[joshuap]

This is looking good! I'm reviewing on the monorepo branch. I also just learned that Lerna is being maintained again 🎉.

For testing the versioning/publishing process, what if you set up a test NPM org that you could publish the packages under?

[joshuap]

cc @shalvah

[subzero10]

This is looking good! I'm reviewing on the monorepo branch. I also just learned that Lerna is being maintained again 🎉.

Indeed! This helped me decide to go with Lerna, because even if the monorepo evolves and we'd need more advanced features, we can always migrate to Nx. With Lerna, we have all we need at the moment with very little learning curve.
For example, if you want to try out the monorepo branch, here are the most important points:

• always install from the root, i.e. npm install only from the root folder, otherwise you may get unexpected issues with the linked packages.
• use lerna add my-pkg --scope="@honeybadger-io/js" to add my-pkg in the @honeybadger-io/js project. Or you can manually add to the target project's package.json file. You still need to run npm install from the root.
• use lerna run to execute commands for all projects. If the command is not found it will not be executed. You can filter the packages using --scope. For example, lerna run test will execute npm run test to all packages that have this script available.

I am going to add all of these into the README.md file.

For testing the versioning/publishing process, what if you set up a test NPM org that you could publish the packages under?

Good idea, I will look into that.

[shalvah]

Nice, nice!

[subzero10]

Released under @hb-test org!
Next step -> test example projects with @hb-test/js

[subzero10]

@joshuap I replaced shipjs with lerna publish. Releasing a new version can be done by manually triggering the workflow (through Github Actions interface). The process I have so far is quite simple:

• generate changelog
• npm version
• npm publish
• tag git

All of the above are done with lerna publish on master branch. The major change is that it does not create a new branch with a pull request.
Are you OK with this simplistic approach or do you feel it's better to create a PR that needs to be approved and merged before publishing? Personally, I don't see why because so far I've been triggering new versions and merging the release PRs immediately.

On the same topic, I haven't made up my mind yet about scheduled deployments. I'm not a big fan of the current process, because if there's a PR from scheduled deployment, I can't simply trigger a newer manual deployment; I must first close the scheduled deployment PR, delete the release branch and then trigger the manual deployment.

[subzero10]

@joshuap @shalvah I think this is ready.
Maybe there is some work to be done depending on the outcome of the comment above, but other than that it should be good to go.

1. Can you please clone this branch and try it out locally?
   It should work out of the box:

• checkout the monorepo branch
• npm install
• npm run build
• npm run test

2. I have created 3 README.md pages: one for the monorepo and one for each package (js and core). Feel free to suggest improvements.

[joshuap]

@joshuap I replaced shipjs with lerna publish. Releasing a new version can be done by manually triggering the workflow (through Github Actions interface). The process I have so far is quite simple:

• generate changelog
• npm version
• npm publish
• tag git

All of the above are done with lerna publish on master branch. The major change is that it does not create a new branch with a pull request. Are you OK with this simplistic approach or do you feel it's better to create a PR that needs to be approved and merged before publishing? Personally, I don't see why because so far I've been triggering new versions and merging the release PRs immediately.

On the same topic, I haven't made up my mind yet about scheduled deployments. I'm not a big fan of the current process, because if there's a PR from scheduled deployment, I can't simply trigger a newer manual deployment; I must first close the scheduled deployment PR, delete the release branch and then trigger the manual deployment.

I'm good with whatever you prefer. My primary concerns are:

1. I want to do regular releases when we have new features on master (i.e. I don't want to forget that we have unreleased changes)
2. I want the release process to be automated+documented so that anyone with access can trigger a release without having a local environment

One question—does the new release command also release to the CDN, like the current one does?

[joshuap]

Looks good here, all the local commands worked for me. 👍

I will let @shalvah approve.

[shalvah]

Will check this out today or tomorrow. @subzero10 I think the last merge caused some conflicts.

[subzero10]

I'm good with whatever you prefer. My primary concerns are:

1. I want to do regular releases when we have new features on master (i.e. I don't want to forget that we have unreleased changes)

Gotcha, makes sense. Then I guess we have to enable scheduled deployments. Can we do this right after we merge this PR? Or is this a blocker for you? The reason is that I have tested this branch heavily and I'd rather merge it before I do further work on it.

2. I want the release process to be automated+documented so that anyone with access can trigger a release without having a local environment

I think this part is handled, but the vocabulary may need improvement. I used the word automation to refer to scheduled/automatic releases, which is something we don't have (at least for the beginning; see the point above). Can you please review this part of the documentation*? Basically, a new release will be possible with the click of a button from the Github Actions page.

*Note: you might need to wait a couple of seconds for the link above to load since it has to load all the file changes.

One question—does the new release command also release to the CDN, like the current one does?

Yes, the release to the CDN is executed as a postpublish script of the js package here. Here's a breakdown of the scripts executed with lerna publish. The postpublish script will be executed at step 8.

[shalvah]

Nice! Well done with all the work on this. Looks good. The core + js structuring makes a lot of sense.

I left a comment about the README, but I know you've already spent a lot of time on this, so I'm pushing the needed changes myself.

One thought I have (which is probably a separate discussion from this PR): I think we shouldn't enforce Conventional Commits locally. I think we should only enforce it at the PR level (which is what matters, since the commits get squashed anyway).

Rationale: I get the usefulness, but it can often be an unnecessary annoyance, especially for outsiders. I've contributed to many open source repos, and the most painful ones were those that had very specific rules you needed to follow, even for a one- or two-line docs change. The best ones avoided majoring on the minors—if you can easily fix it at the final point, don't put the burden on contributors. In this case, the fix is as simple as renaming a PR. Wouldn't that be enough to get a good changelog?

[joshuap]

One thought I have (which is probably a separate discussion from this PR): I think we shouldn't enforce Conventional Commits locally. I think we should only enforce it at the PR level (which is what matters, since the commits get squashed anyway).

Rationale: I get the usefulness, but it can often be an unnecessary annoyance, especially for outsiders. I've contributed to many open source repos, and the most painful ones were those that had very specific rules you needed to follow, even for a one- or two-line docs change. The best ones avoided majoring on the minors—if you can easily fix it at the final point, don't put the burden on contributors. In this case, the fix is as simple as renaming a PR. Wouldn't that be enough to get a good changelog?

I agree with this. Basically, enforce conventional commits on main, but not on other branches, right? 👍

[subzero10]

One thought I have (which is probably a separate discussion from this PR): I think we shouldn't enforce Conventional Commits locally. I think we should only enforce it at the PR level (which is what matters, since the commits get squashed anyway).

OK, no objection from my side. I don't know how to do that though. And it probably means that we have to manually come up with the squashed commit message. On top of that, there are a couple of challenges. Usually, we merge from github's web interface. Github has a default strategy for the merge message, which depends on the PR title and number of commits:

• single commit -> the default message will be the message of that single commit. Since we enforce conventional commit syntax on all commits, this works OK.
• multiple commits -> the default message will be the PR title. The default PR title is the first commit message, which also works for us now.

I think the whole idea of conventional commits is not for the changelog generation, but the practice for meaningful commits. Changelog generation is a consequence of this practice. What we are suggesting now is to avoid this practice but try to use it for changelog generation. Nevertheless, I totally get your point, the repository is open source and we shouldn't enforce too many policies. So:

• Personally, I will stick to conventional commits and we can make this optional (mention it in the readme page)
• I will create an issue to look into this a bit further (on how to achieve the above). Unless you have a solution already.

[shalvah]

Basically, enforce conventional commits on main, but not on other branches, right?

Yes, exactly.

I think the whole idea of conventional commits is not for the changelog generation, but the practice for meaningful commits. Changelog generation is a consequence of this practice. What we are suggesting now is to avoid this practice

That's the theoretical idea, but remember that conventional commits is a specific, opinionated style of committing. Personally, I write meaningful commits by following a simple rule: describe what this commit does. Enforcing conventional commits doesn't equate to enforcing meaningful commits. Nothing stops someone from writing a commit message like fix: fix stuff, just to get it to pass. So it's kind of a false sense of security.

And it probably means that we have to manually come up with the squashed commit message

Yes, but this is actually pretty simple. When merging a commit, GitHub shows a text box (after you click the merge button the first time) where you can edit the merge commit message (defaults to the PR title). It's a tiny bit of work to edit it, but it's negligible, really, and something I do often in certain codebases. Yes, the responsibility is shifted onto us, but you'll see it's a really, really small task.

I know you're skeptical, but thanks for giving it a try.😀

Re: implementation, I think the best option is to NOT enforce conventional commits on any branch (because enforcing on main is kinda pointless, since it won't stop a PR merge via GitHub), but use it to generate changelogs still. Basically, trust us to set the right commit message when merging. If we really need to enforce, we could enforce the PR title, if that's possible.