Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds monorepo bootstrap #377

Merged
merged 8 commits into from
Feb 7, 2017
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 11 additions & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,23 +24,29 @@ We feature [recommended starter-kyts](/docs/commands.md#recommended-starter-kyts

## kyt local development

Since kyt is a monorepo, it can be tricky to test local changes across packages. The following commands can assist in bootstrapping your local kyt repository:
To setup kyt for local development, install `yarn` and run the following:

```
git clone git@github.com:NYTimes/kyt.git
yarn run bootstrap
```
There are commands for bootstrapping, testing and linting all of the monorepo packages in the root directory package.json file.

### bootstrap

Bootstrap` will set you up with a clean slate. Every time it is run, it will remove and re-install the node_modules across all of the kyt packages. It will also link `kyt-cli` so you can run it locally on the command line.
Bootstrap` will set you up with a clean slate. Every time it is run, it will remove and re-install the node_modules across all of the packages, npm link `kyt-cli` and `kyt` so you can run them locally on the command line, and symlink local monorepo dependencies..
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you need to move the `


From the root of kyt, run:

`npm run bootstrap`
`yarn run bootstrap`

### update

Update is useful after you pull down kyt with some minor changes. It will call `npm install` on all of the packages in the repository. For complicated changes to the repository, it is best to use `bootstrap`.

From the root of kyt, run:

`npm run update`
`yarn run update`

### Testing local kyt-core changes

Expand All @@ -52,7 +58,7 @@ It is a common workflow to make changes to kyt-core and test them with `kyt-cli

To test setting up/installing local starter kyts, you need to specify the `--repository-path` option. The following command will create a test directory and install the given local kyt repository and reference the starter kyt using the given repository path.

`kyt-cli setup -d test -r kyt/.git --repository-path packages/starter-kyts/kyt-starter-static`
`kyt-cli setup -d test -r kyt/.git --repository-path packages/kyt-starter-static`

### Testing kyt

Expand Down
58 changes: 58 additions & 0 deletions bootstrap.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
/* eslint-disable import/no-dynamic-require,global-require */
const fs = require('fs');
const path = require('path');
const spawn = require('child_process').spawnSync;

const type = process.env.type || 'upgrade'; // yarn update types
const logTask = msg => console.log(`👍 ${msg}`);

const installPackage = (at) => {
const result = spawn('yarn', [type], { stdio: 'inherit', cwd: at });
if (result.error) {
console.log(result.error);
process.exit(1);
}
logTask(`installed ${at}\n`);
};

const packages = fs.readdirSync('packages').reduce((pkgs, pkg) => {
const packagePath = path.join(process.cwd(), 'packages', pkg);
const packageJSON = path.join(packagePath, 'package.json');
try {
if (fs.statSync(packagePath).isDirectory() && fs.statSync(packageJSON).isFile()) {
pkgs.push({ path: packagePath, name: require(packageJSON).name });
}
} catch (e) { return pkgs; }
return pkgs;
}, []);

console.log(`\n🔥 Bootstrapping\n`);

// Install the root package.json first so that we can use shelljs.
installPackage(process.cwd());
const shell = require('shelljs');

// Install all of the monorepo packages.
packages.forEach(pkg => installPackage(pkg.path));

// Symlink monorepo package dependencies to local packages.
packages.forEach((pkg) => {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how does this system resolve dependency clashes when you flatten here? for example: if one of your monorepo packages depends on [given module]@2.2.0 and a sibling package depends on version [given module]@1.9.0. Do you assume that your project will include both versions of dependency?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of the kyt-specific dependencies in the monorepo should be in sync with the latest published versions. Symlinking makes it easy to do local development on a specific kyt package and its respective kyt dependencies (eg changes to kyt-core and kyt-utils). It's true that this makes the assumption that things will stay in sync, but the symlinks are also easy to override after bootstrap if there is an issue or you want to test against a specific version. Let me know if I can clarify more.

Copy link

@elisechant elisechant Feb 3, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so the assumption is that the last package which contains dependency of said version will be the chosen version as it was the last symlinked? Wondering if this has caused or might cause interdependency related errors in the past/future? Do you have a recommended approach to ensuring the dependencies as you say as "latest published versions"? do you suggest to use @latest for example? Or how do you sanity check this in terms of upgrading workflow?
Should this code be improved by emitting warnings in this event of version conflicts?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This tool is just for local development so you can make changes without having to constantly reinstall node_modules. All published packages will have pinned dependencies.

For example:
I want to add a new utility to kyt-utils to use in a command in kyt-core. When I bootstrap for local development kyt-utils is symlinked in my node_modules for kyt-core and I can make changes directly in my files and see those changes as I run and test kyt-core locally. When I'm ready to commit changes I would update and publish a new version of kyt-utils and then use that pinned version as the dependency of kyt-core.

Dependencies that we don't manage (eg. jest, shelljs) will always use versions from npm even in local development.

const packageJSON = require(path.join(pkg.path, 'package.json'));
const dependencies = Object.assign({}, packageJSON.dependencies, packageJSON.devDependencies);
packages.forEach((spkg) => {
if (dependencies.hasOwnProperty(spkg.name)) { // eslint-disable-line no-prototype-builtins
const to = path.join(spkg.path, 'node_modules', spkg.name);
shell.rm('-rf', to);
shell.ln('-sf', spkg.path, to);
logTask(`symlinked:\n${to} -> ${spkg.path}\n`);
}
});
});

// npm link kyt-cli and kyt
shell.exec('npm link', { cwd: path.join(process.cwd(), 'packages', 'kyt-cli') });
logTask('npm-linked kyt-cli\n');
shell.exec('npm link', { cwd: path.join(process.cwd(), 'packages', 'kyt-core') });
logTask('npm-linked kyt');

console.log(`\n✅ strapped\n`);
11 changes: 5 additions & 6 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
},
"jest": {
"testPathIgnorePatterns": [
"<rootDir>/packages/starter-kyts",
"<rootDir>/packages/kyt-starter-*",
"<rootDir>/packages/kyt-core/cli/actions/test.js",
"<rootDir>/e2e_tests"
],
Expand All @@ -23,16 +23,15 @@
"coveragePathIgnorePatterns": [
"<rootDir>/node_modules",
"<rootDir>/packages/*/node_modules",
"<rootDir>/packages/*/starter-kyts",
"<rootDir>/packages/kyt-starter-*",
"<rootDir>/coverage",
"<rootDir>/packages/kyt-core/utils/jest"
]
},
"scripts": {
"bootstrap": "./scripts/bootstrap.sh",
"update": "./scripts/install-modules.sh",
"yarn-update": "./scripts/yarn-install-modules.sh",
"test": "jest",
"bootstrap": "time node bootstrap.js",
"update": "type=install npm run bootstrap",
"test": "jest && cd packages/kyt-starter-universal && node_modules/kyt/cli/index.js test && cd ../kyt-starter-static && node_modules/kyt/cli/index.js test",
"test-watch": "jest --watch",
"test-coverage": "jest --coverage",
"e2e": "jest --config ./e2e_tests/jest.config.json --verbose --no-cache",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,12 @@
"babel-plugin-transform-runtime": "6.15.0",
"babel-preset-latest": "6.16.0"
},
"keywords": ["babel", "babel-preset", "kyt"],
"files": ["lib"]
"keywords": [
"babel",
"babel-preset",
"kyt"
],
"files": [
"lib"
]
}
Loading