Feature: consider moving js-client deploy logic to the CLI

[erezrokah]

At the moment most of the CLI deploy command logic in handled by the js-client:

js-client/src/deploy/index.js

Line 42 in 9d3945c

const { fnDir, configPath, statusCb, message: title } = opts

This seems out of place for a client that should be wrapping our open-api spec.
Also, it prevents this client from being browser compatible.

It might makes sense to move that logic to the CLI, since we're already implementing some deploy logic in the CLI as a part of edge handlers:
netlify/cli#1244

[ehmicky]

💯

[erezrokah]

Doing this would also allow for better error handling as we could try/catch on each deploy related operation instead of catching everything here:
https://github.com/netlify/cli/blob/7db9ce56c26d25ca43631daaf5d32c3140e15f6a/src/commands/deploy.js#L228

[svdoever]

Please don't - I want to use the deploy logic of this library within my own application but NOT depend on netlify-cli. I'm running within a cramped VM with 200 MB diskspace (using https://glitch.com), so could NOT use netlify-cli, because installing that package downloads 75MB (mostly due to typescript - Why?!):

$ npm ls typescript
netlify-test@1.0.0 /mnt/c/Users/serge/Downloads/netlify
└─┬ netlify@4.9.0
  └─┬ @netlify/zip-it-and-ship-it@1.4.0
    └─┬ precinct@6.3.1
      └─┬ detective-typescript@5.8.0
        └── typescript@3.9.7

Top 5 package sizes in netlify-dev:

$ du -sh -BK ./node_modules/* | sort -nr
53240K  ./node_modules/typescript
2304K   ./node_modules/lodash
1984K   ./node_modules/@babel
1928K   ./node_modules/string.prototype.trimstart
1928K   ./node_modules/string.prototype.trimend

That TypeScript is big, ok, but trimstart 2MB, and trimend 2MB... wow!

I just see that there is actually the same problem with the current netlify/js-client library which depends on the same stuff, so not useable as well!!! Such a pity...

And about netlify-cli, it even downloads all versions of the traffic-mesh-agent (windows, linux, osx), another whooping 90MB!

By the way... just looked up the actual implementation of trimEnd() (package string.prototype.trimstart) and trimStart() (package string.prototype.stringend) - good for 4MB in the package:

trimStart():

'use strict';

var callBind = require('es-abstract/helpers/callBind');
var replace = callBind(String.prototype.replace);

/* eslint-disable no-control-regex */
var startWhitespace = /^[\x09\x0A\x0B\x0C\x0D\x20\xA0\u1680\u180E\u2000\u2001\u2002\u2003\u2004\u2005\u2006\u2007\u2008\u2009\u200A\u202F\u205F\u3000\u2028\u2029\uFEFF]*/;
/* eslint-enable no-control-regex */

module.exports = function trimStart() {
	return replace(this, startWhitespace, '');
};

trimEnd():

'use strict';

var callBound = require('es-abstract/helpers/callBound');
var $replace = callBound('String.prototype.replace');

/* eslint-disable no-control-regex */
var endWhitespace = /[\x09\x0A\x0B\x0C\x0D\x20\xA0\u1680\u180E\u2000\u2001\u2002\u2003\u2004\u2005\u2006\u2007\u2008\u2009\u200A\u202F\u205F\u3000\u2028\u2029\uFEFF]*$/;
/* eslint-enable no-control-regex */

module.exports = function trimEnd() {
	return $replace(this, endWhitespace, '');
};

Combine those two and you are under 200 bytes:-) It is a crazy world....

[erezrokah]

Thanks for your input @svdoever. See more about package size here netlify/cli#494

[eduardoboucas]

Currently, CLI works with two instances of zip-it-and-ship-it: one as a direct dependency and another as a transitive dependency via netlify (the package published from this repo). This is not ideal and introduces unnecessary complexity and maintenance overhead.

Moving the bundling logic to CLI would consolidate them into one, while at the same time simplifying the netlify package, making it simply a wrapper over our Open API spec, as @erezrokah pointed out.

[erquhart]

Potentially related: #229

[eduardoboucas]

This logic has now been moved to the CLI. We now need to remove the logic from this package and release a new major version.

[South-Paw]

How we we call the deploy method programmatically in from the cli package?

netlify/cli doesn't appear (at a quick glance) to be a package I can just import into my code and call deploy after creating a client like I could before - so this was hardly a like for like change and that's a pretty frustrating oversight.

[erezrokah]

How we we call the deploy method programmatically in from the cli package?

Hi @South-Paw, would spawning the CLI as an external process work?
e.g.

const execa = require('execa');
const { stdout } = await execa.command('netlify deploy', { preferLocal: true });
// do something with stdout

You can also pass NETLIFY_AUTH_TOKEN and NETLIFY_SITE_ID as environment variables.

If that doesn't cover your use case, can you please open a feature request on the CLI repo?

[South-Paw]

Hi @erezrokah, thank you for the reply. I'm unsure if spawning a process for the deployment suits the use case?

The deploy is in the middle of a GitHub action which also manages the Netlify deploy messages, comments on the PR/Commit, creating GitHub deploy environments.

Ideally the action's code would handle all this, so I'd like to avoid spawning new processes to keep the GitHub action's runtime down.

Any advice would be appreciated, I'm more than happy to open a feature request if that's the next best step.

[erezrokah]

Hi @South-Paw, I actually think using the CLI would provide a better experience.

Users won't need to configure build-dir, functions-dir and config-path as the CLI resolves those based on the user's repo.

You can pass a json flag to have parsable output and a message flag to set the message. netlify deploy does draft deploys by default so you can pass prod for non drafts.

As for the performance considerations - I think the time it takes to spawn the process should be negligible compared to the time it takes to deploy, but if you experience otherwise it would be great if you can share it.

[IanVS]

I'm trying to figure out how to use the cli in a GitHub action. It's not straightforward, because GitHub says that actions should not rely on any other binaries that aren't preinstalled in their worker boxes. Do you have any ideas, @erezrokah? I think you're right that using the automatic resolution features of the cli would be nice, I'm just not sure how to bundle the cli along with my JavaScript action (I don't want to use a docker action in my situation).

[erezrokah]

Hi @IanVS, you can install the CLI as a regular dependency, e.g. via npm install netlify-cli, then with whatever tool you use to bundle the action, make sure it bundles the CLI.

[IanVS]

Thanks @erezrokah. Do you have any suggestions for bundling a binary out of node_modules into an executable? I've tried https://github.com/vercel/pkg and https://github.com/vercel/ncc without luck so far.

Edit: pkg almost worked, but the resulting file is too large for git.

I think this approach of bundling the cli is not really a great solution, but I'll keep playing with it. One reason it's difficult is that lazy imports are being used, which aren't being bundled correctly by ncc, and I'm not sure any bundler really can bundle them if they're not static imports.

It's a shame that https://github.com/netlify/actions seems to be a dead project with outdated docker files. I'm thinking that there's no good way to build a netlify github action without docker now that deploy has been removed from the js client. And docker actions only run on linux and are slower than javascript actions.

[South-Paw]

It's a shame that https://github.com/netlify/actions seems to be a dead project with outdated docker files. I'm thinking that there's no good way to build a netlify github action without docker now that deploy has been removed from the js client. And docker actions only run on linux and are slower than javascript actions.

Yeah, echoing a lot of what I've found after switching over to using netlify/actions @IanVS instead of South-Paw/action-netlify-deploy. The readme's are also out of date, PRs that update them haven't been merged and there are typo's in the scripts and bugs & gaps in the action outputs to top it off! 😆

The netlify/actions & netlify/cli add about 1-2 mins of extra time to an action because they have to build themselves in the workflow every run instead of being packaged and ready to roll as the JavaScript actions that used deploy from the client were. This is resulting in a small amount of wasted action minutes every month for organisations as well 😞

It would be super to see the Netlify actions repo get some love from the Netlify team (including publishing them to the GH marketplace) OR it would be just as great to have a better replacement for the removal of the deploy function so I can update the original action as an alternative to netlify/actions repo

[mikepianka]

Hey everyone - I'm just commenting here to see if anyone can point me in the right direction because I went down a rabbit hole this afternoon full of dead ends and this thread affirmed why I was hitting issues (netlify/actions abandoned with bugs, the js client dropping support for deploys).

I have a situation where I have a GitHub action that is manually triggered and creates new repositories from a template. I'd like to add another action which piggybacks off that and programmatically sets up a new site in Netlify for those newly created repos. So I don't have a site ID, I want Netlify to just setup the site at a randomly named url via the GitHub action and then I'll put that url in the repo's readme.

Anyone have any suggestions on what the best approach is with current Netlify tooling? Thanks!

@South-Paw @IanVS @erezrokah