fs: add recursive cp method #39372

bcoe · 2021-07-13T00:52:22Z

Introduces recursive cp method, based on fs-extra implementation

Refs: nodejs/tooling#98
Fixes: #35880

Opening to start conversation.

TODO:

still have many tests to write, to increase coverage. (coverage is at 92%).
need to document sync and promise API.

CC: @nodejs/tooling, @jprichardson, @manidlou, @RyanZim

test/parallel/test-copy.js

lib/internal/fs/copy/copy.js

lib/internal/fs/promises.js

lib/internal/fs/copy/copy.js

lib/internal/fs/copy/copy-sync.js

mcollina

lgtm

targos

Just making sure this doesn't land without documentation for the Promises and sync APIs

lib/internal/fs/copy/copy-sync.js

iansu

Looks good overall! 🙌

lib/internal/fs/promises.js

lib/internal/fs/utils.js

bcoe · 2021-07-15T15:33:13Z

@mcollina @targos small, but significant update, I thought force would be a better option than overwrite for when clobbering the destination folder, and have renamed it to this. Also, force now defaults to false.

nodejs-github-bot · 2021-07-15T15:33:47Z

CI: https://ci.nodejs.org/job/node-test-pull-request/39071/

lib/internal/fs/copy/copy-sync.js

RyanZim · 2021-07-15T19:23:23Z

I thought force would be a better option than overwrite for when clobbering the destination folder, and have renamed it to this. Also, force now defaults to false.

While we're rethinking options, should errorOnExist default to false or true? This is another one of those options that has its value for compat reasons.

RyanZim · 2021-07-15T19:27:21Z

Also, for reference; discussion thread on how fs-extra should handle the forthcoming naming conflict: jprichardson/node-fs-extra#912

aduh95

Further simplifications on the subject of replacing open + futimes + close with utimes.

lib/internal/fs/copy/copy-sync.js

lib/internal/fs/copy/copy.js

bcoe · 2021-07-15T22:06:54Z

@RyanZim @iansu

While we're rethinking options, should errorOnExist default to false or true? This is another one of those options that has its value for compat reasons.

I was tempted to go with false, which I believe would make copying to the same folder twice relatively safe, i.e., you don't clobber files, but it's less of a pain in the neck to recover from a partial copy (you don't have to lookup an option).

Thoughts?

lib/internal/fs/utils.js

doc/api/errors.md

doc/api/fs.md

lib/internal/fs/copy/copy-sync.js

lib/internal/fs/copy/copy.js

bcoe · 2021-07-22T15:06:57Z

FYI, waiting on a windows machine at work (there are some hiccups with provisioning). Would happily pair with someone on fixing up the failing windows test, if anyone has a system up and running.

lib/internal/fs/promises.js

Introduces recursive cp method, based on fs-extra implementation.

aduh95 · 2021-08-08T21:45:18Z

lib/internal/fs/cp/cp-sync.js

+    if (isPromise(shouldCopy)) {
+      throw new ERR_INVALID_RETURN_VALUE('boolean', 'filter', shouldCopy);


Non blocking reflection, but probably something worth discussing before calling the API stable:
I'm not sure checking isPromise is the right thing to do here, %Array.prototype.filter% does not handle promises differently (JSON.stringify([0, 1].filter(async()=>false)) === '[0,1]'). Maybe we could emit a warning instead of throwing? That may help users spot if they have passed by mistake an async function, but wouldn't crash if they just happen to return a Promise object here.

The reason that Array.prototype.filter doesn't handle promises differently is mostly historical. I can't think of a correct use case where the user wants to return a Promise and expects it to be treated as a truthy value.

mcollina

lgtm

nodejs-github-bot · 2021-08-11T17:30:55Z

CI: https://ci.nodejs.org/job/node-test-pull-request/39541/

Introduces recursive cp method, based on fs-extra implementation. PR-URL: #39372 Fixes: #35880 Refs: nodejs/tooling#98 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ian Sutherland <ian@iansutherland.ca>

bcoe · 2021-08-12T02:53:56Z

Landed in 87d6fd7

Introduces recursive cp method, based on fs-extra implementation. PR-URL: #39372 Fixes: #35880 Refs: nodejs/tooling#98 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Ian Sutherland <ian@iansutherland.ca>

Notable changes: * fs: * add recursive cp method (Benjamin Coe) #39372

Notable changes: * fs: * add recursive cp method (Benjamin Coe) #39372 PR-URL: #39782

Notable changes: * fs: * experimental: add recursive cp method (Benjamin Coe) #39372 PR-URL: #39782

lostpebble · 2021-08-19T08:31:49Z

Is there any place to give feedback on this function?

Just looking through the docs, I think the option force: <boolean> is a bit ambiguous and would have been better suited to be named overwrite- as that's literally what it achieves, according to the docs.

[EDIT]
Sorry, after reading further comments, I see it was actually changed from overwrite to force - which I'm a little confused about now...

bcoe · 2021-08-26T14:56:22Z

@lostpebble please feel free to open an issue. As much as possible I'd like to model after cp's flags, but perhaps clobber is the better name in this case?

One good thing about force is it's consistent with rm.

As per nodejs/node#39372 (comment) Resolves #918

nodejs-github-bot added lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Jul 13, 2021

bcoe commented Jul 13, 2021

View reviewed changes

test/parallel/test-copy.js Outdated Show resolved Hide resolved

bcoe commented Jul 13, 2021

View reviewed changes

lib/internal/fs/copy/copy.js Outdated Show resolved Hide resolved

bcoe commented Jul 13, 2021

View reviewed changes

lib/internal/fs/copy/copy.js Outdated Show resolved Hide resolved

bcoe force-pushed the copy branch from bb5fe4a to 4107b45 Compare July 13, 2021 01:02

This comment has been minimized.

Sign in to view

aduh95 reviewed Jul 13, 2021

View reviewed changes

mcollina approved these changes Jul 13, 2021

View reviewed changes

targos requested changes Jul 13, 2021

View reviewed changes

merceyz reviewed Jul 13, 2021

View reviewed changes

lib/internal/fs/copy/copy-sync.js Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

iansu reviewed Jul 14, 2021

View reviewed changes

lib/internal/fs/promises.js Outdated Show resolved Hide resolved

lib/internal/fs/utils.js Outdated Show resolved Hide resolved

bcoe added notable-change PRs with changes that should be highlighted in changelogs. semver-minor PRs that contain new features and should be released in the next minor version. labels Jul 15, 2021

aduh95 reviewed Jul 15, 2021

View reviewed changes

lib/internal/fs/copy/copy-sync.js Outdated Show resolved Hide resolved

RyanZim mentioned this pull request Jul 15, 2021

fs.cp coming to Node.js core jprichardson/node-fs-extra#912

Open

aduh95 reviewed Jul 15, 2021

View reviewed changes

lib/internal/fs/copy/copy-sync.js Outdated Show resolved Hide resolved

lib/internal/fs/copy/copy-sync.js Outdated Show resolved Hide resolved

lib/internal/fs/copy/copy.js Outdated Show resolved Hide resolved

lib/internal/fs/copy/copy.js Outdated Show resolved Hide resolved

targos reviewed Jul 16, 2021

View reviewed changes

lib/internal/fs/utils.js Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

manidlou reviewed Jul 18, 2021

View reviewed changes

bcoe force-pushed the copy branch from f94a533 to 83a403c Compare July 19, 2021 02:34

bcoe force-pushed the copy branch from 5988d2c to 86d3eeb Compare July 25, 2021 20:25

Mesteery reviewed Aug 6, 2021

View reviewed changes

lib/internal/fs/promises.js Outdated Show resolved Hide resolved

iansu approved these changes Aug 6, 2021

View reviewed changes

joesepi approved these changes Aug 6, 2021

View reviewed changes

fs: add recursive cp method

63b1c80

Introduces recursive cp method, based on fs-extra implementation.

bcoe force-pushed the copy branch from e52047b to 63b1c80 Compare August 8, 2021 21:26

This comment has been minimized.

Sign in to view

aduh95 approved these changes Aug 8, 2021

View reviewed changes

mcollina approved these changes Aug 9, 2021

View reviewed changes

This comment has been minimized.

Sign in to view

RyanZim mentioned this pull request Aug 10, 2021

Port copy fix from Node.js core jprichardson/node-fs-extra#918

Closed

bcoe closed this Aug 12, 2021

danielleadams added a commit that referenced this pull request Aug 16, 2021

2021-08-17, Version 16.7.0 (Current)

eb1a14c

Notable changes: * fs: * add recursive cp method (Benjamin Coe) #39372

danielleadams mentioned this pull request Aug 16, 2021

v16.7.0 release proposal #39782

Merged

danielleadams added a commit that referenced this pull request Aug 16, 2021

2021-08-17, Version 16.7.0 (Current)

bb06ce8

Notable changes: * fs: * add recursive cp method (Benjamin Coe) #39372 PR-URL: #39782

aduh95 added the experimental Issues and PRs related to experimental features. label Aug 16, 2021

danielleadams added a commit that referenced this pull request Aug 17, 2021

2021-08-17, Version 16.7.0 (Current)

d5987f8

Notable changes: * fs: * experimental: add recursive cp method (Benjamin Coe) #39372 PR-URL: #39782

danielleadams added a commit that referenced this pull request Aug 18, 2021

2021-08-17, Version 16.7.0 (Current)

7ca38f0

Notable changes: * fs: * experimental: add recursive cp method (Benjamin Coe) #39372 PR-URL: #39782

targos added the dont-land-on-v14.x label Sep 4, 2021

theoludwig mentioned this pull request Sep 11, 2022

Stabilize fs.cp(), fs.cpSync(), fsPromises.cp() methods #44598

Closed

RyanZim added a commit to jprichardson/node-fs-extra that referenced this pull request Oct 25, 2022

Remove unneeded and buggy stats check

fe73a03

As per nodejs/node#39372 (comment) Resolves #918

RyanZim mentioned this pull request Oct 25, 2022

Remove unneeded and buggy stats check jprichardson/node-fs-extra#976

Merged

RyanZim added a commit to jprichardson/node-fs-extra that referenced this pull request Nov 19, 2022

Remove unneeded and buggy stats check (#976)

1a3205d

As per nodejs/node#39372 (comment) Resolves #918

targos mentioned this pull request Apr 24, 2024

fs: ensure return value type of options.filter in cpSync consistent with doc #52461

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fs: add recursive cp method #39372

fs: add recursive cp method #39372

bcoe commented Jul 13, 2021 •

edited by aduh95

Loading

This comment has been minimized.

mcollina left a comment

targos left a comment

This comment has been minimized.

This comment has been minimized.

iansu left a comment

bcoe commented Jul 15, 2021

nodejs-github-bot commented Jul 15, 2021

RyanZim commented Jul 15, 2021

RyanZim commented Jul 15, 2021

aduh95 left a comment

bcoe commented Jul 15, 2021

This comment has been minimized.

bcoe commented Jul 22, 2021

This comment has been minimized.

aduh95 Aug 8, 2021

targos Aug 9, 2021

mcollina left a comment

This comment has been minimized.

This comment has been minimized.

nodejs-github-bot commented Aug 11, 2021

bcoe commented Aug 12, 2021

lostpebble commented Aug 19, 2021 •

edited

Loading

bcoe commented Aug 26, 2021

		if (isPromise(shouldCopy)) {
		throw new ERR_INVALID_RETURN_VALUE('boolean', 'filter', shouldCopy);

fs: add recursive cp method #39372

fs: add recursive cp method #39372

Conversation

bcoe commented Jul 13, 2021 • edited by aduh95 Loading

This comment has been minimized.

mcollina left a comment

Choose a reason for hiding this comment

targos left a comment

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

iansu left a comment

Choose a reason for hiding this comment

bcoe commented Jul 15, 2021

nodejs-github-bot commented Jul 15, 2021

RyanZim commented Jul 15, 2021

RyanZim commented Jul 15, 2021

aduh95 left a comment

Choose a reason for hiding this comment

bcoe commented Jul 15, 2021

This comment has been minimized.

bcoe commented Jul 22, 2021

This comment has been minimized.

aduh95 Aug 8, 2021

Choose a reason for hiding this comment

targos Aug 9, 2021

Choose a reason for hiding this comment

mcollina left a comment

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

nodejs-github-bot commented Aug 11, 2021

bcoe commented Aug 12, 2021

lostpebble commented Aug 19, 2021 • edited Loading

bcoe commented Aug 26, 2021

bcoe commented Jul 13, 2021 •

edited by aduh95

Loading

lostpebble commented Aug 19, 2021 •

edited

Loading