clientv3: make saved snapshot readable by user only #9977

dlipovetsky · 2018-08-02T03:35:30Z

dlipovetsky · 2018-08-02T03:42:18Z

Note for reviewer: Previously, the file was opened with os.Create, which opens the file for reading and writing (os.O_RDWR). Because the file does not need to be read, in this change it is opened for writing only (os.O_WRONLY).

codecov-io · 2018-08-02T06:02:29Z

Codecov Report

Merging #9977 into master will decrease coverage by 0.1%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #9977      +/-   ##
==========================================
- Coverage   69.34%   69.24%   -0.11%     
==========================================
  Files         386      386              
  Lines       35914    35914              
==========================================
- Hits        24905    24868      -37     
- Misses       9212     9257      +45     
+ Partials     1797     1789       -8

Impacted Files	Coverage Δ
clientv3/snapshot/v3_snapshot.go	`64.75% <100%> (ø)`	⬆️
pkg/adt/interval_tree.go	`79.27% <0%> (-12.02%)`	⬇️
pkg/logutil/zap_grpc.go	`47.61% <0%> (-4.77%)`	⬇️
pkg/transport/listener.go	`58.67% <0%> (-4.09%)`	⬇️
mvcc/watcher.go	`96.29% <0%> (-3.71%)`	⬇️
clientv3/concurrency/election.go	`79.52% <0%> (-2.37%)`	⬇️
etcdctl/ctlv3/command/printer_simple.go	`72.48% <0%> (-1.35%)`	⬇️
etcdserver/api/rafthttp/peer.go	`76.53% <0%> (-1.12%)`	⬇️
mvcc/watchable_store.go	`82.8% <0%> (-0.71%)`	⬇️
etcdserver/v3_server.go	`73.19% <0%> (-0.47%)`	⬇️
... and 15 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 93be31d...c5c63ab. Read the comment docs.

hexfusion · 2018-08-02T12:16:22Z

clientv3/snapshot/v3_snapshot_test.go

@@ -24,6 +24,8 @@ import (
 	"testing"
 	"time"

+	"github.com/coreos/etcd/pkg/fileutil"
+
 	"github.com/coreos/etcd/clientv3"


extra newline

Will fix, thanks!

hexfusion · 2018-08-02T12:17:22Z

clientv3/snapshot/v3_snapshot.go

@@ -102,7 +102,7 @@ func (s *v3Manager) Save(ctx context.Context, cfg clientv3.Config, dbPath string
 	defer os.RemoveAll(partpath)

 	var f *os.File
-	f, err = os.Create(partpath)
+	f, err = os.OpenFile(partpath, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, fileutil.PrivateFileMode)
 	if err != nil {
 		return fmt.Errorf("could not open %s (%v)", partpath, err)


If possible could you verify this works on Windows?

Good point. I'll look into this.

The os.Chmod docs indicates that on Windows, this change will not affect file permissions:

On Windows, the mode must be non-zero but otherwise only the 0200 bit (owner writable) of mode is used; it controls whether the file's read-only attribute is set or cleared. attribute. The other bits are currently unused. Use mode 0400 for a read-only file and 0600 for a readable+writable file.

I'll run an experiment on Windows and update with the results.

hexfusion · 2018-08-02T12:34:58Z

@dlipovetsky, thank you for the PR a few nits here but otherwise, this works as expected for me locally. As the previous permissions of the snapshot are something that could be assumed in various workflows we will need to document this possible breaking change well. Also as I noted in the review if we could verify the OpenFile usage work fine in Windows I think that would be smart as I am not sure our tests cover this.

I guess the final question is should the client be in charge of this? While it should be up to the admin to provide a secure env I think that giving the user a little help and being proactive about something as important as a snapshot is a good thing and a step forward so I vote +1.

dlipovetsky · 2018-08-04T01:47:52Z

I ran etcdctl snapshot save and etcdctl snapshot restore on Windows 2012 Server. Both work as expected:

C:\Users\Administrator\Desktop>etcdctl.exe snapshot save test.snapshot
{"level":"info","ts":1533346962.415343,"caller":"snapshot/v3_snapshot.go:109","msg":"created temporary db file","path":"test.snapshot.part"}
{"level":"info","ts":1533346962.4163485,"caller":"snapshot/v3_snapshot.go:120","msg":"fetching snapshot","endpoint":"127.0.0.1:2379"}
{"level":"info","ts":1533346962.4261172,"caller":"snapshot/v3_snapshot.go:133","msg":"fetched snapshot","endpoint":"127.0.0.1:2379","took":0.0097689}
{"level":"info","ts":1533346962.4261172,"caller":"snapshot/v3_snapshot.go:142","msg":"saved","path":"test.snapshot"}
Snapshot saved at test.snapshot

C:\Users\Administrator\Desktop>etcdctl.exe snapshot restore --data-dir test.etcd test.snapshot
{"level":"info","ts":1533347158.2669117,"caller":"snapshot/v3_snapshot.go:278","msg":"restoring snapshot","path":"test.snapshot","wal-dir":"test.etcd\\member\\w
al","data-dir":"test.etcd","snap-dir":"test.etcd\\member\\snap"}
{"level":"info","ts":1533347158.353891,"caller":"membership/cluster.go:343","msg":"added member","cluster-id":"cdf818194e3a8c32","local-member-id":"0","added-pe
er-id":"8e9e05c52164694d","added-peer-peer-urls":["http://localhost:2380"]}
{"level":"info","ts":1533347158.3782656,"caller":"wal/wal.go:252","msg":"closing WAL to release flock and retry directory renaming","from":"test.etcd\\member\\w
al.tmp","to":"test.etcd\\member\\wal"}
{"level":"info","ts":1533347158.4029384,"caller":"snapshot/v3_snapshot.go:291","msg":"restored snapshot","path":"test.snapshot","wal-dir":"test.etcd\\member\\wa
l","data-dir":"test.etcd","snap-dir":"test.etcd\\member\\snap"}

For completeness, here's the top of the etcd log (with my commit in the version):

C:\Users\Administrator\Desktop>etcd.exe
2018-08-04 01:50:47.080394 I | etcdmain: etcd Version: 3.3.0+git
2018-08-04 01:50:47.109698 I | etcdmain: Git SHA: c5c63abf5
2018-08-04 01:50:47.109698 I | etcdmain: Go Version: go1.9.2
2018-08-04 01:50:47.109698 I | etcdmain: Go OS/Arch: windows/amd64
2018-08-04 01:50:47.109698 I | etcdmain: setting maximum number of CPUs to 2, total number of available CPUs is 2
2018-08-04 01:50:47.110660 N | etcdmain: failed to detect default host (default host not supported on windows_amd64)
2018-08-04 01:50:47.110660 W | etcdmain: no data-dir provided, using default data-dir ./default.etcd
2018-08-04 01:50:47.110660 N | etcdmain: the server is already initialized as member before, starting as etcd member...
2018-08-04 01:50:47.115535 I | embed: name = default
2018-08-04 01:50:47.115535 I | embed: data dir = default.etcd
2018-08-04 01:50:47.116496 I | embed: member dir = default.etcd\member
2018-08-04 01:50:47.116496 I | embed: heartbeat = 100ms
2018-08-04 01:50:47.116496 I | embed: election = 1000ms
2018-08-04 01:50:47.116496 I | embed: snapshot count = 100000
2018-08-04 01:50:47.116496 I | embed: advertise client URLs = http://localhost:2379
2018-08-04 01:50:47.116496 I | embed: initial advertise peer URLs = http://localhost:2380
2018-08-04 01:50:47.116496 I | embed: initial cluster =
2018-08-04 01:50:47.119409 W | snap: skipped unexpected non snapshot file db.lock
2018-08-04 01:50:47.120399 I | etcdserver: restarting member 8e9e05c52164694d in cluster cdf818194e3a8c32 at commit index 4
{"level":"info","ts":1533347447.120399,"caller":"raft/raft.go:663","msg":"8e9e05c52164694d became follower at term 2"}
{"level":"info","ts":1533347447.120399,"caller":"raft/raft.go:365","msg":"newRaft 8e9e05c52164694d [peers: [], term: 2, commit: 4, applied: 0, lastindex: 4, las
tterm: 2]"}
2018-08-04 01:50:47.157497 W | auth: simple token is not cryptographically signed
2018-08-04 01:50:47.167263 I | etcdserver: starting server... [version: 3.3.0+git, cluster version: to_be_decided]
2018-08-04 01:50:47.170215 E | etcdserver: cannot monitor file descriptor usage (cannot get FDUsage on windows)
2018-08-04 01:50:47.171177 I | embed: listening for peers on 127.0.0.1:2380
2018-08-04 01:50:47.171177 I | etcdserver/membership: added member 8e9e05c52164694d [http://localhost:2380] to cluster cdf818194e3a8c32
2018-08-04 01:50:47.172151 N | etcdserver/membership: set the initial cluster version to 3.3
2018-08-04 01:50:47.172151 I | etcdserver/api: enabled capabilities for version 3.3
{"level":"info","ts":1533347448.321635,"caller":"raft/raft.go:858","msg":"8e9e05c52164694d is starting a new election at term 2"}
{"level":"info","ts":1533347448.321635,"caller":"raft/raft.go:676","msg":"8e9e05c52164694d became candidate at term 3"}
{"level":"info","ts":1533347448.321635,"caller":"raft/raft.go:756","msg":"8e9e05c52164694d received MsgVoteResp from 8e9e05c52164694d at term 3"}
{"level":"info","ts":1533347448.3225787,"caller":"raft/raft.go:713","msg":"8e9e05c52164694d became leader at term 3"}
{"level":"info","ts":1533347448.3225787,"caller":"raft/node.go:306","msg":"raft.node: 8e9e05c52164694d elected leader 8e9e05c52164694d at term 3"}
2018-08-04 01:50:48.323612 I | etcdserver: published {Name:default ClientURLs:[http://localhost:2379]} to cluster cdf818194e3a8c32
2018-08-04 01:50:48.323612 I | embed: ready to serve client requests
2018-08-04 01:50:48.324524 N | embed: serving insecure client requests on 127.0.0.1:2379, this is strongly discouraged!

dlipovetsky · 2018-08-04T01:55:52Z

@hexfusion I think my experiment demonstrates that this PR does not break snapshot save functionality on Windows. I also addressed your review comment. Once I hear back from you, and if there are no other review comments, I'll squash the commits. Thanks!

hexfusion · 2018-08-06T13:17:09Z

@dlipovetsky as you may have seen a lot of effort is being put into the CNCF TOC meeting which is tomorrow. Please allow a few days for this PR to get squared away as I want to get additional feedback from the community. Thank you for the detailed Windows tests I am very happy we had no regression there.

dlipovetsky · 2018-08-06T19:59:04Z

@hexfusion Thanks for your feedback! I'll keep an eye for an update.

gyuho · 2018-08-08T17:39:52Z

Can you fix https://travis-ci.org/coreos/etcd/jobs/411980747#L607?

This should be safe, since we also use private file mode for WAL. And when snapshot file is fetched from remote storage, the user can always updates its file permission for their own use case.

dlipovetsky · 2018-08-08T19:45:53Z

@gyuho Thanks for your feedback. The test failure should be be resolved when I squash the commits. I'll do that now.

Fixes etcd-io#9976

hexfusion · 2018-08-08T20:34:18Z

@dlipovetsky thank you for your contribution, I hope this is the first of many.

gyuho · 2018-08-08T20:35:27Z

@dlipovetsky Could you also add change log here https://github.com/coreos/etcd/blob/master/CHANGELOG-3.4.md#client-v3, so people know who worked on it?

dlipovetsky · 2018-08-08T23:50:34Z

@hexfusion Thanks for guiding me along!
@gyuho Will submit a PR for the changelog, thanks!

dlipovetsky force-pushed the etcdctl-snapshot-permissions branch from 8ac1e51 to c5c63ab Compare August 2, 2018 04:59

hexfusion suggested changes Aug 2, 2018

View reviewed changes

clientv3: Make saved snapshot readable by user only

ddde272

Fixes etcd-io#9976

dlipovetsky force-pushed the etcdctl-snapshot-permissions branch from 418b7ac to ddde272 Compare August 8, 2018 19:46

hexfusion approved these changes Aug 8, 2018

View reviewed changes

hexfusion added the Release-Note label Aug 8, 2018

hexfusion merged commit 2fe4918 into etcd-io:master Aug 8, 2018

dlipovetsky mentioned this pull request Aug 9, 2018

CHANGELOG: Add 3.4 changelog entry for snapshot file permissions change #9991

Merged

hexfusion removed the Release-Note label Aug 9, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

clientv3: make saved snapshot readable by user only #9977

clientv3: make saved snapshot readable by user only #9977

dlipovetsky commented Aug 2, 2018

dlipovetsky commented Aug 2, 2018

codecov-io commented Aug 2, 2018

hexfusion Aug 2, 2018

dlipovetsky Aug 2, 2018

hexfusion Aug 2, 2018

dlipovetsky Aug 2, 2018

hexfusion commented Aug 2, 2018 •

edited

Loading

dlipovetsky commented Aug 4, 2018 •

edited

Loading

dlipovetsky commented Aug 4, 2018

hexfusion commented Aug 6, 2018

dlipovetsky commented Aug 6, 2018

gyuho commented Aug 8, 2018

dlipovetsky commented Aug 8, 2018

hexfusion commented Aug 8, 2018

gyuho commented Aug 8, 2018

dlipovetsky commented Aug 8, 2018

clientv3: make saved snapshot readable by user only #9977

clientv3: make saved snapshot readable by user only #9977

Conversation

dlipovetsky commented Aug 2, 2018

dlipovetsky commented Aug 2, 2018

codecov-io commented Aug 2, 2018

Codecov Report

hexfusion Aug 2, 2018

Choose a reason for hiding this comment

dlipovetsky Aug 2, 2018

Choose a reason for hiding this comment

hexfusion Aug 2, 2018

Choose a reason for hiding this comment

dlipovetsky Aug 2, 2018

Choose a reason for hiding this comment

hexfusion commented Aug 2, 2018 • edited Loading

dlipovetsky commented Aug 4, 2018 • edited Loading

dlipovetsky commented Aug 4, 2018

hexfusion commented Aug 6, 2018

dlipovetsky commented Aug 6, 2018

gyuho commented Aug 8, 2018

dlipovetsky commented Aug 8, 2018

hexfusion commented Aug 8, 2018

gyuho commented Aug 8, 2018

dlipovetsky commented Aug 8, 2018

hexfusion commented Aug 2, 2018 •

edited

Loading

dlipovetsky commented Aug 4, 2018 •

edited

Loading