Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

overlay: add overlay implementation #1544

Merged
merged 1 commit into from
Mar 15, 2024
Merged

overlay: add overlay implementation #1544

merged 1 commit into from
Mar 15, 2024

Conversation

WeiZhang555
Copy link
Contributor

@WeiZhang555 WeiZhang555 commented Jan 25, 2024 •
edited
Loading

With help of newly introduced Overlay FileSystem in fuse-backend-rs library, now we can create writable rootfs in Nydus. Implementation of writable rootfs is based on one passthrough FS(as upper layer) over one readonly rafs(as lower layer).

To do so, configuration is extended with some Overlay options.

Details

An userspace overlay filesystem is usable for control of user data, with this implementation, Nydus can have stronger control on container rootfs.

Types of changes

What types of changes does your PullRequest introduce? Put an x in all the boxes that apply:

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation Update (if none of the other choices apply)

Checklist

Go over all the following points, and put an x in all the boxes that apply.

  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.

@WeiZhang555 WeiZhang555 requested a review from a team as a code owner January 25, 2024 07:35
@WeiZhang555 WeiZhang555 requested review from imeoer, changweige and adamqqqplay and removed request for a team January 25, 2024 07:35
@WeiZhang555 WeiZhang555 changed the title overlay: add overlay implementation [wip]overlay: add overlay implementation Jan 25, 2024
@WeiZhang555
Copy link
Contributor Author

The PR is not ready for merging since it rely heavily on fuse-backend-rs changes(see PR ), this PR can be regarded as a simple demo usage of userspace Overlay FS currently.

@codecov Codecov
Copy link

codecov bot commented Jan 25, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 11.68831% with 68 lines in your changes are missing coverage. Please review.

Project coverage is 61.22%. Comparing base (3dfa9e9) to head (e2bc4bf).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1544      +/-   ##
==========================================
- Coverage   61.29%   61.22%   -0.08%     
==========================================
  Files         144      144              
  Lines       47035    47100      +65     
  Branches    44537    44602      +65     
==========================================
+ Hits        28832    28838       +6     
- Misses      16719    16778      +59     
  Partials     1484     1484
Files Coverage Δ
builder/src/core/context.rs 67.06% <100.00%> (+0.03%) ⬆️
api/src/config.rs 87.58% <80.00%> (-0.03%) ⬇️
storage/src/device.rs 69.26% <0.00%> (+0.22%) ⬆️
rafs/src/fs.rs 34.33% <28.57%> (-0.20%) ⬇️
service/src/fs_service.rs 40.92% <3.27%> (-10.20%) ⬇️

... and 1 file with indirect coverage changes

@WeiZhang555 WeiZhang555 mentioned this pull request Jan 25, 2024
Merged
@WeiZhang555 WeiZhang555 force-pushed the overlay-impl branch 3 times, most recently from a876b1f to faec9e3 Compare February 1, 2024 03:18
@imeoer
Copy link
Collaborator

imeoer commented Feb 20, 2024

Thanks for the implementation! Please also add a test case in https://github.com/dragonflyoss/nydus/tree/master/smoke/tests (maybe create a overlay_writable_test.go ?).

imeoer
imeoer reviewed Feb 27, 2024
View reviewed changes
storage/src/device.rs Outdated Show resolved Hide resolved
service/src/fs_service.rs Outdated Show resolved Hide resolved
imeoer
imeoer reviewed Feb 27, 2024
View reviewed changes
service/src/fs_service.rs Outdated Show resolved Hide resolved
imeoer
imeoer reviewed Feb 27, 2024
View reviewed changes
service/src/fs_service.rs Outdated Show resolved Hide resolved
imeoer
imeoer reviewed Feb 27, 2024
View reviewed changes
Copy link
Collaborator

@imeoer imeoer left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should get macOS to pass unit tests as well: https://github.com/dragonflyoss/nydus/actions/runs/7735769409/job/21091979887?pr=1544

others LGTM, thanks!

@WeiZhang555 WeiZhang555 force-pushed the overlay-impl branch 3 times, most recently from 46173f5 to a7d12c3 Compare March 6, 2024 08:55
@WeiZhang555
Copy link
Contributor Author

Maybe we should get macOS to pass unit tests as well: https://github.com/dragonflyoss/nydus/actions/runs/7735769409/job/21091979887?pr=1544

others LGTM, thanks!

Smoke test cases were added and passed.

imeoer
imeoer reviewed Mar 11, 2024
View reviewed changes
smoke/tests/tool/nydusd.go Outdated Show resolved Hide resolved
smoke/tests/overlay_fs_test.go Outdated Show resolved Hide resolved
smoke/tests/overlay_fs_test.go Outdated Show resolved Hide resolved
@imeoer
Copy link
Collaborator

imeoer commented Mar 11, 2024

image

It seems the CI broken, others LGTM!

@WeiZhang555 WeiZhang555 force-pushed the overlay-impl branch 3 times, most recently from 0f43960 to 8c75893 Compare March 12, 2024 06:51
@WeiZhang555 WeiZhang555 changed the title [wip]overlay: add overlay implementation overlay: add overlay implementation Mar 12, 2024
@imeoer
Copy link
Collaborator

imeoer commented Mar 13, 2024 •
edited
Loading

Thanks, LGTM! We can also add the doc in https://github.com/dragonflyoss/nydus/blob/master/docs/nydusd.md, may be Enable Overlay Writable ?

@WeiZhang555 WeiZhang555 force-pushed the overlay-impl branch 3 times, most recently from b8e6050 to cdbfa27 Compare March 15, 2024 02:41
@WeiZhang555
Copy link
Contributor Author

@imeoer docs updated to docs/nydus-overlayfs.md

@WeiZhang555
overlay: add overlay implementation
e2bc4bf 
With help of newly introduced Overlay FileSystem in `fuse-backend-rs`
library, now we can create writable rootfs in Nydus. Implementation of
writable rootfs is based on one passthrough FS(as upper layer) over one
readonly rafs(as lower layer).

To do so, configuration is extended with some Overlay options.

Signed-off-by: Wei Zhang <weizhang555.zw@gmail.com>
@imeoer
Copy link
Collaborator

imeoer commented Mar 15, 2024

LGTM, thanks!

@imeoer imeoer merged commit 9ae8e3a into dragonflyoss:master Mar 15, 2024
25 checks passed
@WeiZhang555 WeiZhang555 deleted the overlay-impl branch March 15, 2024 06:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@imeoer imeoer imeoer approved these changes

@changweige changweige Awaiting requested review from changweige changweige is a code owner automatically assigned from dragonflyoss/nydus-reviewers

@adamqqqplay adamqqqplay Awaiting requested review from adamqqqplay adamqqqplay is a code owner automatically assigned from dragonflyoss/nydus-reviewers

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@WeiZhang555 @imeoer