Fuse OverlayFs implementation #156
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
WeiZhang555
requested review from
bergwolf,
imeoer,
jiangliu,
liubogithub and
eryugey
as code owners
WeiZhang555
force-pushed
the
overlay-impl
branch
from
2b5ca30
to
cd5987c
Compare
|
xfstests CI action costs about 39min, one example log: https://github.com/WeiZhang555/fuse-backend-rs/actions/runs/6322500147/job/17168241410 |
Closed
WeiZhang555
force-pushed
the
overlay-impl
branch
3 times, most recently
from
6031c63
to
48dcb15
Compare
WeiZhang555
force-pushed
the
overlay-impl
branch
3 times, most recently
from
dc5b618
to
e30c271
Compare
jiangliu
reviewed
Nov 23, 2023
jiangliu
reviewed
Nov 23, 2023
jiangliu
reviewed
Nov 23, 2023
| @@ -30,6 +30,11 @@ pub use async_io::{AsyncFileSystem, AsyncZeroCopyReader, AsyncZeroCopyWriter}; | |||
| mod sync_io; | |||
| pub use sync_io::FileSystem; | |||
|
|
|||
| #[cfg(all(any(feature = "fusedev", feature = "virtiofs"), target_os = "linux"))] | |||
There was a problem hiding this comment.
It would be preferrable to mount overlayfs inside guest, so is there any need to turn on overlayfs for virtiofs?
There was a problem hiding this comment.
The idea of Fuse OverlayFS is to take over Container RootFs totally through Nydus and Fuse-backend-rs, so I suppose mounting overlayfs on host is strong requirement.
Besides, the OverlayFs replies heavily on Passthroughfs currently, so I copied #[cfg] for 'passthroughfs' from src/lib.rs:
#[cfg(all(any(feature = "fusedev", feature = "virtiofs"), target_os = "linux"))]
pub mod passthrough;
df8537a
to
1af13d1
Compare
imeoer
reviewed
Jan 10, 2024
WeiZhang555
force-pushed
the
overlay-impl
branch
6 times, most recently
from
1fdbb6b
to
1b20bbd
Compare
|
Add unionmount-testsuite as Overlay function test cases, this gave us more confidence on the functionality. |
hhhhsdxxxx
reviewed
Jan 18, 2024
WeiZhang555
force-pushed
the
overlay-impl
branch
2 times, most recently
from
8f623a5
to
757cec5
Compare
WeiZhang555
force-pushed
the
overlay-impl
branch
from
757cec5
to
b9a541f
Compare
6 tasks
|
The Integration into Nydus can be seen in Nydus PR |
WeiZhang555
force-pushed
the
overlay-impl
branch
3 times, most recently
from
a001087
to
e5bce4a
Compare
|
ping @hhhhsdxxxx @jiangliu @eryugey |
|
lgtm |
hhhhsdxxxx
approved these changes
Feb 20, 2024
Add first implementation of overlay fs which can imitate kernel overlay fs. This allow you to customize a union fs with many customized layers, for example, one passthrough fs as upper layer and one NFS as lower layer. This can be quite useful for some projects relying on fuse-backend-rs, e.g. Nydus, which can add writable upper layer on readonly RAFS lower layer, to act as a complete container rootfs. More details will be added in docs. Signed-off-by: Yang Bo <bo@hyper.sh>
WeiZhang555
force-pushed
the
overlay-impl
branch
from
e5bce4a
to
a9c41b1
Compare
This commit refactor a lot to previous one, including: * Modified layer trait * Lots of bugfixes to make xfstests happy. * Performance improvement. * Whiteout logics * Remove plugin & Direct layer logics. * A better inode allocator with inode consistency guarantee. Signed-off-by: Wei Zhang <weizhang555.zw@gmail.com>
Add xfstests into CI for overlayfs, it will work as Github Actions. Currently not all test cases can be run, only about half of `generic/*` for the Fuse Overlay FS, some test cases are expected to fail due to missing function implementation, little are unexpected due to potential bugs which will be investigated laterly. Signed-off-by: Wei Zhang <weizhang555.zw@gmail.com>
Add basic introduction for overlay fs implementations, also architecture design are illustrated. Signed-off-by: Wei Zhang <weizhang555.zw@gmail.com>
Add unionmount-testsuite into CI for Overlay implementation Signed-off-by: Wei Zhang <weizhang555.zw@gmail.com>
WeiZhang555
force-pushed
the
overlay-impl
branch
from
a9c41b1
to
cec82c2
Compare
eryugey
approved these changes
Mar 11, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
These commits add first implementation of userspace OverlayFs, which aims to add writable upper layer for Nydus, and makes Nydus capable to get more control over user's file writing behavior.
The implementation refers a lot to kernel Overlay Fs, but there're still some differences due to limitation of Fuse
protocol. And the first implementation is not perfect but could be a start.
It was tested under
xfstesttestsuite, about 300+ test cases were tested and only 25 failed as designed or being limited by Fuse. I'll add external CI process later to makexfstestworkable later.Let me know your review opinions.