Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement statfs with synthetic values #1118

Merged
merged 7 commits into from
Nov 21, 2024

Conversation

c-hagem
Copy link
Contributor

@c-hagem c-hagem commented Nov 7, 2024

Description of change

This PR adds support for calling statfs on virtual file system created using mountpoint.
Some applications depend on the filesystem reporting non-zero available space; currently mountpoint reports 0 as number of available blocks, which can cause these applications to not work as expected.

This PR (building on #871) implements statfs with synthetic values (4611686018427387904 free blocks).
For example, the DF output now is:

mountpoint-s3  4611686018427387904        0 4611686018427387904   0% /local/home/chagem/mnt/bucket

Thus, checks for available space should no longer fail.

Relevant issues: #710.

Does this change impact existing behavior?

This change impacts existing behaviour, as Mountpoint will report non-zero value for total blocks, free blocks, free inodes and maximum file name length.

Does this change need a changelog entry?

Yes, addressed.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and I agree to the terms of the Developer Certificate of Origin (DCO).

@c-hagem c-hagem had a problem deploying to PR integration tests November 7, 2024 18:37 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests November 7, 2024 18:37 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests November 7, 2024 18:37 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests November 7, 2024 18:37 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests November 7, 2024 18:37 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests November 7, 2024 18:37 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests November 7, 2024 18:37 — with GitHub Actions Failure
@c-hagem c-hagem temporarily deployed to PR integration tests November 8, 2024 12:09 — with GitHub Actions Inactive
@c-hagem c-hagem temporarily deployed to PR integration tests November 8, 2024 12:09 — with GitHub Actions Inactive
@c-hagem c-hagem temporarily deployed to PR integration tests November 8, 2024 12:09 — with GitHub Actions Inactive
@c-hagem c-hagem had a problem deploying to PR integration tests November 8, 2024 12:09 — with GitHub Actions Failure
@c-hagem c-hagem temporarily deployed to PR integration tests November 8, 2024 12:09 — with GitHub Actions Inactive
@c-hagem c-hagem temporarily deployed to PR integration tests November 8, 2024 12:09 — with GitHub Actions Inactive
@c-hagem c-hagem temporarily deployed to PR integration tests November 8, 2024 12:09 — with GitHub Actions Inactive
@dannycjones dannycjones self-requested a review November 8, 2024 13:26
@c-hagem c-hagem had a problem deploying to PR integration tests November 12, 2024 15:04 — with GitHub Actions Failure
@c-hagem c-hagem temporarily deployed to PR integration tests November 13, 2024 13:17 — with GitHub Actions Inactive
@c-hagem c-hagem temporarily deployed to PR integration tests November 13, 2024 13:17 — with GitHub Actions Inactive
@c-hagem c-hagem temporarily deployed to PR integration tests November 13, 2024 13:18 — with GitHub Actions Inactive
@c-hagem c-hagem temporarily deployed to PR integration tests November 13, 2024 13:18 — with GitHub Actions Inactive
@c-hagem c-hagem temporarily deployed to PR integration tests November 13, 2024 13:18 — with GitHub Actions Inactive
@c-hagem c-hagem temporarily deployed to PR integration tests November 13, 2024 13:18 — with GitHub Actions Inactive
@c-hagem c-hagem temporarily deployed to PR integration tests November 13, 2024 13:18 — with GitHub Actions Inactive
Copy link
Contributor

@dannycjones dannycjones left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Awesome to see this working.

There are a few comments inline.

Can you add some tests for this, just to verify these values make it out of the file system? I'd recommend creating a new file in mountpoint-s3/tests/fuse_tests/ named statfs_test.rs, and write some tests that test end-to-end. statvfs isn't actually in the Rust standard library, but we have some example of using a crate named nix for querying this information.

CacheLimit::AvailableSpace { min_ratio } => {
let stats = match nix::sys::statvfs::statvfs(&self.cache_directory) {
Ok(stats) if stats.blocks() == 0 => {
warn!("unable to determine available space (0 blocks reported)");
return false;
}
Ok(stats) => stats,
Err(error) => {
warn!(?error, "unable to determine available space");
return false;
}
};
(stats.blocks_free() as f64) < min_ratio * (stats.blocks() as f64)
}

mountpoint-s3/src/fs.rs Outdated Show resolved Hide resolved
mountpoint-s3/src/fs.rs Outdated Show resolved Hide resolved
mountpoint-s3/CHANGELOG.md Outdated Show resolved Hide resolved
mountpoint-s3/src/fs.rs Outdated Show resolved Hide resolved
@c-hagem c-hagem had a problem deploying to PR integration tests November 18, 2024 10:12 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests November 18, 2024 10:12 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests November 18, 2024 10:12 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests November 18, 2024 10:12 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests November 18, 2024 10:12 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests November 21, 2024 14:42 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests November 21, 2024 14:42 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests November 21, 2024 14:42 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests November 21, 2024 14:42 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests November 21, 2024 14:42 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests November 21, 2024 14:42 — with GitHub Actions Failure
@c-hagem c-hagem had a problem deploying to PR integration tests November 21, 2024 14:42 — with GitHub Actions Failure
Copy link
Contributor

@dannycjones dannycjones left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@c-hagem The rebase appears to have included changes unrelated to this PR, can you fix?

mountpoint-s3/config.toml Outdated Show resolved Hide resolved
mountpoint-s3/Cargo.toml Outdated Show resolved Hide resolved
Signed-off-by: Christian Hagemeier <chagem@amazon.com>
Signed-off-by: Christian Hagemeier <chagem@amazon.com>
Signed-off-by: Christian Hagemeier <chagem@amazon.com>
Signed-off-by: Christian Hagemeier <chagem@amazon.com>
Signed-off-by: Christian Hagemeier <chagem@amazon.com>
Signed-off-by: Christian Hagemeier <chagem@amazon.com>
Signed-off-by: Christian Hagemeier <chagem@amazon.com>
@c-hagem c-hagem temporarily deployed to PR integration tests November 21, 2024 15:19 — with GitHub Actions Inactive
@c-hagem c-hagem temporarily deployed to PR integration tests November 21, 2024 15:19 — with GitHub Actions Inactive
@c-hagem c-hagem temporarily deployed to PR integration tests November 21, 2024 15:19 — with GitHub Actions Inactive
@c-hagem c-hagem temporarily deployed to PR integration tests November 21, 2024 15:19 — with GitHub Actions Inactive
@c-hagem c-hagem temporarily deployed to PR integration tests November 21, 2024 15:19 — with GitHub Actions Inactive
@c-hagem c-hagem temporarily deployed to PR integration tests November 21, 2024 15:19 — with GitHub Actions Inactive
@c-hagem c-hagem temporarily deployed to PR integration tests November 21, 2024 15:19 — with GitHub Actions Inactive
Copy link
Contributor

@dannycjones dannycjones left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, ship it! 🎉

@dannycjones dannycjones added this pull request to the merge queue Nov 21, 2024
Merged via the queue into awslabs:main with commit 630e5a0 Nov 21, 2024
23 checks passed
@dannycjones
Copy link
Contributor

Released in Mountpoint v1.12.0! 🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants