Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PVF: more filesystem sandboxing #1373

Merged
merged 22 commits into from
Sep 28, 2023
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
99a9efd
rsync `mrcnski/pvf-sandbox-whole-process` branch from Polkadot
mrcnski Aug 28, 2023
c8f2962
Do the rest
mrcnski Sep 3, 2023
6f7d3fe
Merge branch 'master' into mrcnski/pvf-worker-sandbox-with-pivot-root
mrcnski Sep 3, 2023
9d2ce42
Some fixes
mrcnski Sep 3, 2023
15897c3
Merge branch 'master' into mrcnski/pvf-worker-sandbox-with-pivot-root
mrcnski Sep 4, 2023
f926505
cargo fmt
mrcnski Sep 4, 2023
32cfbcb
Fix clippy error
mrcnski Sep 5, 2023
eacb956
Address review comments; refactor/simplify Landlock code
mrcnski Sep 10, 2023
45b99a9
Merge branch 'master' into mrcnski/pvf-worker-sandbox-with-pivot-root
mrcnski Sep 10, 2023
dc6fe04
cargo fmt
mrcnski Sep 10, 2023
ed344ab
Address most review comments (will do the last one after lunch)
mrcnski Sep 11, 2023
8cedd7b
Use cache as location for worker dirs
mrcnski Sep 11, 2023
396a7b6
Merge branch 'master' into mrcnski/pvf-worker-sandbox-with-pivot-root
mrcnski Sep 12, 2023
ccc329e
Clear env vars when spawning process
mrcnski Sep 12, 2023
2e6bb65
Fix compiler error, add comment
mrcnski Sep 14, 2023
a5efc37
Rearrange worker startup
mrcnski Sep 14, 2023
b413c27
Fix runtime crash (tokio socket not created in context of a runtime)
mrcnski Sep 14, 2023
70e62d8
Fix compiler error
mrcnski Sep 14, 2023
ea1b48d
Fix pivot_root and add assertions
mrcnski Sep 17, 2023
a696d40
Add a couple of tests to de-befuddle myself
mrcnski Sep 17, 2023
ad0ed8a
Move socket to beginning of worker startup
mrcnski Sep 17, 2023
279db25
Use cfg_if macro
mrcnski Sep 25, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 38 additions & 3 deletions polkadot/node/core/pvf/common/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,17 @@ pub enum PrepareError {
/// The response from the worker is received, but the file cannot be renamed (moved) to the
/// final destination location. This state is reported by the validation host (not by the
/// worker).
RenameTmpFileErr(String),
RenameTmpFileErr {
err: String,
// Unfortunately `PathBuf` doesn't implement `Encode`/`Decode`, so we do a fallible
// conversion to `Option<String>`.
src: Option<String>,
dest: Option<String>,
},
/// The response from the worker is received, but the worker cache could not be cleared. The
/// worker has to be killed to avoid jobs having access to data from other jobs. This state is
/// reported by the validation host (not by the worker).
ClearWorkerDir(String),
}

impl PrepareError {
Expand All @@ -58,7 +68,11 @@ impl PrepareError {
use PrepareError::*;
match self {
Prevalidation(_) | Preparation(_) | Panic(_) => true,
TimedOut | IoErr(_) | CreateTmpFileErr(_) | RenameTmpFileErr(_) => false,
TimedOut |
IoErr(_) |
CreateTmpFileErr(_) |
RenameTmpFileErr { .. } |
ClearWorkerDir(_) => false,
// Can occur due to issues with the PVF, but also due to local errors.
RuntimeConstruction(_) => false,
}
Expand All @@ -76,7 +90,9 @@ impl fmt::Display for PrepareError {
TimedOut => write!(f, "prepare: timeout"),
IoErr(err) => write!(f, "prepare: io error while receiving response: {}", err),
CreateTmpFileErr(err) => write!(f, "prepare: error creating tmp file: {}", err),
RenameTmpFileErr(err) => write!(f, "prepare: error renaming tmp file: {}", err),
RenameTmpFileErr { err, src, dest } =>
write!(f, "prepare: error renaming tmp file ({:?} -> {:?}): {}", src, dest, err),
ClearWorkerDir(err) => write!(f, "prepare: error clearing worker cache: {}", err),
}
}
}
Expand All @@ -89,8 +105,17 @@ impl fmt::Display for PrepareError {
pub enum InternalValidationError {
/// Some communication error occurred with the host.
HostCommunication(String),
/// Host could not create a hard link to the artifact path.
CouldNotCreateLink(String),
/// Could not find or open compiled artifact file.
CouldNotOpenFile(String),
/// Host could not clear the worker cache after a job.
CouldNotClearWorkerDir {
err: String,
// Unfortunately `PathBuf` doesn't implement `Encode`/`Decode`, so we do a fallible
// conversion to `Option<String>`.
path: Option<String>,
},
/// An error occurred in the CPU time monitor thread. Should be totally unrelated to
/// validation.
CpuTimeMonitorThread(String),
Expand All @@ -104,8 +129,18 @@ impl fmt::Display for InternalValidationError {
match self {
HostCommunication(err) =>
write!(f, "validation: some communication error occurred with the host: {}", err),
CouldNotCreateLink(err) => write!(
f,
"validation: host could not create a hard link to the artifact path: {}",
err
),
CouldNotOpenFile(err) =>
write!(f, "validation: could not find or open compiled artifact file: {}", err),
CouldNotClearWorkerDir { err, path } => write!(
f,
"validation: host could not clear the worker cache ({:?}) after a job: {}",
path, err
),
CpuTimeMonitorThread(err) =>
write!(f, "validation: an error occurred in the CPU time monitor thread: {}", err),
NonDeterministicPrepareError(err) => write!(f, "validation: prepare: {}", err),
Expand Down
2 changes: 1 addition & 1 deletion polkadot/node/core/pvf/common/src/execute.rs
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ use polkadot_primitives::ExecutorParams;
use std::time::Duration;

/// The payload of the one-time handshake that is done when a worker process is created. Carries
/// data from the host to the worker.
/// data from the host to the worker that would be too large for CLI parameters..
alindima marked this conversation as resolved.
Show resolved Hide resolved
#[derive(Encode, Decode)]
pub struct Handshake {
/// The executor parameters.
Expand Down
10 changes: 10 additions & 0 deletions polkadot/node/core/pvf/common/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ pub mod executor_intf;
pub mod prepare;
pub mod pvf;
pub mod worker;
pub mod worker_dir;

pub use cpu_time::ProcessTime;

Expand All @@ -41,6 +42,15 @@ pub mod tests {
pub const TEST_PREPARATION_TIMEOUT: Duration = Duration::from_secs(30);
}

/// Status of security features on the current system.
#[derive(Debug, Clone, Default)]
pub struct SecurityStatus {
/// Whether the landlock features we use are fully available on this system.
pub can_enable_landlock: bool,
// Whether we are able to unshare the user namespace and change the filesystem root.
pub can_unshare_user_namespace_and_change_root: bool,
}

/// Write some data prefixed by its length into `w`.
pub async fn framed_send(w: &mut (impl AsyncWrite + Unpin), buf: &[u8]) -> io::Result<()> {
let len_buf = buf.len().to_le_bytes();
Expand Down
Loading