Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(source-scan): docker checks #166

Merged
merged 31 commits into from
Jun 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
4f6e302
chore: set `CARGO_NEAR_BUILD_COMMAND` only if unset previously
May 9, 2024
9de9279
feat: add print of bin hashsum to `ArtifactMessages`
May 9, 2024
c62e17b
chore: export env var, indicating that abi is being generated
May 13, 2024
62d7bcf
chore: export build fn and its arg type
May 13, 2024
41a4cc0
ci: fix clippy
May 15, 2024
69f0c0b
chore: add wrapping msg for missing commit on remote
May 15, 2024
5ef7893
chore: print hex format of code checksum as well
May 16, 2024
210e8e3
feat: compute `CARGO_NEAR_BUILD_COMMAND` either from cli args or from…
May 16, 2024
815b727
chore: make `BuildArtifact` the return type of `cargo_near::build_ext…
May 21, 2024
7d2e11d
chore: generalize `BuildOptsExtended`
May 21, 2024
5e48ff3
doc: improve doc a bit on added opts
May 22, 2024
fbed198
chore: replace `contains` with `ends_with`
May 22, 2024
b3f7044
chore: remove docker interactive run with `CARGO_NEAR_SERVER_BUILD_DI…
May 27, 2024
fc966ca
feat: docker sanity check + image check
May 27, 2024
ac64571
chore: extra guidance for `permission denied` case
May 29, 2024
8836317
review: addresses https://github.com/near/cargo-near/pull/166#discuss…
Jun 10, 2024
6ba7710
review: addresses https://github.com/near/cargo-near/pull/166#discuss…
Jun 10, 2024
c45d75e
review: addresses https://github.com/near/cargo-near/pull/166#discuss…
Jun 10, 2024
b85e46c
review: addresses https://github.com/near/cargo-near/pull/166#discuss…
Jun 10, 2024
dde5ec8
review: addresses https://github.com/near/cargo-near/pull/166#discuss…
Jun 10, 2024
2139f80
review: addresses https://github.com/near/cargo-near/pull/166#discuss…
Jun 12, 2024
5dbb2eb
review: addresses https://github.com/near/cargo-near/pull/166#discuss…
Jun 13, 2024
ad113d8
review: address https://github.com/near/cargo-near/pull/166#discussio…
Jun 13, 2024
31d087a
ci: fix clippy
Jun 13, 2024
787cfbd
review: addresses https://github.com/near/cargo-near/pull/166#discuss…
Jun 14, 2024
484b785
chore: remove todo
Jun 14, 2024
c0c70ba
review: address https://github.com/near/cargo-near/pull/134#discussio…
Jun 14, 2024
af76989
review: addresses https://github.com/near/cargo-near/pull/166#discuss…
Jun 17, 2024
538acf0
doc: add missing parts
Jun 17, 2024
48b8181
review: addresses https://github.com/near/cargo-near/pull/134#discuss…
Jun 17, 2024
e410b0d
doc: fix a typo
Jun 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

39 changes: 36 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,14 +50,14 @@ npm install cargo-near
<summary>Compile and install from source code (Cargo)</summary>

```sh
cargo install cargo-near
cargo install --locked cargo-near
```

or, install the most recent version from git repository:

```sh
$ git clone https://github.com/near/cargo-near
$ cargo install --path cargo-near
$ cargo install --locked --path cargo-near
```
</details>

Expand All @@ -71,32 +71,65 @@ cargo near

Starts interactive mode that will allow to explore all the available commands.

---

```console
cargo near new
```

Initializes a new project skeleton to create a contract from a template.

---

```console
cargo near build
```

Builds a NEAR smart contract along with its [ABI](https://github.com/near/abi) (while in the directory containing contract's Cargo.toml).

You can also make this command embed ABI into your WASM artifact by adding `--embed-abi` parameter. Once deployed, this will allow you to call a view function `__contract_abi` to retrieve a [ZST](https://facebook.github.io/zstd/)-compressed ABI.
By default, this runs a reproducible build in a [Docker](https://docs.docker.com/) container, which:

- runs against source code version, committed to git, ignoring any uncommitted changes
- requires that `Cargo.lock` of project is created (e.g. via `cargo update`) and added to git
- will use configuration in `[package.metadata.near.reproducible_build]` section of contract's `Cargo.toml`

`--no-docker` flag can be used to perform a regular build with rust toolchain installed onto host, running the `cargo-near` cli. *NO*-Docker builds run against actual state of code in filesystem and not against a version, committed to source control.

`--no-locked` flag is allowed in *NO*-Docker builds, e.g. to generate a `Cargo.lock` *and* simultaneously build the contract.

---

```console
cargo near abi
```

Generates NEAR smart contract's [ABI](https://github.com/near/abi) (while in the directory containing contract's Cargo.toml).

Once contract is deployed, this will allow you to call a view function `__contract_abi` to retrieve a [ZST](https://facebook.github.io/zstd/)-compressed ABI.

---

```console
cargo near create-dev-account
```

Guides you through creation of a new NEAR account on [testnet](https://explorer.testnet.near.org).

---

```console
cargo near deploy
```

Builds the smart contract (equivalent to `cargo near build`) and guides you to deploy it to the blockchain.

By default, this runs a reproducible build in a Docker container.

`deploy` command from Docker build requires that contract's source code has been pushed to remote repository,
doesn't have any modified tracked files, any staged changes or any untracked content.

`--no-docker` flag can be used to perform a regular *NO*-Docker build *and* deploy. Similar to `build` command,
in this case none of the git-related concerns and restrictions apply.

## Contribution

Expand Down
4 changes: 3 additions & 1 deletion cargo-near/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ eula = false

[dependencies]
bs58 = "0.4"
hex = "0.4.3"
camino = "1.1.1"
cargo_metadata = "0.18.1"
clap = { version = "4.0.18", features = ["derive", "env"] }
Expand Down Expand Up @@ -54,8 +55,9 @@ home = "0.5.9"
pathdiff = { version = "0.2.1", features = ["camino"]}
unix_path = "1.0.1"
url = { version = "2.5.0", features = ["serde"]}
tmp_env = "0.1.1"

[target.'cfg(unix)'.dependencies]
[target.'cfg(target_os = "linux")'.dependencies]
nix = { version = "0.28.0", features = ["user", "process"] }

[features]
Expand Down
61 changes: 61 additions & 0 deletions cargo-near/src/build_extended.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
use crate::{BuildArtifact, BuildOpts};

mod build_script;
pub use build_script::BuildScriptOpts;
use rustc_version::Version;

#[derive(Debug, Clone)]
pub struct OptsExtended<'a> {
pub workdir: &'a str,
/// vector of key-value pairs of temporary env overrides during build process
pub env: Vec<(&'a str, &'a str)>,
pub build_opts: BuildOpts,
pub build_script_opts: BuildScriptOpts<'a>,
}

pub fn build(args: OptsExtended) -> Result<BuildArtifact, Box<dyn std::error::Error>> {
let actual_version = rustc_version::version()?;
let (artifact, skipped) = args.skip_or_compile(&actual_version)?;

args.build_script_opts
.post_build(skipped, &artifact, args.workdir, &actual_version)?;
Ok(artifact)
}

impl<'a> OptsExtended<'a> {
pub fn skip_or_compile(
&self,
version: &Version,
) -> Result<(BuildArtifact, bool), Box<dyn std::error::Error>> {
let _tmp_workdir = tmp_env::set_current_dir(self.workdir)?;
let result = if self.build_script_opts.should_skip(version) {
let artifact = self.build_script_opts.create_empty_stub()?;
(artifact, true)
} else {
let artifact = self.compile_near_artifact()?;
(artifact, false)
};
Ok(result)
}

/// `CARGO_TARGET_DIR` export is needed to avoid attempt to acquire same `target/<profile-path>/.cargo-lock`
/// as the `cargo` process, which is running the build-script
pub fn compile_near_artifact(&self) -> Result<BuildArtifact, Box<dyn std::error::Error>> {
let mut tmp_envs = vec![];
for (env_key, value) in self.env.iter() {
let tmp_env = tmp_env::set_var(env_key, value);
tmp_envs.push(tmp_env);
}

let artifact = if let Some(distinct_target_dir) = self.build_script_opts.distinct_target_dir
{
let _tmp_cargo_target_env = tmp_env::set_var("CARGO_TARGET_DIR", distinct_target_dir);

crate::build(self.build_opts.clone())?
} else {
crate::build(self.build_opts.clone())?
};

Ok(artifact)
}
}
169 changes: 169 additions & 0 deletions cargo-near/src/build_extended/build_script.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
use crate::BuildArtifact;
use rustc_version::Version;

/// `cargo::` prefix for build script outputs, that `cargo` recognizes
/// was implemented <https://github.com/rust-lang/cargo/pull/12201> since this version
const DEPRECATE_SINGLE_COLON_SINCE: Version = Version::new(1, 77, 0);

macro_rules! print_warn {
($version: ident, $($tokens: tt)*) => {
let separator = if $version >= &DEPRECATE_SINGLE_COLON_SINCE {
"::"
} else {
":"
};
println!("cargo{}warning={}", separator, format!($($tokens)*))
}
}

#[derive(Debug, Clone)]
pub struct BuildScriptOpts<'a> {
/// environment variable name to export result `*.wasm` path to with [`cargo::rustc-env=`](https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-env)
/// instruction
pub result_env_key: Option<&'a str>,
/// list of paths for [`cargo::rerun-if-changed=`](https://doc.rust-lang.org/cargo/reference/build-scripts.html#rerun-if-changed)
/// instruction
///
/// if relative, it's relative to path of crate, where build.rs is compiled
pub rerun_if_changed_list: Vec<&'a str>,
/// vector of key-value pairs of environment variable name and its value,
/// when compilation should be skipped on a variable's value match;
/// e.g.
/// skipping emitting output `*.wasm` may be helpful when `PROFILE` is equal to `debug`
/// for using `rust-analyzer/flycheck`, `cargo check`, `bacon` and other dev-tools
pub build_skipped_when_env_is: Vec<(&'a str, &'a str)>,
/// path of stub file, where a placeholder empty `wasm` output is emitted to, when
/// build is skipped due to match in [`Self::build_skipped_when_env_is`]
///
/// if this path is relative, then the base is [`crate::BuildOptsExtended::workdir`]
pub stub_path: Option<&'a str>,
/// substitution export of [`CARGO_TARGET_DIR`](https://doc.rust-lang.org/cargo/reference/environment-variables.html),
/// which is required to avoid deadlock <https://github.com/rust-lang/cargo/issues/8938>;
/// should best be a subfolder of [`CARGO_TARGET_DIR`](https://doc.rust-lang.org/cargo/reference/environment-variables.html)
/// of crate being built to work normally in docker builds
///
/// if this path is relative, then the base is [`crate::BuildOptsExtended::workdir`]
pub distinct_target_dir: Option<&'a str>,
}

impl<'a> BuildScriptOpts<'a> {
pub fn should_skip(&self, version: &Version) -> bool {
let mut return_bool = false;
for (env_key, value_to_skip) in self.build_skipped_when_env_is.iter() {
if let Ok(actual_value) = std::env::var(env_key) {
if actual_value == *value_to_skip {
return_bool = true;
print_warn!(
version,
"`{}` env set to `{}`, build was configured to skip on this value",
env_key,
actual_value
);
}
}
}

return_bool
}
pub fn create_empty_stub(&self) -> Result<BuildArtifact, Box<dyn std::error::Error>> {
if self.stub_path.is_none() {
return Err(
"build must be skipped, but `BuildScriptOpts.stub_path` wasn't configured"
.to_string(),
)?;
}
let stub_path = std::path::Path::new(self.stub_path.as_ref().unwrap());
create_stub_file(stub_path)?;
let stub_path = stub_path.canonicalize()?;

let artifact = {
let stub_path = camino::Utf8PathBuf::from_path_buf(stub_path)
.map_err(|err| format!("`{}` isn't a valid UTF-8 path", err.to_string_lossy()))?;
BuildArtifact {
path: stub_path,
fresh: true,
from_docker: false,
}
};
Ok(artifact)
}

pub fn post_build(
&self,
skipped: bool,
artifact: &BuildArtifact,
workdir: &str,
version: &Version,
) -> Result<(), Box<dyn std::error::Error>> {
let colon_separator = if version >= &DEPRECATE_SINGLE_COLON_SINCE {
"::"
} else {
":"
};
if let Some(ref result_env_key) = self.result_env_key {
pretty_print(skipped, artifact, version)?;
println!(
"cargo{}rustc-env={}={}",
colon_separator,
result_env_key,
artifact.path.clone().into_string()
);
print_warn!(
version,
"Path to result artifact of build in `{}` is exported to `{}`",
workdir,
result_env_key,
);
}
for path in self.rerun_if_changed_list.iter() {
println!("cargo{}rerun-if-changed={}", colon_separator, path);
}
Ok(())
}
}

fn create_stub_file(out_path: &std::path::Path) -> Result<(), Box<dyn std::error::Error>> {
std::fs::OpenOptions::new()
.write(true)
.truncate(true)
.create(true)
.open(out_path)?;
Ok(())
}

fn pretty_print(
skipped: bool,
artifact: &BuildArtifact,
version: &Version,
) -> Result<(), Box<dyn std::error::Error>> {
if skipped {
print_warn!(
version,
"Build empty artifact stub-file written to: `{}`",
artifact.path.clone().into_string()
);
return Ok(());
}
let hash = artifact.compute_hash()?;

print_warn!(version, "");
print_warn!(version, "");
print_warn!(
version,
"Build artifact path: {}",
artifact.path.clone().into_string()
);
print_warn!(
version,
"Sub-build artifact SHA-256 checksum hex: {}",
hash.to_hex_string()
);
print_warn!(
version,
"Sub-build artifact SHA-256 checksum bs58: {}",
hash.to_base58_string()
);
print_warn!(version, "");
print_warn!(version, "");
Ok(())
}
2 changes: 2 additions & 0 deletions cargo-near/src/commands/abi_command/abi.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ use color_eyre::eyre::ContextCompat;
use colored::Colorize;
use near_abi::AbiRoot;

use crate::commands::build_command::BUILD_RS_ABI_STEP_HINT_ENV_KEY;
use crate::common::ColorPreference;
use crate::types::{manifest::CargoManifestPath, metadata::CrateMetadata};
use crate::util;
Expand Down Expand Up @@ -93,6 +94,7 @@ pub(crate) fn generate_abi(
("CARGO_PROFILE_DEV_OPT_LEVEL", "0"),
("CARGO_PROFILE_DEV_DEBUG", "0"),
("CARGO_PROFILE_DEV_LTO", "off"),
(BUILD_RS_ABI_STEP_HINT_ENV_KEY, "true"),
frol marked this conversation as resolved.
Show resolved Hide resolved
],
util::dylib_extension(),
hide_warnings,
Expand Down
Loading
Loading