Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

move rust crates to chromium_crate_io #21759

Merged
merged 2 commits into from
Mar 12, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@ patches/**/*.patchinfo
/third_party/playlist_component/src
/third_party/rapidjson/src
/third_party/reclient_configs/src
/third_party/rust/challenge_bypass_ristretto/v1/crate
/third_party/rust/futures_retry/v0_5/crate
/third_party/cryptography
/third_party/macholib
Expand Down
1 change: 0 additions & 1 deletion DEPS
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,6 @@ deps = {
'dep_type': 'cipd',
},
"third_party/playlist_component/src": "https://github.com/brave/playlist-component.git@673d40f017a1559bb685a15cf608ad1d4a94f8fb",
"third_party/rust/challenge_bypass_ristretto/v1/crate": "https://github.com/brave-intl/challenge-bypass-ristretto.git@48910ab1e0c6a52cd757abdc234aa4d6db214c09",
"third_party/rust/futures_retry/v0_5/crate": "https://github.com/brave-intl/futures-retry.git@2aaaafbc3d394661534d4dbd14159d164243c20e",
"third_party/macholib": {
"url": "https://github.com/ronaldoussoren/macholib.git@36a6777ccd0891c5d1b44ba885573d7c90740015",
Expand Down
5 changes: 3 additions & 2 deletions components/brave_news/rust/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,13 @@
name = "brave-news-cxx"
version = "1.0.0"
edition = "2018"
license = "MPL-2.0"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
cxx = { version = "1.0", features = [ "alloc", "std" ] }
feed-rs = { path = "../../../third_party/rust/feed_rs/v1/crate" }
cxx = "1"
feed-rs = "1.3.0"
log = "0.4.14"
lazy_static = "1.4.0"
regex = "1.5.5"
Expand Down
3 changes: 2 additions & 1 deletion components/brave_shields/adblock/rs/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
[package]
name = "adblock-cxx"
version = "0.1.0"
version = "1.0.0"
edition = "2021"
license = "MPL-2.0"

[dependencies]
adblock = { version = "0.8.8", default-features = false, features = ["full-regex-handling", "regex-debug-info", "css-validation"] }
Expand Down
3 changes: 2 additions & 1 deletion components/brave_wallet/rust/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
[package]
name = "brave-wallet-cxx"
version = "0.1.0"
version = "1.0.0"
edition = "2021"
license = "MPL-2.0"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

Expand Down
5 changes: 3 additions & 2 deletions components/challenge_bypass_ristretto/rust/cxx/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
[package]
name = "challenge-bypass-ristretto-cxx"
version = "0.1.0"
version = "1.0.0"
authors = ["eV <ev@7pr.xyz>"]
edition = "2018"
license = "MPL-2.0"

[dependencies]
challenge-bypass-ristretto = { version = "1.0.2", features = ["base64"] }
cxx = { version = "1.0", features = [ "alloc", "std" ] }
cxx = { version = "1.0" }
derive_more = "0.99"
hmac = "0.10"
lazy_static = "1.4.0"
Expand Down
1 change: 0 additions & 1 deletion components/filecoin/rs/BUILD.gn
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@ rust_static_library("rust_lib") {
"//brave/third_party/rust/fvm_ipld_encoding/v0_3:lib",
"//brave/third_party/rust/fvm_shared/v3:lib",
"//brave/third_party/rust/libsecp256k1/v0_7:lib",
"//brave/third_party/rust/libsecp256k1/v0_7:lib",
"//brave/third_party/rust/multihash/v0_16:lib",
"//brave/third_party/rust/thiserror/v1:lib",
"//third_party/rust/base64/v0_13:lib",
Expand Down
6 changes: 3 additions & 3 deletions components/filecoin/rs/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
[package]
name = "filecoin-cxx"
description = "FFI Interface to Filecoin"
version = "0.0.1"
version = "1.0.0"
edition = "2018"
publish = false
license = "MPL-2.0"

[dependencies]
base64 = "0.13.0"
bls-signatures = { version = "0.12", default-features = false, features = ["pairing"] }
cxx = { version = "1.0", features = [ "alloc", "std" ] }
cxx = "1"
fvm_shared = { version = "3.3.0" }
libsecp256k1 = "0.7"
num_bigint_chainsafe = { package = "forest_bigint", version = "0.1.2"}
Expand Down
4 changes: 2 additions & 2 deletions components/json/rs/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
[package]
name = "json-cxx"
description = "FFI Interface to Rust Json"
version = "0.0.1"
version = "1.0.0"
edition = "2018"
publish = false
license = "MPL-2.0"

[dependencies]
cxx = { version = "1.0", features = [ "alloc", "std" ] }
Expand Down
3 changes: 2 additions & 1 deletion components/p3a/constellation/rs/cxx/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,11 @@
name = "constellation-cxx"
version = "0.1.0"
edition = "2021"
license = "MPL-2.0"

[dependencies]
star-constellation = "0.2.3"
cxx = "1.0"
cxx = "1"

[lib]
name = "constellation_cxx"
Expand Down
5 changes: 3 additions & 2 deletions components/skus/browser/rs/cxx/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,16 @@
name = "skus-cxx"
version = "0.1.0"
edition = "2018"
license = "MPL-2.0"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
cxx = { version = "1.0", features = [ "alloc", "std" ] }
cxx = "1"
async-trait = "0.1.64"
futures = "0.3.19"
libc = { version = "0.2" }
skus = { path = "../lib" }
skus = "1"
serde_json = "1.0"
tracing-subscriber = { version = "0.2.0", default-features = false, features = ["ansi", "ansi_term", "fmt", "registry", "sharded-slab", "thread_local"] }
tracing = "0.1"
Expand Down
5 changes: 3 additions & 2 deletions components/skus/browser/rs/lib/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
[package]
name = "skus"
version = "0.1.0"
version = "1.0.0"
authors = ["eV <ev@7pr.xyz>"]
edition = "2018"
license = "MPL-2.0"

autotests = false

Expand All @@ -27,7 +28,7 @@ challenge-bypass-ristretto = { version = "1.0.2", features = ["serde_base64"] }
futures-retry = "0.5.0"
urlencoding = "1.1.1"
hmac = "0.10"
base64 = "0.9.3"
base64 = "0.13.0"
git-version = { version = "0.3.5", optional = true }

[dev-dependencies]
Expand Down
3 changes: 2 additions & 1 deletion components/speedreader/rust/ffi/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,10 @@ name = "speedreader-ffi"
version = "0.1.0"
authors = ["Andrius Aucinas <aaucinas@brave.com>"]
edition = "2018"
license = "MPL-2.0"

[dependencies]
speedreader = { path = "../lib/" }
speedreader = "1"
libc = "0.2.107"

[profile.release]
Expand Down
3 changes: 2 additions & 1 deletion components/speedreader/rust/lib/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
[package]
name = "speedreader"
version = "0.1.0"
version = "1.0.0"
authors = [
"gpestana <gpestana@brave.com>",
"Andrius Aucinas <aaucinas@brave.com>"
]
edition = "2018"
license = "MPL-2.0"

include = [
"/Cargo.toml",
Expand Down
12 changes: 12 additions & 0 deletions patches/tools-crates-gnrt-gen.rs.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
diff --git a/tools/crates/gnrt/gen.rs b/tools/crates/gnrt/gen.rs
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at least some of these patches can go to upstream, we'll figure out the details in a follow-up

index c056a69e03a96a762718f4dd5cd8e23accf5e455..8229e46a96c8581e35e9b5e44c699f73031c94e9 100644
--- a/tools/crates/gnrt/gen.rs
+++ b/tools/crates/gnrt/gen.rs
@@ -266,6 +266,7 @@ fn generate_for_third_party(args: GenCommandArgs, paths: &paths::ChromiumPaths)
// Remove any excluded dep entries.
dependencies
.retain(|dep| !config.resolve.remove_crates.iter().any(|r| **r == dep.package_name));
+ dependencies.retain(|dep| !dep.is_local );

// Remove dev dependencies since tests aren't run.
dependencies.retain(|dep| {
23 changes: 23 additions & 0 deletions patches/tools-crates-gnrt-lib-gn.rs.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
diff --git a/tools/crates/gnrt/lib/gn.rs b/tools/crates/gnrt/lib/gn.rs
index 8961d575419f079369685ba7721788676ac67c91..1f3a8acfae8ab844639dfb33246c540202e399df 100644
--- a/tools/crates/gnrt/lib/gn.rs
+++ b/tools/crates/gnrt/lib/gn.rs
@@ -199,8 +199,9 @@ pub fn build_rule_from_std_dep(
let mut aliases = Vec::new();
for dep in &normal_deps {
let target_name = NormalizedName::from_crate_name(&dep.package_name).to_string();
+ let epoch = Epoch::from_version(&dep.version).to_string();
if target_name != dep.use_name {
- aliases.push((dep.use_name.clone(), format!(":{target_name}")));
+ aliases.push((dep.use_name.clone(), format!("{target_name}/{epoch}")));
}
}
aliases.sort_unstable();
@@ -536,6 +537,7 @@ static TRIPLE_TO_GN_CONDITION: &[(&str, &str)] = &[
static TARGET_OS_TO_GN_CONDITION: &[(&str, &str)] = &[
("android", "is_android"),
("darwin", "is_mac"),
+ ("macos", "is_mac"),
("fuchsia", "is_fuchsia"),
("ios", "is_ios"),
("linux", "is_linux || is_chromeos"),
24 changes: 24 additions & 0 deletions patches/tools-crates-gnrt-lib-paths.rs.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
diff --git a/tools/crates/gnrt/lib/paths.rs b/tools/crates/gnrt/lib/paths.rs
index 3c1ea93728fc624701d6c7755dbeb9fcc6765ef3..80d09707ad6667892d38fd39f3959892cefe95ab 100644
--- a/tools/crates/gnrt/lib/paths.rs
+++ b/tools/crates/gnrt/lib/paths.rs
@@ -110,7 +110,7 @@ pub fn normalize_unix_path_separator(path: &Path) -> String {
.join("/")
}

-static RUST_THIRD_PARTY_DIR: &str = "third_party/rust";
+static RUST_THIRD_PARTY_DIR: &str = "brave/third_party/rust";
static RUST_SRC_LIBRARY_SUBDIR: &str = "library";
static RUST_SRC_VENDOR_SUBDIR: &str = "vendor";
static RUST_SRC_INSTALLED_DIR: &str = "third_party/rust-toolchain/lib/rustlib/src/rust";
@@ -121,8 +121,8 @@ static STD_FAKE_ROOT: &str = "build/rust/std/fake_root";
static STD_FAKE_ROOT_CONFIG_TEMPLATE: &str = "build/rust/std/fake_root/.cargo/config.toml.template";
static STD_FAKE_ROOT_CARGO_TEMPLATE: &str = "build/rust/std/fake_root/Cargo.toml.template";

-static THIRD_PARTY_CARGO_ROOT: &str = "third_party/rust/chromium_crates_io";
-static THIRD_PARTY_CONFIG_FILE: &str = "third_party/rust/chromium_crates_io/gnrt_config.toml";
+static THIRD_PARTY_CARGO_ROOT: &str = "brave/third_party/rust/chromium_crates_io";
+static THIRD_PARTY_CONFIG_FILE: &str = "brave/third_party/rust/chromium_crates_io/gnrt_config.toml";

static VET_CONFIG_FILE: &str = "third_party/rust/chromium_crates_io/supply-chain/config.toml";

13 changes: 13 additions & 0 deletions patches/tools-crates-gnrt-lib-platforms.rs.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
diff --git a/tools/crates/gnrt/lib/platforms.rs b/tools/crates/gnrt/lib/platforms.rs
index 2e710ad94da6053174d1fbb464c808d8d0862e40..dfec666b5d60d0290bb14b303c22663520c96d3f 100644
--- a/tools/crates/gnrt/lib/platforms.rs
+++ b/tools/crates/gnrt/lib/platforms.rs
@@ -234,7 +234,7 @@ fn supported_os_cfgs() -> &'static [Cfg] {
CFG_SET.get_or_init(|| {
[
// Set of supported OSes for `cfg(target_os = ...)`.
- "android", "darwin", "fuchsia", "ios", "linux", "windows",
+ "android", "macos", "darwin", "fuchsia", "ios", "linux", "windows",
]
.into_iter()
.map(|os| Cfg::KeyPair("target_os".to_string(), os.to_string()))
24 changes: 24 additions & 0 deletions patches/tools-crates-gnrt-lib-readme.rs.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
diff --git a/tools/crates/gnrt/lib/readme.rs b/tools/crates/gnrt/lib/readme.rs
index 8532dd95edf5ed2f83731f257589531049eb9255..d4b7890a663c4b8e7df7a0fc14f4970103004f9f 100644
--- a/tools/crates/gnrt/lib/readme.rs
+++ b/tools/crates/gnrt/lib/readme.rs
@@ -193,7 +193,7 @@ pub fn readme_file_from_package<'a>(

// Allowed licenses, in the format they are specified in Cargo.toml files from
// crates.io, and the format to write to README.chromium.
-static ALLOWED_LICENSES: [(&str, &str); 20] = [
+static ALLOWED_LICENSES: [(&str, &str); 24] = [
// ("Cargo.toml string", "License for README.chromium")
("Apache-2.0", "Apache 2.0"),
("MIT OR Apache-2.0", "Apache 2.0"),
@@ -218,6 +218,10 @@ static ALLOWED_LICENSES: [(&str, &str); 20] = [
"Apache 2.0 AND Unicode License Agreement - Data Files and Software (2016)",
),
("Apache-2.0 WITH LLVM-exception OR Apache-2.0 OR MIT", "Apache 2.0"),
+ ("CC0-1.0 OR MIT-0 OR Apache-2.0", "Apache 2.0"),
+ ("MIT OR Apache-2.0 OR BSD-1-Clause", "Apache 2.0"),
+ ("BSD-2-Clause", "BSD 2-Clause"),
+ ("MPL-2.0", "Mozilla Public License 2.0"),
];

static EXPECTED_LICENSE_FILE: [(&str, &str); 15] = [
22 changes: 22 additions & 0 deletions patches/tools-crates-gnrt-vendor.rs.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
diff --git a/tools/crates/gnrt/vendor.rs b/tools/crates/gnrt/vendor.rs
index be4dbf2dba2a67804e684d743d815fc7fdace87e..3b10471f6535679fe4a56bea80fa653d929c5291 100644
--- a/tools/crates/gnrt/vendor.rs
+++ b/tools/crates/gnrt/vendor.rs
@@ -96,7 +96,7 @@ fn vendor_impl(args: VendorCommandArgs, paths: &paths::ChromiumPaths) -> Result<
if config.resolve.remove_crates.contains(&p.name) {
println!("Generating placeholder for removed crate {}-{}", p.name, p.version);
placeholder_crate(p, &nodes, paths, &config)?;
- } else if !dirs.contains(&crate_dir) {
+ } else if !dirs.contains(&crate_dir) && p.source.is_some() {
println!("Downloading {}-{}", p.name, p.version);
download_crate(&p.name, &p.version, paths)?;
let skip_patches = match &args.no_patches {
@@ -266,7 +266,7 @@ fn apply_patches(
let crate_dir = vendor_dir.join(format!("{name}-{version}"));

let mut patches = Vec::new();
- let Ok(patch_dir) = std::fs::read_dir(paths.third_party_cargo_root.join("patches").join(name))
+ let Ok(patch_dir) = std::fs::read_dir(paths.third_party_cargo_root.join("patches").join(format!("{name}-{version}")))
else {
// No patches for this crate.
return Ok(());
2 changes: 1 addition & 1 deletion script/audit_deps.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ def get_remote_audit_config(
NPM_EXCLUDE_PATHS = ['build', os.path.join('node_modules')]

# Only check Cargo.lock for this path.
CARGO_INCLUDE_PATH = os.path.join('third_party', 'rust')
CARGO_INCLUDE_PATH = os.path.join('third_party', 'rust', 'chromium_crates_io')


def main():
Expand Down
5 changes: 5 additions & 0 deletions script/brave_license_helper.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,11 @@ def AddBraveCredits(root, prune_paths, special_cases, prune_dirs,
os.path.join('brave', 'vendor', 'brave-ios'),
os.path.join('brave', 'vendor', 'brave_base'),

# These have auto-generated license files and
# GetThirdPartyDepsFromGNDepsOutput causes strange license errors
# unless the this entire directory is excluded.
os.path.join('brave', 'third_party', 'rust'),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can just exclude //brave/third_party/rust/chromium_crates_io/vendor.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please see the comment explaining why the entire directory is excluded

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I read the comment, but I couldn't reproduce the errors. I suppose it doesn't matter much, but the narrower scope seemed a safer hack.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can try it again, but it failed with a very strange error before


# No third-party code directly under android_deps. It's all under
# android_deps/libs instead and it's special-cased further down.
os.path.join('brave', 'third_party', 'android_deps'),
Expand Down
Loading
Loading