Skip to content

Commit

Permalink
Update README
Browse files Browse the repository at this point in the history
This commit addresses #3827 (comment)
  • Loading branch information
ysaito1001 committed Sep 16, 2024
1 parent 4e95038 commit a10e14a
Showing 1 changed file with 28 additions and 10 deletions.
38 changes: 28 additions & 10 deletions tools/ci-build/sdk-lockfiles/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,23 +2,41 @@ sdk-lockfiles
=============

This CLI tool audits the `Cargo.lock` files in the `smithy-rs` repository. These lockfiles are used to ensure
reproducible builds. The `sdk-lockfiles` tool specifically audits the following lockfiles:
- The [lockfile](https://github.com/smithy-lang/smithy-rs/blob/main/rust-runtime/Cargo.lock) for Smithy runtime crates
- The [lockfile](https://github.com/smithy-lang/smithy-rs/blob/main/aws/rust-runtime/Cargo.lock) for AWS runtime crates
- The [lockfile](https://github.com/smithy-lang/smithy-rs/blob/main/aws/rust-runtime/aws-config/Cargo.lock) for the `aws-config` crate
- The [lockfile](https://github.com/smithy-lang/smithy-rs/blob/main/aws/sdk/Cargo.lock) for the workspace containing code-generated AWS SDK crates (*)
reproducible builds during our release process for both `smithy-rs` and `aws-sdk-rust`. When a crate dependency is not
pinned to a fixed version, it risks being affected by newer versions of that dependency published to crates.io, which
could potentially break the build.

Specifically, the tool ensures that the lockfile marked with (*) is a superset containing all dependencies listed in
the rest of the runtime lockfiles. If it detects a new dependency in the AWS SDK crates introduced by any of the runtime
lockfiles (unless the dependency is introduced by a server runtime crate), it will output a message similar to the
following:
We track the following lockfiles in the `smithy-rs` repository:
1. The [lockfile](https://github.com/smithy-lang/smithy-rs/blob/main/rust-runtime/Cargo.lock) for Smithy runtime crates
2. The [lockfile](https://github.com/smithy-lang/smithy-rs/blob/main/aws/rust-runtime/Cargo.lock) for AWS runtime crates
3. The [lockfile](https://github.com/smithy-lang/smithy-rs/blob/main/aws/rust-runtime/aws-config/Cargo.lock) for the `aws-config` crate
4. The [lockfile](https://github.com/smithy-lang/smithy-rs/blob/main/aws/sdk/Cargo.lock) for the workspace containing code-generated AWS SDK crates

The first three lockfiles can be easily updated during development with a `cargo` command. However, the fourth lockfile
, known as the SDK lockfile, is generated by the code generator and is not checked into to the `smithy-rs` repository as
frequently as the first three runtime lockfiles. As a result, new dependencies added to any of the runtime lockfiles may
not be reflected in the SDK lockfile.

The `sdk-lockfiles` tool ensures that the SDK lockfile is a superset containing all dependencies listed in the three
runtime lockfiles. If it detects a new dependency in the AWS SDK crates introduced by any of the runtime lockfiles it
will output a message similar to the following (unless the dependency is introduced by a server specific runtime crate):
```
$ sdk-lockfiles audit
2024-09-10T16:48:38.460518Z INFO sdk_lockfiles::audit: checking whether `rust-runtime/Cargo.lock` is covered by the SDK lockfile...
2024-09-10T16:48:38.489879Z INFO sdk_lockfiles::audit: checking whether `aws/rust-runtime/Cargo.lock` is covered by the SDK lockfile...
2024-09-10T16:48:38.490306Z INFO sdk_lockfiles::audit: checking whether `aws/rust-runtime/aws-config/Cargo.lock` is covered by the SDK lockfile...
`minicbor` (0.24.2), used by `rust-runtime/Cargo.lock`, is not contained in SDK lockfile!
`minicbor` (0.24.2), used by `rust-runtime/Cargo.lock`, is not contained in the SDK lockfile!
Error: there are lockfile audit failures
```

This tool is intended for automated use.

## Limitation
The `sdk-lockfiles` tool does not verify whether new dependencies introduced in [CargoDependency.kt](https://github.com/smithy-lang/smithy-rs/blob/main/codegen-core/src/main/kotlin/software/amazon/smithy/rust/codegen/core/rustlang/CargoDependency.kt)
are included in the SDK lockfile. This is because dependencies in `CargoDependency.kt` are represented as a Kotlin data
class. Consequently, dependencies added via the code generator, `inlineable`, or `aws-inlineable` are not considered by
`sdk-lockfiles`.

This limitation is acceptable for our operational purposes. Our release script always executes
`./gradlew aws:sdk:syncAwsSdkLockfile`, which ensures that any dependencies added in `CargoDependency.kt` are properly
reflected in the SDK lockfile.

0 comments on commit a10e14a

Please sign in to comment.