Enhance gradle tasks for managing lockfiles

[ysaito1001]

Description

This PR introduces and updates gradle tasks for managing lockfiles. Here are the highlights:

• The SDK lockfile can now be generated directly within the smithy-rs repository without the aws-sdk-rust repository.
• The SDK lockfile can be synchronized with runtime lockfiles, updating only new dependencies while preserving the versions of existing ones.
• To prevent updating broken dependencies to the latest versions, we track the last known good versions and downgrade them to those versions.

New/updated gradle tasks are intended for automation:

• This existing task no longer requires -Paws-sdk-rust-path. We plan to incorporate it into a weekly GitHub Action to automate lockfile updates:

./gradlew aws:sdk:cargoUpdateAllLockfiles

• This new task synchronizes the SDK lockfile with runtime lockfiles. We plan to integrate it into pre-commit hooks:

./gradlew aws:sdk:syncAwsSdkLockfile  

In addition, this PR has updated the SDK lockfile by executing ./gradlew aws:sdk:syncAwsSdkLockfile. The updated lockfile no longer includes many SDK crates that are unused in CI/CD processes. The new SDK lockfile is in sync with the runtime lockfiles:

➜  smithy-rs git:(ysaito/enhance-gradle-tasks-for-lockfile) sdk-lockfiles audit
2024-09-12T16:02:25.193765Z  INFO sdk_lockfiles::audit: checking whether `rust-runtime/Cargo.lock` is covered by the SDK lockfile...
2024-09-12T16:02:25.224862Z  INFO sdk_lockfiles::audit: checking whether `aws/rust-runtime/Cargo.lock` is covered by the SDK lockfile...
2024-09-12T16:02:25.225389Z  INFO sdk_lockfiles::audit: checking whether `aws/rust-runtime/aws-config/Cargo.lock` is covered by the SDK lockfile...
SUCCESS

Testing

I have verified the change against basic use cases:

When running cargoUpdateAllLockfiles, dependencies will be updated to their latest versions, while broken crates will be pinned to the last known good versions.

Expand for more details...

When we execute

smithy-rs git:(ysaito/enhance-gradle-tasks-for-lockfile) ✗ ./gradlew aws:sdk:cargoUpdateAllLockfiles
...
BUILD SUCCESSFUL in 1m 7s

all lockfiles include the latest versions of dependencies, except for those that are pinned due to being broken. Currently, minicbor is pinned to 0.24.2:

➜  smithy-rs git:(ysaito/enhance-gradle-tasks-for-lockfile) ✗ git status
On branch ysaito/enhance-gradle-tasks-for-lockfile
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   aws/rust-runtime/Cargo.lock
	modified:   aws/rust-runtime/aws-config/Cargo.lock
	modified:   aws/sdk/Cargo.lock
	modified:   rust-runtime/Cargo.lock

no changes added to commit (use "git add" and/or "git commit -a")

➜  smithy-rs git:(ysaito/enhance-gradle-tasks-for-lockfile) ✗ cat aws/sdk/Cargo.lock | rg -C1 minicbor
...
---
[[package]]
name = "minicbor"
version = "0.24.2"
---
...

➜  smithy-rs git:(ysaito/enhance-gradle-tasks-for-lockfile) ✗ cat rust-runtime/Cargo.lock | rg -C1 minicbor
...
---
[[package]]
name = "minicbor"
version = "0.24.2"
---
...

Finally, the sdk-lockfiles audit command should run successfully after updating all lockfiles:

➜  smithy-rs git:(ysaito/enhance-gradle-tasks-for-lockfile) ✗ sdk-lockfiles audit                       
2024-09-12T15:35:47.890530Z  INFO sdk_lockfiles::audit: checking whether `rust-runtime/Cargo.lock` is covered by the SDK lockfile...
2024-09-12T15:35:47.922468Z  INFO sdk_lockfiles::audit: checking whether `aws/rust-runtime/Cargo.lock` is covered by the SDK lockfile...
2024-09-12T15:35:47.922898Z  INFO sdk_lockfiles::audit: checking whether `aws/rust-runtime/aws-config/Cargo.lock` is covered by the SDK lockfile...
SUCCESS

I also specified multiple broken dependencies and verified they were all downgraded to the specified versions.

When a new dependency is added to a runtime crate, running syncAwsSdkLockfile will ensure that this new dependency is included in the SDK lockfile.

Expand for more details...

For instance, with this hypothetical new dependency:

smithy-rs git:(ysaito/enhance-gradle-tasks-for-lockfile) ✗ sdk-lockfiles audit
2024-09-12T15:40:52.795951Z  INFO sdk_lockfiles::audit: checking whether `rust-runtime/Cargo.lock` is covered by the SDK lockfile...
2024-09-12T15:40:52.827407Z  INFO sdk_lockfiles::audit: checking whether `aws/rust-runtime/Cargo.lock` is covered by the SDK lockfile...
2024-09-12T15:40:52.827835Z  INFO sdk_lockfiles::audit: checking whether `aws/rust-runtime/aws-config/Cargo.lock` is covered by the SDK lockfile...
`jiff` (0.1.13), used by `rust-runtime/Cargo.lock`, is not contained in SDK lockfile!
Error: there are lockfile audit failures

If we then execute

smithy-rs git:(ysaito/enhance-gradle-tasks-for-lockfile) ✗ ./gradlew aws:sdk:syncAwsSdkLockfile
...
BUILD SUCCESSFUL in 1m 17s

the SDK lockfile will be updated to reflect only the change from rust-runtime/Cargo.lock:

smithy-rs git:(ysaito/enhance-gradle-tasks-for-lockfile) ✗ git diff aws/sdk/Cargo.lock 
diff --git a/aws/sdk/Cargo.lock b/aws/sdk/Cargo.lock
index bc3870e20..c52040432 100644
--- a/aws/sdk/Cargo.lock
+++ b/aws/sdk/Cargo.lock
@@ -1627,6 +1627,7 @@ dependencies = [
  "aws-smithy-types 1.2.6",
  "chrono",
  "futures-core",
+ "jiff",
  "time",
 ]
 
@@ -2895,6 +2896,12 @@ version = "1.0.11"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "49f1f14873335454500d59611f1cf4a4b0f786f9ac11f4312a78e4cf2566695b"
 
+[[package]]
+name = "jiff"
+version = "0.1.13"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "8a45489186a6123c128fdf6016183fcfab7113e1820eb813127e036e287233fb"
+
 [[package]]
 name = "jobserver"
 version = "0.1.32"
(END)

The updated SDK lockfile should now be in sync with runtime crates:

➜  smithy-rs git:(ysaito/enhance-gradle-tasks-for-lockfile) ✗ sdk-lockfiles audit
2024-09-12T15:41:28.004702Z  INFO sdk_lockfiles::audit: checking whether `rust-runtime/Cargo.lock` is covered by the SDK lockfile...
2024-09-12T15:41:28.034118Z  INFO sdk_lockfiles::audit: checking whether `aws/rust-runtime/Cargo.lock` is covered by the SDK lockfile...
2024-09-12T15:41:28.034555Z  INFO sdk_lockfiles::audit: checking whether `aws/rust-runtime/aws-config/Cargo.lock` is covered by the SDK lockfile...
SUCCESS

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[github-actions]

A new generated diff is ready to view.

• AWS SDK (ignoring whitespace)
• No codegen difference in the Client Test
• No codegen difference in the Server Test
• No codegen difference in the Server Test Python
• No codegen difference in the Server Test Typescript

A new doc preview is ready to view.