Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add generator code #203

Merged
merged 14 commits into from
Jul 30, 2024
Merged

feat: add generator code #203

merged 14 commits into from
Jul 30, 2024

Conversation

samika98
Copy link
Contributor

@samika98 samika98 commented Jul 12, 2024

This PR introduces the initial logic for the runner. It includes the following key features:

  • Adds a new command to the runner.
  • Implements functionality to generate load against Ceramic instances.

This PR does not introduce any functional changes. It primarily adds interfaces and structures that will be used in future work. Once the operator changes are made, we will be able to call this command effectively.

This PR is part of a larger effort and contains several TODOs that will be addressed in subsequent PRs.
For a quick review, the focus is on the runner logic and the new command implementation, rust coding practices/improvements.

@samika98 samika98 requested review from dav1do and smrz2001 July 15, 2024 15:50
@samika98 samika98 marked this pull request as ready for review July 15, 2024 15:59
Copy link
Contributor

@dav1do dav1do left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! A lot of small comments/rust things but overall looks good.. I'm still really sad about tracing and would like to figure that out 😭

runner/src/load_generator/gen.rs Outdated Show resolved Hide resolved
runner/src/load_generator/gen.rs Show resolved Hide resolved
runner/src/load_generator/gen.rs Show resolved Hide resolved
runner/src/load_generator/gen.rs Show resolved Hide resolved
runner/src/load_generator/gen.rs Outdated Show resolved Hide resolved
runner/src/load_generator/utils/generator_utils.rs Outdated Show resolved Hide resolved
runner/src/load_generator/utils/mod.rs Outdated Show resolved Hide resolved
runner/src/main.rs Show resolved Hide resolved
runner/src/main.rs Outdated Show resolved Hide resolved
metrics_controller.force_flush()?;
}
drop(metrics_controller);
shutdown_meter_provider();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the drop/shutdown should maybe be in the if let statement since it's not used otherwise but it doesn't look like it's caused an issue

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should add back the drop(metrics_controller);. Probably doesn't matter now but it might cause some confusion when metrics are fixed and prevent them from showing up as expected.

@dav1do dav1do changed the title feature: add generator code feat: add generator code Jul 16, 2024
@samika98 samika98 requested a review from dav1do July 25, 2024 20:46
Copy link
Contributor

@dav1do dav1do left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM other than the one comment about dropping the metrics_controller. Probably need to run a cargo update to fix CI. It looks like the new clippy/rust release doesn't like the old version of time and cargo update -p time should do it, but updating all dependencies is probably fine... normally I'd suggest we should do that in another PR and merge it right away, but as this is close so I think doing it here is fine.

@@ -4,7 +4,7 @@ CARGO = CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse RUSTFLAGS="--cfg tokio_unstab
all: build check-fmt check-clippy test

.PHONY: test
test:
test: update
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we want to run update as part of these tasks. Adding an update step is fine, but we should run make update and commit the lockfile in a commit. I suspect this will cause unstaged changes in CI that could fail the pipeline.

Copy link
Collaborator

@smrz2001 smrz2001 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just minor comments. LGTM!

(review the Rust-y bits as well as I could 😊)

pub enum Command {
/// Bootstrap peers in the network
Bootstrap(bootstrap::Opts),
/// Simulate a load scenario against the network
Simulate(simulate::Opts),
/// Do nothing and exit
Noop,
/// Generate load, currently this command is not used
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: mind moving noop to the last position in the enum?

mod scenario;
mod simulate;
mod utils;

use keramik_common::telemetry;

use crate::gen::simulate_load;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: Can you move the crate use statements to after the external deps?

// so this is unused
#[allow(dead_code)]
#[derive(Clone, Debug)]
pub enum WeekLongSimulationScenarios {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about a different term instead of WeekLong since the run time is configurable? Maybe LoadGenScenarios? If so, can replace that throughout this file.

@samika98 samika98 added this pull request to the merge queue Jul 30, 2024
Merged via the queue into main with commit 24ed857 Jul 30, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants