Skip to content

Commit

Permalink
fix: clippy errors
Browse files Browse the repository at this point in the history
  • Loading branch information
zyy17 committed Sep 29, 2024
1 parent 72e5145 commit f858d75
Show file tree
Hide file tree
Showing 12 changed files with 35 additions and 47 deletions.
10 changes: 5 additions & 5 deletions config/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@
| `logging.slow_query` | -- | -- | The slow query log options. |
| `logging.slow_query.enable` | Bool | `false` | Whether to enable slow query log. |
| `logging.slow_query.threshold` | String | Unset | The threshold of slow query. |
| `logging.slow_query.sample_rate` | Float | Unset | The sampling rate of slow query log. |
| `logging.slow_query.sample_ratio` | Float | Unset | The sampling ratio of slow query log. The value should be in the range of (0, 1]. |
| `export_metrics` | -- | -- | The datanode can export its metrics and send to Prometheus compatible service (e.g. send to `greptimedb` itself) from remote-write API.<br/>This is only used for `greptimedb` to export its own metrics internally. It's different from prometheus scrape. |
| `export_metrics.enable` | Bool | `false` | whether enable export metrics. |
| `export_metrics.write_interval` | String | `30s` | The interval of export metrics. |
Expand Down Expand Up @@ -256,7 +256,7 @@
| `logging.slow_query` | -- | -- | The slow query log options. |
| `logging.slow_query.enable` | Bool | `false` | Whether to enable slow query log. |
| `logging.slow_query.threshold` | String | Unset | The threshold of slow query. |
| `logging.slow_query.sample_rate` | Float | Unset | The sampling rate of slow query log. |
| `logging.slow_query.sample_ratio` | Float | Unset | The sampling ratio of slow query log. The value should be in the range of (0, 1]. |
| `export_metrics` | -- | -- | The datanode can export its metrics and send to Prometheus compatible service (e.g. send to `greptimedb` itself) from remote-write API.<br/>This is only used for `greptimedb` to export its own metrics internally. It's different from prometheus scrape. |
| `export_metrics.enable` | Bool | `false` | whether enable export metrics. |
| `export_metrics.write_interval` | String | `30s` | The interval of export metrics. |
Expand Down Expand Up @@ -325,7 +325,7 @@
| `logging.slow_query` | -- | -- | The slow query log options. |
| `logging.slow_query.enable` | Bool | `false` | Whether to enable slow query log. |
| `logging.slow_query.threshold` | String | Unset | The threshold of slow query. |
| `logging.slow_query.sample_rate` | Float | Unset | The sampling rate of slow query log. |
| `logging.slow_query.sample_ratio` | Float | Unset | The sampling ratio of slow query log. The value should be in the range of (0, 1]. |
| `export_metrics` | -- | -- | The datanode can export its metrics and send to Prometheus compatible service (e.g. send to `greptimedb` itself) from remote-write API.<br/>This is only used for `greptimedb` to export its own metrics internally. It's different from prometheus scrape. |
| `export_metrics.enable` | Bool | `false` | whether enable export metrics. |
| `export_metrics.write_interval` | String | `30s` | The interval of export metrics. |
Expand Down Expand Up @@ -481,7 +481,7 @@
| `logging.slow_query` | -- | -- | The slow query log options. |
| `logging.slow_query.enable` | Bool | `false` | Whether to enable slow query log. |
| `logging.slow_query.threshold` | String | Unset | The threshold of slow query. |
| `logging.slow_query.sample_rate` | Float | Unset | The sampling rate of slow query log. |
| `logging.slow_query.sample_ratio` | Float | Unset | The sampling ratio of slow query log. The value should be in the range of (0, 1]. |
| `export_metrics` | -- | -- | The datanode can export its metrics and send to Prometheus compatible service (e.g. send to `greptimedb` itself) from remote-write API.<br/>This is only used for `greptimedb` to export its own metrics internally. It's different from prometheus scrape. |
| `export_metrics.enable` | Bool | `false` | whether enable export metrics. |
| `export_metrics.write_interval` | String | `30s` | The interval of export metrics. |
Expand Down Expand Up @@ -531,6 +531,6 @@
| `logging.slow_query` | -- | -- | The slow query log options. |
| `logging.slow_query.enable` | Bool | `false` | Whether to enable slow query log. |
| `logging.slow_query.threshold` | String | Unset | The threshold of slow query. |
| `logging.slow_query.sample_rate` | Float | Unset | The sampling rate of slow query log. |
| `logging.slow_query.sample_ratio` | Float | Unset | The sampling ratio of slow query log. The value should be in the range of (0, 1]. |
| `tracing` | -- | -- | The tracing options. Only effect when compiled with `tokio-console` feature. |
| `tracing.tokio_console_addr` | String | Unset | The tokio console address. |
4 changes: 2 additions & 2 deletions config/datanode.example.toml
Original file line number Diff line number Diff line change
Expand Up @@ -595,9 +595,9 @@ enable = false
## @toml2docs:none-default
threshold = "10s"

## The sampling rate of slow query log.
## The sampling ratio of slow query log. The value should be in the range of (0, 1].
## @toml2docs:none-default
sample_rate = 1.0
sample_ratio = 1.0

## The datanode can export its metrics and send to Prometheus compatible service (e.g. send to `greptimedb` itself) from remote-write API.
## This is only used for `greptimedb` to export its own metrics internally. It's different from prometheus scrape.
Expand Down
4 changes: 2 additions & 2 deletions config/flownode.example.toml
Original file line number Diff line number Diff line change
Expand Up @@ -93,9 +93,9 @@ enable = false
## @toml2docs:none-default
threshold = "10s"

## The sampling rate of slow query log.
## The sampling ratio of slow query log. The value should be in the range of (0, 1].
## @toml2docs:none-default
sample_rate = 1.0
sample_ratio = 1.0

## The tracing options. Only effect when compiled with `tokio-console` feature.
[tracing]
Expand Down
4 changes: 2 additions & 2 deletions config/frontend.example.toml
Original file line number Diff line number Diff line change
Expand Up @@ -200,9 +200,9 @@ enable = false
## @toml2docs:none-default
threshold = "10s"

## The sampling rate of slow query log.
## The sampling ratio of slow query log. The value should be in the range of (0, 1].
## @toml2docs:none-default
sample_rate = 1.0
sample_ratio = 1.0

## The datanode can export its metrics and send to Prometheus compatible service (e.g. send to `greptimedb` itself) from remote-write API.
## This is only used for `greptimedb` to export its own metrics internally. It's different from prometheus scrape.
Expand Down
4 changes: 2 additions & 2 deletions config/metasrv.example.toml
Original file line number Diff line number Diff line change
Expand Up @@ -187,9 +187,9 @@ enable = false
## @toml2docs:none-default
threshold = "10s"

## The sampling rate of slow query log.
## The sampling ratio of slow query log. The value should be in the range of (0, 1].
## @toml2docs:none-default
sample_rate = 1.0
sample_ratio = 1.0

## The datanode can export its metrics and send to Prometheus compatible service (e.g. send to `greptimedb` itself) from remote-write API.
## This is only used for `greptimedb` to export its own metrics internally. It's different from prometheus scrape.
Expand Down
4 changes: 2 additions & 2 deletions config/standalone.example.toml
Original file line number Diff line number Diff line change
Expand Up @@ -639,9 +639,9 @@ enable = false
## @toml2docs:none-default
threshold = "10s"

## The sampling rate of slow query log.
## The sampling ratio of slow query log. The value should be in the range of (0, 1].
## @toml2docs:none-default
sample_rate = 1.0
sample_ratio = 1.0

## The datanode can export its metrics and send to Prometheus compatible service (e.g. send to `greptimedb` itself) from remote-write API.
## This is only used for `greptimedb` to export its own metrics internally. It's different from prometheus scrape.
Expand Down
2 changes: 1 addition & 1 deletion src/cmd/src/frontend.rs
Original file line number Diff line number Diff line change
Expand Up @@ -353,7 +353,7 @@ impl StartCommand {
catalog_manager,
Arc::new(client),
meta_client,
StatementStatistics::new(&opts.logging.slow_query),
StatementStatistics::new(opts.logging.slow_query.clone()),
)
.with_plugin(plugins.clone())
.with_local_cache_invalidator(layered_cache_registry)
Expand Down
2 changes: 1 addition & 1 deletion src/cmd/src/standalone.rs
Original file line number Diff line number Diff line change
Expand Up @@ -558,7 +558,7 @@ impl StartCommand {
catalog_manager.clone(),
node_manager.clone(),
ddl_task_executor.clone(),
StatementStatistics::default(),
StatementStatistics::new(opts.logging.slow_query.clone()),
)
.with_plugin(plugins.clone())
.try_build()
Expand Down
12 changes: 6 additions & 6 deletions src/common/telemetry/src/logging.rs
Original file line number Diff line number Diff line change
Expand Up @@ -67,22 +67,22 @@ pub struct LoggingOptions {
pub tracing_sample_ratio: Option<TracingSampleOptions>,

/// The logging options of slow query.
pub slow_query: SlowQueryLoggingOptions,
pub slow_query: SlowQueryOptions,
}

/// The slow query log options.
/// The options of slow query.
#[derive(Clone, Debug, Serialize, Deserialize, Default)]
#[serde(default)]
pub struct SlowQueryLoggingOptions {
pub struct SlowQueryOptions {
/// Whether to enable slow query log.
pub enable: bool,

/// The threshold of slow queries.
#[serde(with = "humantime_serde")]
pub threshold: Option<Duration>,

/// The sample rate of slow queries.
pub sample_rate: Option<f64>,
/// The sample ratio of slow queries.
pub sample_ratio: Option<f64>,
}

#[derive(Clone, Debug, Copy, PartialEq, Eq, Serialize, Deserialize)]
Expand Down Expand Up @@ -115,7 +115,7 @@ impl Default for LoggingOptions {
otlp_endpoint: None,
tracing_sample_ratio: None,
append_stdout: true,
slow_query: SlowQueryLoggingOptions::default(),
slow_query: SlowQueryOptions::default(),
}
}
}
Expand Down
1 change: 1 addition & 0 deletions src/operator/src/statement.rs
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ pub struct StatementExecutor {
pub type StatementExecutorRef = Arc<StatementExecutor>;

impl StatementExecutor {
#[allow(clippy::too_many_arguments)]
pub fn new(
catalog_manager: CatalogManagerRef,
query_engine: QueryEngineRef,
Expand Down
34 changes: 11 additions & 23 deletions src/query/src/stats.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

use std::time::Duration;

use common_telemetry::logging::SlowQueryLoggingOptions;
use common_telemetry::logging::SlowQueryOptions;
use common_telemetry::slow;
use rand::random;

Expand All @@ -24,24 +24,13 @@ use crate::parser::QueryStatement;
#[derive(Default, Clone, Debug)]
pub struct StatementStatistics {
/// slow_query is used to configure slow query log.
pub slow_query: SlowQuery,
}

#[derive(Default, Clone, Debug)]
pub struct SlowQuery {
pub enable: bool,
pub threshold: Option<Duration>,
pub sample_rate: Option<f64>,
pub slow_query: SlowQueryOptions,
}

impl StatementStatistics {
pub fn new(slow_query_logging_options: &SlowQueryLoggingOptions) -> Self {
pub fn new(slow_query_options: SlowQueryOptions) -> Self {
Self {
slow_query: SlowQuery {
enable: slow_query_logging_options.enable,
threshold: slow_query_logging_options.threshold,
sample_rate: slow_query_logging_options.sample_rate,
},
slow_query: slow_query_options,
}
}

Expand All @@ -51,19 +40,20 @@ impl StatementStatistics {
start: std::time::Instant::now(),
stmt,
threshold: self.slow_query.threshold,
rate: self.slow_query.sample_rate,
sample_ratio: self.slow_query.sample_ratio,
})
} else {
None
}
}
}

/// SlowQueryTimer is used to log slow query when it's dropped.
pub struct SlowQueryTimer {
start: std::time::Instant,
stmt: QueryStatement,
threshold: Option<Duration>,
rate: Option<f64>,
sample_ratio: Option<f64>,
}

impl SlowQueryTimer {
Expand Down Expand Up @@ -93,15 +83,13 @@ impl Drop for SlowQueryTimer {
if let Some(threshold) = self.threshold {
let elapsed = self.start.elapsed();
if elapsed > threshold {
if let Some(rate) = self.rate {
if rate >= 1.0 {
if let Some(ratio) = self.sample_ratio {
// Generate a random number in [0, 1) and compare it with sample_ratio.
if ratio >= 1.0 || random::<f64>() <= ratio {
self.log_slow_query(elapsed, threshold);
} else {
if random::<f64>() < rate {
self.log_slow_query(elapsed, threshold);
}
}
} else {
// If sample_ratio is not set, log all slow queries.
self.log_slow_query(elapsed, threshold);
}
}
Expand Down
1 change: 0 additions & 1 deletion tests-integration/tests/http.rs
Original file line number Diff line number Diff line change
Expand Up @@ -898,7 +898,6 @@ fn drop_lines_with_inconsistent_results(input: String) -> String {
let inconsistent_results = [
"dir =",
"log_format =",
"log_format =",
"data_home =",
"bucket =",
"root =",
Expand Down

0 comments on commit f858d75

Please sign in to comment.