Skip to content

Commit

Permalink
[data ingestion] redirect traffic to the bucket directly (#20672)
Browse files Browse the repository at this point in the history
## Description 

temporary PR. Implicitly redirects traffic to the GCS bucket directly
making external developers to use a cheaper version of service


---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
  • Loading branch information
phoenix-o authored Dec 18, 2024
1 parent 84141f5 commit 9229371
Showing 1 changed file with 12 additions and 1 deletion.
13 changes: 12 additions & 1 deletion crates/sui-data-ingestion-core/src/reader.rs
Original file line number Diff line number Diff line change
Expand Up @@ -333,7 +333,7 @@ impl CheckpointReader {
let (exit_sender, exit_receiver) = oneshot::channel();
let reader = Self {
path,
remote_store_url,
remote_store_url: remote_store_url.map(transform_ingestion_url),
remote_store_options,
current_checkpoint_number: starting_checkpoint_number,
last_pruned_watermark: starting_checkpoint_number,
Expand Down Expand Up @@ -429,3 +429,14 @@ impl DataLimiter {
self.in_progress = self.queue.values().sum();
}
}

fn transform_ingestion_url(ingestion_url: String) -> String {
// temporary code to redirect ingestion traffic directly to the bucket
if ingestion_url.contains("checkpoints.mainnet.sui.io") {
"https://storage.googleapis.com/mysten-mainnet-checkpoints".to_string()
} else if ingestion_url.contains("checkpoints.testnet.sui.io") {
"https://storage.googleapis.com/mysten-testnet-checkpoints".to_string()
} else {
ingestion_url
}
}

0 comments on commit 9229371

Please sign in to comment.