feat: unify queues in proxy #976

SantiagoPittella · 2024-11-19T21:15:41Z

closes #947

bin/tx-prover/src/proxy/mod.rs

tomyrd · 2024-11-22T15:12:35Z

bin/tx-prover/src/proxy/mod.rs

+            // We use a new scope for each iteration to release the lock before sleeping
            {


I don't think we need this new scope as we are not acquiring a lock anymore

Yes, you are right. Removing the scope.

bobbinth

Looks good! Thank you! I left some comments inline.

bin/tx-prover/src/proxy/mod.rs

bobbinth · 2024-11-23T01:00:19Z

bin/tx-prover/src/proxy/mod.rs

+    fn new() -> Self {
+        Self {
+            tries: 0,
+            request_id: rand::random::<u64>(),


Previously, we were getting request ID from the session. Is there a reason not to do it now?

Yes, I changed it to use the context because it was fairly less complex this way. Using the session required to store the request_id in the headers and access it at almost every method in the life cycle of the request. This way the initialization is way easier (CTX::new is implicitly called at the start of every request) and the access is much easier. And since it is something that was not useful to the worker nor the user, it sounded like the most correct option.

Makes sense. I think it is probably fine to keep as is, but could we increase the value to u128 (to make the probability of collisions truly negligible).

A very common approach would be to use uuid here, but anything of that size should work fine

bin/tx-prover/src/proxy/mod.rs

bobbinth

Looks good! Thank you! I left a few small comments inline. After these are addressed, we should be good to merge.

bobbinth · 2024-11-26T06:49:44Z

bin/tx-prover/src/proxy/mod.rs

+    /// Set an available worker
+    ///
+    /// This method will add a worker to the list of available workers.
+    /// If the worker is already available, it will panic.
+    pub async fn add_available_worker(&self, worker: Backend) {


Line 59 needs to be updated. But also we could probably re-write this comment (and other similar related comments) as:

/// Adds the provided worker to the list of available workers. /// /// # Panics /// Panics if the provided worker is already in the list of available workers.

bobbinth · 2024-11-26T06:50:09Z

bin/tx-prover/src/proxy/mod.rs

+    /// Get an available worker
+    ///
+    /// This method will return the first available worker from the list of available workers, and
+    /// remove it from the list.
+    /// If no worker is available, it will return None.
+    pub async fn get_available_worker(&self) -> Option<Backend> {


Similar comment as above.

bobbinth · 2024-11-26T06:55:32Z

bin/tx-prover/src/proxy/mod.rs

+    fn new() -> Self {
+        Self {
+            tries: 0,
+            request_id: rand::random::<u64>(),


Makes sense. I think it is probably fine to keep as is, but could we increase the value to u128 (to make the probability of collisions truly negligible).

igamigo

LGTM. Left a couple of minor comments, mostly stylistic

igamigo · 2024-11-26T14:34:50Z

bin/tx-prover/src/proxy/mod.rs


-const RESOURCE_EXHAUSTED_CODE: u16 = 8;
+// LoadBalancer


nit: This would be written in all caps

igamigo · 2024-11-26T14:35:43Z

bin/tx-prover/src/proxy/mod.rs

+/// Rate limiter
+static RATE_LIMITER: Lazy<Rate> = Lazy::new(|| Rate::new(Duration::from_secs(1)));
+
+// Request queue


Same as the last comment

igamigo · 2024-11-26T14:37:55Z

bin/tx-prover/src/proxy/mod.rs

-/// Shared state. It is a map of workers to a vector of request IDs
-static QUEUES: Lazy<RwLock<HashMap<Backend, Vec<String>>>> =
-    Lazy::new(|| RwLock::new(HashMap::new()));
+// RequestContext


Same stylistic comment here

igamigo · 2024-11-26T14:42:18Z

bin/tx-prover/src/proxy/mod.rs

+    fn new() -> Self {
+        Self {
+            tries: 0,
+            request_id: rand::random::<u64>(),


A very common approach would be to use uuid here, but anything of that size should work fine

igamigo · 2024-11-26T14:44:26Z

bin/tx-prover/src/proxy/mod.rs

+            // Check if there is an available worker
+            if let Some(worker) = self.get_available_worker().await {
+                ctx.set_worker(worker);
+                info!("Worker picked up the request with ID: {}", request_id);


It would be nice to identify which worker picked up the request here for request traceability.

Ok! Changing it.

igamigo · 2024-11-26T14:46:48Z

bin/tx-prover/src/proxy/mod.rs

-        // deletion.
-        Self::remove_request_from_queue(request_id).await;
+        // Mark the worker as available
+        self.add_available_worker(ctx.worker.take().expect("Failed to get worker"))


I think I'd add an info!() log here (or maybe better within the add_available_worker function) that mentions that the worker became available. Basically I'm trying to think if we have enough request and traceability to be able to tell how the system was working if something went wrong.

tomyrd

Looks good, left a couple of minor comments

tomyrd · 2024-11-26T15:03:12Z

bin/tx-prover/src/proxy/mod.rs

    /// Note that the request is not removed from the queue here. It will be returned later in
    /// [Self::logging()] once the worker processes the it.


We should remove this part as the dequeue now happens here.

tomyrd · 2024-11-26T15:05:42Z

bin/tx-prover/src/proxy/mod.rs

+    /// Gets an available worker and removes it from the list of available workers.
+    ///
+    /// If no worker is available, it will return None.
+    pub async fn get_available_worker(&self) -> Option<Backend> {


nit: could we rename this to pop_available_worker just to make it clear that it's a get+remove

much better

…o utils

… info logs, fix documentation

SantiagoPittella requested review from bobbinth, igamigo and tomyrd and removed request for bobbinth and igamigo November 20, 2024 13:51

igamigo reviewed Nov 20, 2024

View reviewed changes

SantiagoPittella force-pushed the santiagopittella-unify-queues-in-proxy branch from b756f62 to 37c906e Compare November 20, 2024 20:25

SantiagoPittella requested a review from igamigo November 22, 2024 14:20

tomyrd reviewed Nov 22, 2024

View reviewed changes

bobbinth reviewed Nov 23, 2024

View reviewed changes

SantiagoPittella requested review from bobbinth and tomyrd November 25, 2024 16:11

bobbinth approved these changes Nov 26, 2024

View reviewed changes

igamigo approved these changes Nov 26, 2024

View reviewed changes

igamigo reviewed Nov 26, 2024

View reviewed changes

tomyrd approved these changes Nov 26, 2024

View reviewed changes

SantiagoPittella added 11 commits November 26, 2024 15:21

wip: unify queues in proxy

4c31ed0

fix: remove prints, update documentation

0a6d4f2

fix: remove sleep

c6eb324

docs: update changelog

145156d

feat: use struct for queue, add methods for context, move functions t…

50aa3f3

…o utils

docs: add separator for structs, add doc comments

5fd99fe

feat: use VecDequeue instead of Vec for queues

f539c6f

fix: add concurrent to makefile installation

8d9a697

rename set_available_worker, use assert instead of if

f6912f4

fix: use constant for interval time

b0472f5

fix: fix docs, remove unused scope, remove _ from var

3df47cd

SantiagoPittella added 4 commits November 26, 2024 15:21

fix: changelog entry

156ee3a

feat: use u128 for request_id

e8bdaad

docs: update {add, get}_available_worker documentation

c6b5321

fix: use mayus for structs sections, rename get_available_worker, add…

d11c8e8

… info logs, fix documentation

SantiagoPittella force-pushed the santiagopittella-unify-queues-in-proxy branch from a31360a to d11c8e8 Compare November 26, 2024 18:43

SantiagoPittella merged commit 3fefe18 into next Nov 26, 2024
9 checks passed

SantiagoPittella deleted the santiagopittella-unify-queues-in-proxy branch November 26, 2024 21:30

bobbinth mentioned this pull request Nov 27, 2024

Rework queue and errors in prover proxy #947

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: unify queues in proxy #976

feat: unify queues in proxy #976

SantiagoPittella commented Nov 19, 2024

tomyrd Nov 22, 2024

SantiagoPittella Nov 25, 2024

bobbinth left a comment

bobbinth Nov 23, 2024

SantiagoPittella Nov 25, 2024

bobbinth Nov 26, 2024

igamigo Nov 26, 2024

bobbinth left a comment

bobbinth Nov 26, 2024

bobbinth Nov 26, 2024

bobbinth Nov 26, 2024

igamigo left a comment

igamigo Nov 26, 2024

igamigo Nov 26, 2024

igamigo Nov 26, 2024

igamigo Nov 26, 2024

igamigo Nov 26, 2024

SantiagoPittella Nov 26, 2024

igamigo Nov 26, 2024

SantiagoPittella Nov 26, 2024

tomyrd left a comment

tomyrd Nov 26, 2024

tomyrd Nov 26, 2024

SantiagoPittella Nov 26, 2024

		// We use a new scope for each iteration to release the lock before sleeping
		{

		/// Note that the request is not removed from the queue here. It will be returned later in
		/// [Self::logging()] once the worker processes the it.

feat: unify queues in proxy #976

feat: unify queues in proxy #976

Conversation

SantiagoPittella commented Nov 19, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bobbinth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bobbinth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

igamigo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tomyrd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment