Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blocks queue #71

Closed
bararchy opened this issue Aug 26, 2020 · 1 comment · Fixed by #72
Closed

Blocks queue #71

bararchy opened this issue Aug 26, 2020 · 1 comment · Fixed by #72
Assignees
Labels
Priority: high Type: perf A code change that improves performance

Comments

@bararchy
Copy link
Member

target direct repeater
www.neuralegion.com 5.4 req/s 0.3 req/s
owasp benchamrk 104.6 req/s 1.0 req/s

We need to figure out how to manage a faster event bus to cater to more requests.
Old crystal based repeater managed ~60Rps when I tested.

@bararchy bararchy added Type: bug Something isn't working. Priority: high labels Aug 26, 2020
@derevnjuk
Copy link
Member

derevnjuk commented Aug 27, 2020

@bararchy EventBus implements a regular RPC. It ensures reliably handle messages but blocks queue while performing a request.

I performed 5 tests which sent by 1000 requests with the following payload to development.playground.nexploit.app:

{
  "method": "GET",
  "url": "https://development.playground.nexploit.app/"
}
# Pattern req/s
1 Direct (async) 281.7
2 RPC current 1.4
3 Pub/Sub previous implementation 71.1
4 Queue per request (RPC) 77.7
5 Direct reply-to (RPC) 73.4

As you see 3, 4 and 5 implementations are much faster than 1 (regular RPC). But there're a few issues:

  • Pub/Sub: it doesn't guarantee delivery event. You need to close the consumer after the reply and requeue unexpected messages.
  • Queue per request: it is too expensive from a memory usage perspective
  • Direct reply-to: that same RPC, but requires no-ack mode. It will affect on latency if customers start multiple scans.

@derevnjuk derevnjuk changed the title Speed Issues when Using Repeater Blocks queue Aug 27, 2020
@derevnjuk derevnjuk added Type: perf A code change that improves performance and removed Type: bug Something isn't working. labels Aug 27, 2020
derevnjuk added a commit that referenced this issue Aug 27, 2020
derevnjuk added a commit that referenced this issue Aug 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Priority: high Type: perf A code change that improves performance
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants