Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CSPM] [Azure] [SDH] Timeout on Azure fetchers #2624

Open
moukoublen opened this issue Oct 18, 2024 · 1 comment
Open

[CSPM] [Azure] [SDH] Timeout on Azure fetchers #2624

moukoublen opened this issue Oct 18, 2024 · 1 comment
Labels
bug Something isn't working Team:Cloud Security Cloud Security team related

Comments

@moukoublen
Copy link
Member

moukoublen commented Oct 18, 2024

Describe the bug
Azure fetchers in accounts with large number of resources take more time than anticipated and exceed the timeout in cloudbeat internal fetchers handling that is hardcoded to 20 mins.

Preconditions
CSPM Azure is a subscription with a large number of resources.

Potential Solutions
We should either increase the hardcoded fetcher timeout to e.g. 3 hours (it seems there is no reason for less since the cycle is usually 24h) [quick fix].

or

Change the approach entirely and not limit each fetcher with a standalone timeout, but instead, use a context with a timeout from the beginning of the cycle + cycle time.

@moukoublen moukoublen added bug Something isn't working Team:Cloud Security Cloud Security team related labels Oct 18, 2024
@orouz
Copy link
Collaborator

orouz commented Oct 20, 2024

the second approach sounds better. i think we should utilize period (from cloudbeat config) and let the fetchers take as much time as the period allows.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Team:Cloud Security Cloud Security team related
Projects
None yet
Development

No branches or pull requests

2 participants