Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase write quorum timeout to 1m #3521

Merged
merged 1 commit into from
Jul 20, 2022
Merged

Conversation

theoilie
Copy link
Contributor

Description

Increase timeout for write quorum to wait 1 minute for a sync to complete to at least 1 secondary. This will impact the buckets for the audius_cn_write_quorum_duration_seconds metric, so I confirmed with @jonaylor89 that this will be OK for Prometheus since it's NoSQL.

Tests

N/A

Monitoring - How will this change be monitored? Are there sufficient logs / alerts?

Timeouts in the logs should show 1 minute (or stop timing out) instead of 5 seconds. The log message "Failed to reach 2/3 write quorum for user <WALLET> in <TIME>ms" should have a larger than 5000.

@theoilie theoilie added the content-node Content Node (previously known as Creator Node) label Jul 20, 2022
@theoilie theoilie requested a review from SidSethi July 20, 2022 21:15
creator-node/src/config.js Show resolved Hide resolved
@theoilie theoilie merged commit 43e43fe into master Jul 20, 2022
@theoilie theoilie deleted the theo-increase-write-quorum branch July 20, 2022 21:45
audius-infra pushed a commit that referenced this pull request Jul 22, 2022
## Changelog

- 2022-07-21 [6db3974] Sanity check setIsCreatorToTrue (#3531) [Sid Sethi]
- 2022-07-21 [4af6a34] Revert Revert [PLAT-254] Remove is_creator usage across protocol (#3475) (#3529) (#3530) [Dylan Jeffers]
- 2022-07-21 [6311d71] Revert [PLAT-254] Remove is_creator usage across protocol (#3475) (#3529) [Dylan Jeffers]
- 2022-07-21 [e91d0ef] [PAY-455] SDK: Convert TransactionHandler to TypeScript and export on core (#3515) [Marcus Pasell]
- 2022-07-21 [17c9130] [CON-233] Add queue history for all remaining queues (#3524) [Johannes Naylor]
- 2022-07-21 [9ce2208] More state machine cleanup (#3522) [Theo Ilie]
- 2022-07-20 [2fad6f0] [CON-262] Add more test coverage for /export (#3520) [Johannes Naylor]
- 2022-07-20 [15006a2] [CON-243] - Use prometheus middleware package to track route duration (#3494) [vicky :)]
- 2022-07-20 [43e43fe] Increase write quorum timeout to 1m (#3521) [Theo Ilie]
- 2022-07-20 [914bfa3] [CON-103] - Enable cid cache (#3519) [vicky :)]
- 2022-07-20 [338f8eb] Clean up misc state machine (#3518) [Theo Ilie]
- 2022-07-20 [bfc6505] [C-587] Verify sdk types in ci (#3495) [Sebastian Klingler]
- 2022-07-19 [af2e650] [C-688] Move docs over to monorepo (#3504) [Raymond Jacobson]
- 2022-07-19 [c993992] CON-258 Modify findSyncRequest.jobProcessor with more detailed metrics (#3478) [Sid Sethi]
- 2022-07-19 [afe0b61] Fix primarySyncFromSecondary circular dep by moving initAudiusLibs to separate file (#3507) [Sid Sethi]
sliptype pushed a commit that referenced this pull request Sep 10, 2023
[a16abc2] Revert force mobile full staging build (#3564) nicoback2
[27f0a1b] [PAY-1386] DMs: Mark chat as read on client when new message from self comes through websocket (#3562) Marcus Pasell
[ab41a52] Force mobile build again (#3561) nicoback2
[ac383dc] DMs: Fix broken messages (#3559) Marcus Pasell
[29b49ee] Force mobile staging build - Android (#3560) nicoback2
[57ff5c5] Force mobile staging build (#3558) nicoback2
[5456271] Update plist to 1.1.66 (#3557) Marcus Pasell
[48c70b1] [PAY-1398] Fix ChatUserListScreen scrolling with keyboard (#3552) Reed
[ba19c1d] [PAY-1423] Fix empty chat message box with currently playing track (#3553) Reed
[970224f] Mobile ChatMessageListItem uses message id + selector (#3551) Reed
[49ff351] [PAY-1426] DMs: Simplify unfurl styles for mobile, fix border radius shadow (#3555) Marcus Pasell
[63faad0] [PAY-1403] Fix chat track unfurl reactions (#3550) Reed
[a8e92b7] [PAY-1425][PAY-1424] DMs: Fix image unfurls, fix purple halo around unfurls (#3554) Marcus Pasell
[940aebe] [PAY-1367][PAY-1400][PAY-1401][PAY-1402][PAY-1419] Fix chat tile unfurl bugs (#3541) Saliou Diallo
[61b6735] [PAY-1378] Cap image size on mobile DMs (#3548) Reed
[a495c35] [PAY-1409] DMs: Deep link into chat from notification (#3549) Marcus Pasell
[a4e6426] [PAY-1393] DMs: Hide message underneath popover (#3547) Marcus Pasell
[a6711f1] [C-2691] Carry values through the form fields (#3539) Andrew Mendelsohn
[d653f5c] [C-2747] Fix track title layout with offline + ai attribution (#3546) Andrew Mendelsohn
[8e54ff7] [PAY-1408][PAY-1351][PAY-1383][PAY-1417][PAY-1240][PAY-1345] DMs: Web UI polishes and bugfixes (#3543) Marcus Pasell
[cc1bc46] [C-2746] Fix favorites album tab styling (#3544) Kyle Shanks
[1dbb753] [PAY-1406][PAY-1374][PAY-1353][PAY-1411] Fix DMs UI Issues (#3542) Michael Piazza
[b76eef2] [PAY-1348] clean up ChatUserListScreen and use followers as default list (#3538) Randy Schott
[3ed58ac] DMs: Wait for tip confirmation then refetch permissions (#3537) Marcus Pasell
[bb4f317] [PAY-1366] Fix web chat tile artwork and mobile link preview bugs (#3528) Saliou Diallo
[ea47ecf] DMs: Fix deletions not causing perms rechecks (#3540) Marcus Pasell
[0b8c993] [PAY-1356] Mobile chats scroll to bottom when re-entering chat screen (#3536) Reed
[52f7b94] [PAY-1363] DMs: Add CTA to Create Chat user rows (#3530) Marcus Pasell
[4c22b12] [C-2461, C-2476] Prevent user from dragging hidden tracks to public playlists (#3535) Kyle Shanks
[f2ad1ce] [C-1873, C-2551, C-2552, C-2553] Add hidden track logic for playlists on web (#3529) Kyle Shanks
[083b0ff] [PAY-1315] Fix mobile chat search screen takes 2 presses (#3534) Reed
[da64d93] [PAY-1347] DMs: Don't show unread indicator for first messages (#3532) Marcus Pasell
[7ae88cb] [PAY-1394] DMs: Show global unread indicator if there's unread messages in a loaded chat (#3533) Marcus Pasell
[40abf58] [PAY-1350] Update send button styling on web (#3523) Michael Piazza
[cd1e238] [PAY-1149] Mobile chat reactions add android offset (#3531) Reed
[3595ce8] [PAY-1362] Mobile press chat message closes reaction popup (#3527) Reed
[06d10e2] [PAY-1361] Fix reactions button hit area (#3524) Michael Piazza
[e97ed82] [PAY-1309] Use proper tracking of hasMore for chats list (#3517) Randy Schott
[fe72398] [PAY-1365] Disable fullscreen gestures on mobile chat screen (#3526) Reed
[03cbcac] [PAY-1392] Mobile mark chat as read onscroll, on navigate away (#3522) Reed
[e6af9d7] [PAY-919][PAY-1312] Chat track and playlist tiles on mobile (#3455) Saliou Diallo
[e2896fe] [C-2719] Clean up and document non v1 usage (#3520) Raymond Jacobson
[2cf5bb7] Add reason to account slice, handle fetch account failure with sentry (#3525) Raymond Jacobson
[4451447] [PAY-1342][PAY-1360] DMs: Show users as chat unlocked by default, optimistic chat navigation (#3514) Marcus Pasell
[8262221] [PAY-1391] ensure all collections are fetched before attempting to download (#3521) Randy Schott
[e29d35f] [PAY-1385] Mobile chat send icon updates (#3516) Reed
[8cda7b2] [PAY-1390] fix inverted condition for computing track count (#3519) Randy Schott
[3a3b206] [PAY-1333] Mobile dms end of history only shows with >10 messages (#3515) Reed
[f1f5dc0] [PAY-1370] Fix reaction touch targets (#3513) Reed
[9b81487] Upgrade sdk to 2.0.3-beta.36 (#3512) Dylan Jeffers
[b48777a] Remove content node cid query route (#3408) Michelle Brier
[69e5656] [C-2717] Fix playlist create tile bug (#3510) Kyle Shanks
[828015f] Remove yield* in js files (#3511) Sebastian Klingler
[7d26233] Add mobile hidden track playlist logic (#3505) Kyle Shanks
[cf9c5d5] [PAY-1137][PAY-1132] Infinite loading of albums on mobile favorites screen (#3398) Randy Schott
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
content-node Content Node (previously known as Creator Node) size/XS
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants