Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: health probe #96

Merged
merged 4 commits into from
Oct 10, 2023
Merged

chore: health probe #96

merged 4 commits into from
Oct 10, 2023

Conversation

mfw78
Copy link
Contributor

@mfw78 mfw78 commented Oct 6, 2023

Description

This PR provides overall health monitoring for the block watcher (chain contexts). This assists with Kubernetes deployment to enable health probes for automatic restarting and monitoring.

Changes

  • All chain contexts added to a static mapping for use in other services (ie. API).
  • Implemented /health API endpoint, returning >= 400 when the chain is not synced (in warm-up).

How to test

  1. Start syncing chain from scratch.
  2. Observe the /health API endpoint returns 500 status error, with JSON showing current chain context status.
  3. Once in sync, observe the /health API endpoint returns 200 status with JSON showing the current chain status.

Related Issues

Fixes #70

@mfw78 mfw78 added infra Infra, devops, CI and related tasks E:1.2: Watch Tower Service https://github.com/cowprotocol/pm/issues/8 labels Oct 6, 2023
@mfw78 mfw78 requested review from anxolin, ahhda and a team October 6, 2023 07:25
@mfw78 mfw78 self-assigned this Oct 6, 2023
@mfw78 mfw78 added the enhancement New feature or request label Oct 6, 2023
src/utils/api.ts Outdated Show resolved Hide resolved
src/domain/chainContext.ts Show resolved Hide resolved
src/domain/chainContext.ts Outdated Show resolved Hide resolved
src/domain/chainContext.ts Show resolved Hide resolved
src/domain/chainContext.ts Outdated Show resolved Hide resolved
src/domain/chainContext.ts Outdated Show resolved Hide resolved
src/utils/api.ts Show resolved Hide resolved
src/utils/api.ts Outdated Show resolved Hide resolved
src/utils/api.ts Outdated Show resolved Hide resolved
src/utils/api.ts Outdated Show resolved Hide resolved
src/utils/api.ts Outdated Show resolved Hide resolved
@mfw78 mfw78 force-pushed the chore-metrics branch 3 times, most recently from 7e80376 to e7b3d4e Compare October 7, 2023 03:41
@mfw78 mfw78 linked an issue Oct 8, 2023 that may be closed by this pull request
2 tasks
);
if (isRunningInKubernetesPod()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why mentioning here Kubernetes?
I think this could just be a setting: exit on stale chain state

this.sync = ChainSync.STALE;

if (exitOnStaleChainState) {
  await registry.storage.close();
  exit(1)
}

Base automatically changed from chore-metrics to main October 10, 2023 01:19
@mfw78 mfw78 merged commit f8eea81 into main Oct 10, 2023
3 checks passed
@mfw78 mfw78 deleted the health-probe branch October 10, 2023 11:53
@github-actions github-actions bot locked and limited conversation to collaborators Oct 10, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
E:1.2: Watch Tower Service https://github.com/cowprotocol/pm/issues/8 enhancement New feature or request infra Infra, devops, CI and related tasks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feat: observability
3 participants