fix: notify flush receiver after write buffer is released

[evenyag]

I hereby agree to the terms of the GreptimeDB CLA.

Refer to a related PR or issue link (optional)

fixes #4475

What's changed and what's your intention?

This PR notifies all workers once a write buffer (memtable) is released so workers can handle stalled requests.

It also adds worker id to some logs and metrics so we can know the relationship between regions and workers.

Checklist

• I have written the necessary rustdoc comments.
• I have added the necessary unit tests and integration tests.
• This PR requires documentation updates.

Summary by CodeRabbit

• New Features

  • Introduced a notification system for memory management in the Write Buffer Manager, improving responsiveness.
  • Enhanced metrics tracking with REGION_COUNT now providing per-worker granularity.

• Bug Fixes

  • Improved logging to include worker IDs, aiding in better traceability during operations.

• Refactor

  • Shifted from global metrics to instance-based tracking for region counts, enhancing modularity and maintainability.

• Tests

  • Added a new test to validate the notifier functionality within the Write Buffer Manager.

[coderabbitai]

Walkthrough

The recent changes introduce a robust memory management notification system within the WriteBufferManagerImpl, enhancing responsiveness by notifying worker threads when memory is freed. Additionally, metrics tracking has been improved with the transition from global to instance-specific counters, particularly for REGION_COUNT, allowing for better performance monitoring. These modifications collectively enhance the control flow and observability in the storage engine, addressing potential performance issues.

Changes

Files	Change Summary
src/mito2/src/{flush.rs,worker.rs}	Enhanced WriteBufferManagerImpl with memory management notifications and updated worker initialization.
src/mito2/src/metrics.rs	Changed REGION_COUNT from IntGauge to IntGaugeVec for per-worker tracking.
src/mito2/src/worker/{handle_close.rs,handle_create.rs,handle_drop.rs,handle_open.rs}	Improved logging and shifted region count management from global to instance-specific metrics.

Sequence Diagram(s)

sequenceDiagram
    participant W as Worker
    participant M as WriteBufferManagerImpl
    participant N as Notifier

    W->>M: Request memory allocation
    M-->>W: Memory allocated
    M->>N: Notify memory change
    N-->>W: Memory freed
    W->>M: Continue processing

Loading

Assessment against linked issues

Objective	Addressed	Explanation
Data ingestion recovery when flush finishes (#4475)	✅
Resolve region stalls during high data ingestion (#4475)	✅

🐰 "In the fields where bunnies dwell,
Memory's tales we now can tell.
Workers hop with joy anew,
Metrics bright, and logging too!
In the garden of code we play,
Performance blooms, come what may!" 🌼

---

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

• X
• Mastodon
• Reddit
• LinkedIn

Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>.
  • Generate unit testing code for this file.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
  • @coderabbitai generate unit testing code for this file.
  • @coderabbitai modularize this function.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai generate interesting stats about this repository and render them as a table.
  • @coderabbitai show all the console.log statements in this repository.
  • @coderabbitai read src/utils.ts and generate unit testing code.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
  • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai full review to do a full review from scratch and review all the files again.
• @coderabbitai summary to regenerate the summary of the PR.
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
• @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configuration File (.coderabbit.yaml)

• You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
• Please see the configuration documentation for more information.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[codecov]

Codecov Report

Attention: Patch coverage is 90.19608% with 5 lines in your changes missing coverage. Please review.

Project coverage is 84.60%. Comparing base (90301a6) to head (488716b).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4476      +/-   ##
==========================================
- Coverage   84.88%   84.60%   -0.28%     
==========================================
  Files        1081     1081              
  Lines      193177   193208      +31     
==========================================
- Hits       163971   163472     -499     
- Misses      29206    29736     +530     

[evenyag]

@coderabbitai summary

[coderabbitai]

Actions performed

Summary regeneration triggered.

[waynexia]

This way is kind of error-prone and doesn't smell good

[evenyag]

This way is kind of error-prone and doesn't smell good

The write buffer limit is global so typically we don't have a perfect way.