Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(improvement) Migrate from clickhouse to @clickhouse/client #246

Merged
merged 18 commits into from
Jul 22, 2024

Conversation

Blaumaus
Copy link
Member

@Blaumaus Blaumaus commented Jul 20, 2024

Ticket

Swetrix/roadmap#217

  1. Migrated everything from clickhouse to @clickhouse/client - now everything is much easier to use and maintain, there's also no need to write raw (parametrised) SQL insert queries, because the new library offers lots of methods for that.

  2. Dropped Redis for analytics data queuing. Now we rely on asynchronous queries for that. Clickhouse server will create batches on it's own and inject them into the database whenever it's ready. Now the data will be saved to the DB and displayed in dashboard almost instantly.
    image

  3. Errors like Error: 500: Code: 241. DB::Exception: Memory limit (total) exceeded: would use x.xx GiB (attempt to allocate chunk of xxxxxxx bytes), maximum: x.xx GiB. OvercommitTracker decision: Query was selected to stop by OvercommitTracker. (MEMORY_LIMIT_EXCEEDED) (version x.x.x.xx (official build)) will now be prevented because chunks are managed server-side.

Self-hosted support

  • Your feature is implemented for the selfhosted version of Swetrix
  • This PR only updates the production code (e.g. paddle webhooks, CAPTCHA, blog, etc.)

Database migrations

  • Clickhouse / MySQL migrations added for this PR
  • No table schemas changed in this PR

Documentation

  • You have updated the documentation according to your PR
  • This PR did not change any publicly documented endpoints

@Blaumaus Blaumaus self-assigned this Jul 20, 2024
@Blaumaus Blaumaus marked this pull request as ready for review July 21, 2024 15:49
@Blaumaus Blaumaus marked this pull request as draft July 21, 2024 16:13
@Blaumaus Blaumaus marked this pull request as ready for review July 22, 2024 19:05
@pro1code1hack
Copy link
Contributor

Great stuff! 💥

@Blaumaus Blaumaus merged commit d7e1e27 into main Jul 22, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants