Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Improve Git handler and introduce caching #1689

Merged
merged 5 commits into from
Sep 20, 2024
Merged

feat: Improve Git handler and introduce caching #1689

merged 5 commits into from
Sep 20, 2024

Commits on Sep 20, 2024

  1. feat: Improve git handler and introduce caching

    @dominik003 @MoritzWeber0
    dominik003 authored and MoritzWeber0 committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    ad0429a View commit details
    Browse the repository at this point in the history

  2. chore: Update openapi client

    @dominik003 @MoritzWeber0
    dominik003 authored and MoritzWeber0 committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    67539c0 View commit details
    Browse the repository at this point in the history

  3. feat: Several improvements for caching of diagram cache 

    This commit requests changes on the PR #1689. In particular, the following has
changed:

- Replace cache key from path and revision with git_model_id (database id). The
  unique database ID of the Git model in the Collaboration Manager has to be
  part of the cache key. For safety reasons, I'd not share a cache between
  projects even though they link to the same repository. This doesn't really
  matter currently since we make new requests against the Git API every time,
  but we can't assure that for the future.
  - An attack could look like:
  - Project 1 links Git repository "https://example.com" to a model.
  - Project 2 links the same Git repository to a model with a different token.
  - The token of project 1 expires. Project 2 keeps updating the cache with a
    valid token.
  - Users of project 1 can still access the cache for project 2 without a valid
    token.
- Since a git_model only has one revision, it's safe to drop the revision for
  artifacts. The revision is only relevant for files from the repository. In
  those cases, the value was passed as parameter every time. Therefore, I've
  removed it from the Cache object.
- For license reasons, I switched from Redis to Valkey.
- Use a Kubernetes secret for the valkey password.
- For REST compatibility, I've changed the resource name of "empty_cache" to
  "cache".
- I've reenabled the model badge error handling.
- Added a new function loadDiagramCacheMetadata in
  model-diagram-dialog.component.ts to reduce duplicated code.
- Added the job id to the diagram cache code snippet.
- Changed the styling of the "Clear cache" button.
    @MoritzWeber0
    MoritzWeber0 committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    60b4c20 View commit details
    Browse the repository at this point in the history

  4. feat: Apply PR comments

    @dominik003 @MoritzWeber0
    dominik003 authored and MoritzWeber0 committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    ab5061c View commit details
    Browse the repository at this point in the history

  5. feat: Several optimizations for valkey 

    - Add vertical autoscaler to track resource usage of valkey
- Increase TTL for valkey data
- Clear cache when git model is updated
- Mount data and config properly to valkey
- Fallback to traditional loading if saving or loading in valkey fails
    @MoritzWeber0
    MoritzWeber0 committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    9bc25bd View commit details
    Browse the repository at this point in the history