Memory caches have an infinite lifetime and grow to consume infinite RAM

[warriordog]

💡 Summary

Several memory caches do not have an eviction policy and can grow to contain millions of entries per active process. This is highly inefficient and is a partial cause of the slow memory leak that affects some instances. In particular, these caches have an infinite eviction time:

• User objects (this cache is duplicated 4 additional times with varying keys)
• Public keys (this cache is duplicated with an alternate key)
• Client app metadata

As a solution, caches should define a maximum capacity and reasonable eviction time. If the capacity is exceeded, then the oldest (least recently used) entry should be evicted to make room. This will prevent caches from growing indefinitely while still ensuring a high cache hit rate.

---

This bug was first discovered in the Sharkey fork, but exists in Misskey as well. For additional details, see Sharkey issue #600 and Sharkey MR #580.

Related to #14311.

🥰 Expected Behavior

Caches should have a limited size and reasonable eviction method to prevent infinite memory usage.

🤬 Actual Behavior

Memory caches are unbounded and consume infinite server memory.

📝 Steps to Reproduce

Run a federated instance for an extended period of time and observe memory usage. Background memory will increase indefinitely as caches are filled with remote users, public keys, and other data.

💻 Frontend Environment

N/A - this is a backend issue. The frontend environment does not matter.

🛰 Backend Environment (for server admin)

N/A - this effects all instances running a Misskey version newer than March 2021. The backend environment is irrelevant.

Do you want to address this bug yourself?

• Yes, I will patch the bug myself and send a pull request

[kakkokari-gtyih]

日本語訳

一部のメモリー・キャッシュにはキャッシュ削除のポリシーが存在せず、アクティブなプロセスごとに何百万ものエントリーを含むまでに膨れ上がる可能性がある
これは単に非効率的だというだけでなく、長い時間をかけてゆっくりと進行するメモリリークの原因ともなりうる（→ #10984 ）

特に、これらのキャッシュには削除までの時間としてInfinityが指定されている

• User オブジェクト（このキャッシュは、異なるキーでさらに4回複製される）
• 公開鍵（このキャッシュも別のキーで複製される）
• クライアント・アプリのメタデータ

解決策として、すべての"永続的な"キャッシュに対して、合理的なTTLを定義することを提案する。さらに、いくつかのキャッシュには、エントリーの最大数を指定できるようにするべき。そのサイズを超えると、最も古いエントリーが削除されるという仕組み。この後者の変更は、公開鍵とクライアント・アプリのキャッシュに最も効果的だろう。

[warriordog]

I plan to port the patch from Sharkey and submit it here, but I won't be able to exhaustively test it. This behavior only appears when an instance is highly federated and receives lots of activity, and I can't replicate that in a testing environment.

Just to clarify: I will test the Sharkey version on my own instance. It's specifically the upstreamed Misskey patch that I can't test.