You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We need this for blobs but the same mechanism will also be used for cleaning up stale views for any document types. So it's worth discussing this in general, not only refering to blobs.
This issue only relates to garbage collecting document views, not entries/operations/documents, this is slightly different concern and should be discussed elsewhere.
We want a garbage collection service to remove views which are no longer required. A view is no longer required when both of the following are true:
they aren't the "current view" of any document
they aren't the target of a pinned relation from another document
I can see two ways to implement garbage collection:
a stand-alone service which periodically removes any documents from the database which match the above conditions
a "garbage_collection" task in the materialization service which is issued whenever a "reduce" task completed. The input for the task would be a specific document, the task itself would check for the above conditions and clean up the old views for that document, and likely issue more "garbage_collection" tasks for any pinned relations which may now also need garbage collecting
I like both approaches, the second is more matching our current materialisation task patterns, but the first seems less complex in some ways. Don't know which would be more computationally efficient....
One potential challenge I see is dealing with cyclical relations eg. A1 -> B1 -> A1 if these were all document views containing pinned relations, they would never be garbage collected even if they weren't wanted anymore.
The text was updated successfully, but these errors were encountered:
We need this for
blobs
but the same mechanism will also be used for cleaning up stale views for any document types. So it's worth discussing this in general, not only refering to blobs.This issue only relates to garbage collecting document views, not entries/operations/documents, this is slightly different concern and should be discussed elsewhere.
We want a garbage collection service to remove views which are no longer required. A view is no longer required when both of the following are true:
I can see two ways to implement garbage collection:
materialization
service which is issued whenever a "reduce" task completed. The input for the task would be a specific document, the task itself would check for the above conditions and clean up the old views for that document, and likely issue more "garbage_collection" tasks for any pinned relations which may now also need garbage collectingI like both approaches, the second is more matching our current materialisation task patterns, but the first seems less complex in some ways. Don't know which would be more computationally efficient....
One potential challenge I see is dealing with cyclical relations eg.
A1 -> B1 -> A1
if these were all document views containing pinned relations, they would never be garbage collected even if they weren't wanted anymore.The text was updated successfully, but these errors were encountered: