Proposal: Automatic caching of condition functions

[csadorf]

Feature description

We are currently operating under the assumption that the project data space does not change while we are gathering the project status.

We should take advantage of that assumption and automatically cache all condition functions.

Proposed solution

Caching should be done on a per-instance level, but should be parallelizable.

[vyasr]

@csadorf what was your idea here? Is the idea that if a given condition function decorates multiple operations, then we could reuse cached results? We still definitely need to start from scratch again every time that _fetch_status is called, right?

[csadorf]

Yes, the same condition function is evaluated potentially many times during one status update. That could be automatically cashed. I think that we assume "no changes" to the data space during _fetch_status() so that would be the appropriate scope.

[b-butler]

In addition, often labels are used as condition functions. This would prevent the calculating of labels more than once as well.

[vyasr]

I think that various internal variables are named somewhat ambiguously because in earlier versions of signac there was less conceptual separation between labels and conditions (basically before the decorator syntax). If we were to handle this with respect to both labels and conditions, it would probably be helpful to clearly separate these two objects in the code.

[b-butler]

We could definitely do that, but to see the most performance gains we would need to identify both conditions and labels by some means that would tell us when the underlying function is the same.