Make index counts configurable and disable by default

[masih]

Index count is enabled by default across both ingestion and find server. While we look into how to do it without extensive lock contention make the feature configurable and disable it by default on both ingest and find server.

Relates to #2062

[codecov-commenter]

Codecov Report

Patch coverage has no change and project coverage change: -0.33 ⚠️

Comparison is base (860860c) 47.95% compared to head (e9e0b28) 47.62%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2070      +/-   ##
==========================================
- Coverage   47.95%   47.62%   -0.33%     
==========================================
  Files          94       92       -2     
  Lines       10306    10266      -40     
==========================================
- Hits         4942     4889      -53     
- Misses       4790     4810      +20     
+ Partials      574      567       -7     

Impacted Files	Coverage Δ
command/daemon.go	0.00% <0.00%> (ø)
config/finder.go	42.85% <ø> (ø)
config/ingest.go	35.00% <ø> (ø)

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[gammazero]

Maybe we just get rid of these counters, if we are going to support unique multihash counts in database. Persisting this is already expensive in a local datastore, because we keep counts per context ID.

[masih]

Maybe we just get rid of these counters, if we are going to support unique multihash counts in database. Persisting this is already expensive in a local datastore, because we keep counts per context ID.

I think from visibility perspective we want both, i.e. globally unique CID count, and unique index record count. The former tells us the number of destinations and the latter tells us how many ways there are to get there. Having said that, if we have to pick the former is perhaps more important to know.

Considering the work on measuring the latter, it's worthwhile keeping this capability but optimising it such that it has minimal impact on ingest rate I think.

[gammazero]

One thing that we can do to reduce read contention is to move this out of the info reported providers info, and make it require a separate request or only use the counters to update metrics.

[gammazero]

Lets consider removing the counters from the provider info to reduce read contention. Or, at least remove from the single provider read which is used to populate find results.