Filter data sources based on partitions and clustering where possible

[scholtzan]

A significant cost saving can be achieved by querying data sources only for relevant partitions or clusters. One such example is to query main_v4 only for nightly data for every experiment that is run on nightly. Even more cost can be saved when querying events and making sure only event_categorys with relevant data get queried.

Currently, these optimizations need to be made manually in custom configs. Normal users are not familiar with this, so it would be good if there was some kind of automated or more guided way that could be provided.

Cost savings are quite significant here (often cost can be cut by up to 10x)

┆Issue is synchronized with this Jira Task

[ncalexan]

I will emphasize this for events: a custom data source with the event_category filter is orders of magnitude more efficient than the existing events datasource. See, e.g., mozilla/metric-hub@a4f3625.

What I'd like to see is some thinking about how the data source TOML can accommodate this. We're hitting a BigQuery pessimization where-in the order of the filters matters. Can we "parameterize" data sources, so that using the events datasource requires something like events('event_category') or similar? Can we deprecate events entirely, so that it'll be more clear to custom analysis writers that the custom data source pattern is the way to go? (When doing this myself, I discovered the pattern "by hand"; the existing code base is actively misleading because it doesn't do this.)