Ingestion Filtering by Account

[sreuland]

What problem does your feature solve?

Horizon ingestion of full ledger space from core can take up massive amounts of disk space on db.

What would you like to see?

As a service provider using my own installation of Horizon, I want to track transactions and operations on my customer accounts that have been onboarded to my service.

I want to configure which accounts trigger related transaction and operation info to be ingested by my Horizon installation and persisted to my horizon historical database.

I don't want my horizon historical database to contain data related to other accounts, so, I expect to see my horizon db size on disk remain relatively small with a slower growth rate compared to no-filtering(i.e. data for entire ledger was saved)

Acceptance Criteria:

• Horizon provides an Account Filter rule configuration API endpoint on the secure Horizon Admin port to perform CRUD on the account whitelist rule model.
• Design the account whitelist rule model, likely a list/array of strings representing whitelisted account ids.
• All Horizon ingestion processes(current-forward/reingest) run filtration, apply the account whitelist rules and result in only persisting historical data that passes the filter.
• The account filter effectively applies latest config rules to current and ongoing ingestion activity which occurs after a filter rule is updated.
• The Horizon Account Filter feature can be configured as a toggle/switch, enabled or disabled.
• The Horizon Account Filter feature should be available as early/alpha drop builds to some willing early-adopter partners for trial usage.
• Need to choose feature branch or mainline development(i.e. merged to master often).

What alternatives are there?