Turn fil-deal-ingester into service

[juliangruber]

Instead of manually running the script, automate it in a service. This way we can also increase the ingestion frequency.

Add monitoring:

• no data point received
• an error happened during processing
• added below threshold
• deleted above threshold

[pyropy]

This could potentially be automated using github actions.

[juliangruber]

Great idea!

[pyropy]

There seems to be a limit on duration and storage for Github runners (in our requirements we state that job between 3 - 5hrs and could take up to 40GB of space). We could self-host Github runners to mitigate this problem.

[pyropy]

There seems to be a limit on duration and storage for Github runners (in our requirements we state that job between 3 - 5hrs and could take up to 40GB of space). We could self-host Github runners to mitigate this problem.

Simpler version would be just to run scheduled machines on fly.io

[juliangruber]

@pyropy and me discussed this and agreed that this would be nice to add to M4.12, since it's a quick win and will remove an annoying manual task

[bajtos]

It would be great if the new automated process could post a message to Slack with the summary. I think that's the very minimum we need for observability - if we see the message, then we know all is good; if we don't then we know we need to investigate.

**FINISHED INGESTION OF f05 DEALS**
Deleted: 258774
Added: 323142
Updated allocator info for 9 clients

[juliangruber]

Good idea to make this visible. Posting to Slack is one option, tracking in Influx, with Grafana alerts, is another, if we know how to set the alerts.