Signals: Filtering and Formatting Source Payloads into Non-Urgent Messages

[andrewbenington]

What problem would you like to solve? Please describe:
Target would like to add a new feature to GoAlert that will:

• allow any data input from a source
• accept or discard that input based on predefined filters
• format the data and send it to a destination as a non-urgent message

Describe the solution you'd like:
Signals will be an Alert equivalent for non-urgent events.

A Signal differs from an Alert in that:

• It is stateless (no ack/clear)
• It is not urgent and does not require immediate action
  • While tied to a service, it simply sends a message to a shared channel
  • It does not start an escalation

To filter and format the incoming requests appropriately, each Service will have a set of Rules defined by the user. These rules include:

• A name
• A set of integration keys whose requests it will apply to
• A set of filters
  • If an incoming request doesn't satisfy the filters, it will be discarded and no action will be taken
  • Filters will be based on expr expressions
• A set of actions
  • For example, an action might be "send a Slack message to #Channel1" or "send an email to my.team@example.com"
  • Each action will have a custom outgoing payload configured by the user, with the ability to templatize messages with fields from the request body
  • If an incoming request satisfies a rule's filters, a Signal will be created for each of that rule's actions

A Signal is tied to a single ServiceRule, and has a destination channel and an outgoing payload whose structure depends on the channel.

The Signal engine should handle message destinations using a plugin implementation as detailed in #2639

To avoid swamping GoAlert with Signals, throttling should be implemented on a per-rule basis:

• A rule will automatically filter out an incoming request if that rule has created a Signal in the last X minutes (time period to be decided)

Similar to the history logs on an Alert page, each Service will have a Signals page with a history of the Signals sent for that service.

Development Strategy:

Because of the scope of this feature, a separate branch (go-signal) will be the target of pull requests until the feature is ready to add to the master branch with an experimental flag.

Tasks:

As these tasks are completed and merged into the go-signal, mark them as complete.

• Add database tables and Stores for Signals and Service Rules
• Add a signal/incoming http endpoint to the API that uses Service Rule filters
• Update GraphQL schema with Signal and Service Rule queries and data types
• Spike: Decide on plugin implementation as detailed in notification: plug-able providers #2639
  • Create Signals engine for sending messages
• Add logic to convert filters between expr string and frontend/user facing structs
• Implement rule-level signal throttling
• UI: Add Service-level Signals page
• UI: Add Service-level Rules page and Rule create/edit form #3398