[ResponseOps] URLs longer than 4K bytes are split in Slack and email messages

[pmuellr]

I did a test with the Slack connector, sending http://example.com/app/r?l=DISCOVER_APP_LOCATOR&v=8.7.1&lz=1234567890 with the 1..0 numbers repeated 400 times. Total of 4060 bytes. This is from a support case, for a use of {{context.link}} variable from an elasticsearch query rule in an action.

Slack split it into 3992 bytes in the first message, remainder in the second. We've seen that Slack has a 40K message limit, but this is below that. It's possible the 40K limit is the total limit of the message, but that there's a ~4K limit on line lengths, or something. I didn't see any mention of possible line length limits.

This was a change from 8.6.2 to 8.7.1, for one user's rule we looked at, where the URL went from 254 bytes to 1072 bytes. The same customer hit the "split line" problem with a URL > 4000 bytes.

Seems like the change to the URL was made here: #146403

Generating links in the "saved objects" format will result in much smaller URLs, however that also means each of those queries will be persisted in an index, forever.

We'll need to figure out some story for these ...

[elasticmachine]

Pinging @elastic/response-ops (Team:ResponseOps)

[pmuellr]

We're now seeing some issues with emails as well. One email we saw was truncated at 2048 bytes. Not out of the question that broke some email processor's url-length-limit, or something, especially in an email.

[elasticmachine]

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

[pmuellr]

Added the data discovery team, as we will eventually have to depend on them for a solution here.

We obviously know the lengths of these URLs when we create them, so it seems like we can do something at the alerting side - when we actually know a mustache variable is a URL - but generally only rule types know what their mustache variables are. So, if it's less than some limit (say 1K), continue writing out the zipped-uuencoded-JSON version as a query string version of the URL, and if it's not ... something else

We could try making these "short URLs" by using the existing SO's where short links get saved. But since these will just pile up over time, perhaps we could put a configurable TTL on them - two weeks default or so.

[kertal]

We could try making these "short URLs" by using the existing SO's where short links get saved. But since these will just pile up over time, perhaps we could put a configurable TTL on them - two weeks default or so.

yes I think TTL for such URLs would be a nice feature FYI @elastic/appex-sharedux
@jughosta is working on a quick fix that will bring help to cases when the URL gets very long due to data views with lots of field attributes.
@pmuellr like you mention we could generate only generate short URLs when the length of a serialized encoded exceeds a certain limit, then the link would work in any case

[mikecote]

@jughosta is working on a quick fix that will bring help to cases when the URL gets very long due to data views with lots of field attributes.

@kertal do you think the quick fix will be sufficient to close this GitHub issue afterwards? I can link the PR as resolving this issue or do you think there is leftover work that should be tracked with this issue?

[kertal]

@jughosta is working on a quick fix that will bring help to cases when the URL gets very long due to data views with lots of field attributes.

@kertal do you think the quick fix will be sufficient to close this GitHub issue afterwards? I can link the PR as resolving this issue or do you think there is leftover work that should be tracked with this issue?

The PR improves the situation by shortening the alert link. The given cases were triggered by lots of fields with field counts being part of the link. However, it can be still be the case to have very long links. We intend to further improve by migrating persisted data view containing links to no longer be migrated to temporary data views (which contain the spec)

Also with this change, there can be long URL, given users can still use temporary data view.

To completely resolve it we would need to move to short URLs, but this could cause a lot of redundant saved objects, so it would need e.g. functionality to cleanup links with a TTL. However this is a big multi team effort (but on the long run, I think it would be beneficial)

[kertal]

Here's another issue to track which will improve the link length : #158729

[pmuellr]

Here's another issue to track which will improve the link length : #158729

Ah, I bet this will save TONs of space, should be great.

We may want to enhance our docs to note that if using temporary data views, the URLs may be too long, and so you should use persistent data views. Or I wonder if we could check that and perhaps provide a different link - like to the rule details or something ...

[kertal]

I've opened another issue for a long term solution: #158734