[RAC] Persistent timeline fields fix #110685

semd · 2021-08-31T18:08:43Z

Summary

Solves the issue: #110524

Fixes the local storage timeline, it had a circular dependency inside the filterManager field, causing an error when trying to JSON.stringify in the local storage util module. The field has been cleaned, together with some other fields that we do not need to store.

To test:

Open a new private window and go to /app/security and then:

Open Alerts table on the Detections.
Change any field using the field modal.
Check the timeline key is saved in the localStorage (Application/LocalStorage section in chrome dev tools).
Refresh the page and check the field change has persisted.

Repeat the same 1-5 steps for:

Alerts table on the Rule Details page
Events and External alerts tables on the Host Details page
Events and External alerts tables on the Host Page
External alerts on the Network page

elasticmachine · 2021-08-31T18:08:45Z

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)

elasticmachine · 2021-08-31T18:08:45Z

Pinging @elastic/security-solution (Team: SecuritySolution)

XavierM · 2021-09-01T01:21:34Z

x-pack/plugins/security_solution/public/timelines/containers/local_storage/index.tsx

  });
 };

+const cleanStorageTimeline = (timeline: TimelineModel) => {
+  // clean filterManager as it contains circular references within observer subscriptions
+  const { filterManager, ...timelineToStore } = timeline;


I think that we need to remove all of these attributes to be like it was in 7.13. The attribute defaultModel is now defaultColumn, we simplified that.

https://github.com/elastic/kibana/blob/7.13/x-pack/plugins/security_solution/public/timelines/components/manage_timeline/index.tsx#L31-L43

…et_localstorage_fix

XavierM · 2021-09-01T16:36:33Z

x-pack/plugins/security_solution/public/timelines/containers/local_storage/index.tsx

+const cleanStorageTimeline = (timeline: TimelineModel) => {
+  // discard unneeded fields to make sure the object serialization works
+  const {
+    documentType,


i think we forget defaultColumn or maybe you wanted this way.

I did try removing the defaultColumns (I guess this one is the one you mean), but doing that it started to appear problems on the Reset Fields action.
There are several hydration points where this problem would need to be fixed, so I kept the field in the storage as it is the esasiest way to prevent the bug, but we can take another look later on.

kibanamachine · 2021-09-02T13:09:49Z

💛 Build succeeded, but was flaky

continuous-integration/kibana-ci/pull-request
Commit: 708fb9e
Storybooks Preview
Flaky suites:
- xpack-kibana-ciGroup5

Test Failures

Kibana Pipeline / general / X-Pack Alerting API Integration Tests.x-pack/test/alerting_api_integration/security_and_spaces/tests/alerting/find·ts.alerting api integration security and spaces enabled Alerts alerts find "after each" hook for "should filter out types that the user is not authorized to `get` retaining pagination"

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]       │
[00:00:00]         └-: alerting api integration security and spaces enabled
[00:00:00]           └-> "before all" hook in "alerting api integration security and spaces enabled"
[00:03:59]           └-: Alerts
[00:03:59]             └-> "before all" hook in "Alerts"
[00:05:28]             └-: alerts
[00:05:28]               └-> "before all" hook in "alerts"
[00:05:28]               └-> "before all" hook in "alerts"
[00:05:28]                 │ debg creating space
[00:05:29]                 │ debg created space
[00:05:29]                 │ debg creating space
[00:05:30]                 │ debg created space
[00:05:30]                 │ debg creating space
[00:05:31]                 │ debg created space
[00:05:31]                 │ debg creating user no_kibana_privileges
[00:05:31]                 │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [no_kibana_privileges]
[00:05:31]                 │ debg created user no_kibana_privileges
[00:05:31]                 │ debg creating role no_kibana_privileges
[00:05:31]                 │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [no_kibana_privileges]
[00:05:31]                 │ debg creating user superuser
[00:05:31]                 │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [superuser]
[00:05:31]                 │ debg created user superuser
[00:05:31]                 │ debg creating user global_read
[00:05:31]                 │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [global_read]
[00:05:31]                 │ debg created user global_read
[00:05:31]                 │ debg creating role global_read_role
[00:05:31]                 │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [global_read_role]
[00:05:31]                 │ debg creating user space_1_all
[00:05:31]                 │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [space_1_all]
[00:05:31]                 │ debg created user space_1_all
[00:05:31]                 │ debg creating role space_1_all_role
[00:05:31]                 │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [space_1_all_role]
[00:05:31]                 │ debg creating user space_1_all_with_restricted_fixture
[00:05:31]                 │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [space_1_all_with_restricted_fixture]
[00:05:31]                 │ debg created user space_1_all_with_restricted_fixture
[00:05:31]                 │ debg creating role space_1_all_with_restricted_fixture_role
[00:05:31]                 │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [space_1_all_with_restricted_fixture_role]
[00:05:31]                 │ debg creating user space_1_all_alerts_none_actions
[00:05:31]                 │ info [o.e.x.s.a.u.TransportPutUserAction] [node-01] added user [space_1_all_alerts_none_actions]
[00:05:31]                 │ debg created user space_1_all_alerts_none_actions
[00:05:31]                 │ debg creating role space_1_all_alerts_none_actions_role
[00:05:32]                 │ info [o.e.x.s.a.r.TransportPutRoleAction] [node-01] added role [space_1_all_alerts_none_actions_role]
[00:05:32]               └-: find
[00:05:32]                 └-> "before all" hook in "find"

Stack Trace

Error: expected 204 "No Content", got 409 "Conflict"
    at Test._assertStatus (/dev/shm/workspace/parallel/7/kibana/node_modules/supertest/lib/test.js:268:12)
    at Test._assertFunction (/dev/shm/workspace/parallel/7/kibana/node_modules/supertest/lib/test.js:283:11)
    at Test.assert (/dev/shm/workspace/parallel/7/kibana/node_modules/supertest/lib/test.js:173:18)
    at assert (/dev/shm/workspace/parallel/7/kibana/node_modules/supertest/lib/test.js:131:12)
    at /dev/shm/workspace/parallel/7/kibana/node_modules/supertest/lib/test.js:128:5
    at Test.Request.callback (/dev/shm/workspace/parallel/7/kibana/node_modules/supertest/node_modules/superagent/lib/node/index.js:718:3)
    at /dev/shm/workspace/parallel/7/kibana/node_modules/supertest/node_modules/superagent/lib/node/index.js:906:18
    at IncomingMessage.<anonymous> (/dev/shm/workspace/parallel/7/kibana/node_modules/supertest/node_modules/superagent/lib/node/parsers/json.js:19:7)
    at endReadableNT (internal/streams/readable.js:1317:12)
    at processTicksAndRejections (internal/process/task_queues.js:82:21)

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`securitySolution`	6.5MB	6.5MB	+341.0B

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @semd

machadoum

I tested it and it is working fine!

* fix stringify circular ref crash and default columns on createTimeline * rollback reset buton fix to split PR * adding fields to the storage cleaning * tests fixed * test fix

kibanamachine · 2021-09-02T14:35:59Z

💚 Backport successful

Status	Branch	Result
✅	7.15
✅	7.x

The backport PRs will be merged automatically after passing CI.

…eporting-to-v2 * 'master' of github.com:elastic/kibana: (65 commits) Move to vis_types folder part 2 (elastic#110574) [SOR] use initialNamespaces when checking for conflict for `create` and `bulkCreate` (elastic#111023) [Discover] Remove export* syntax (elastic#110934) [Event log][7.x] Updated event log client to search across legacy IDs (elastic#109365) [Security Solution][Detection Rules] Changes 'activated' text on rule details page (elastic#111044) [Metrics UI] Filter out APM nodes from the inventory view (elastic#110300) [package testing] Update logging and pid configuration (elastic#111059) [Dashboard] Read App State from URL on Soft Refresh (elastic#109354) Add correct roles to test user for functional tests in dashboard (elastic#110880) [DOCS] Adds Lens Inspector and minor edits (elastic#109736) [DOCS] Updates Spaces page (elastic#111005) normalize initialNamespaces (elastic#110936) [Reporting] Clean up `any` usage, reorganize server route files (elastic#110740) [Security Solution] [CTI] Fixes bug that caused Threshold and Indicator Match rules to ignore custom rule filters if a saved query was used in the rule definition. (elastic#109253) skip flaky suites: elastic#111001, elastic#111022 [Security Solution][RAC] - Update reason field text (elastic#110308) [RAC][Security Solution] Make analyzer work with EuiDataGrid full screen (elastic#110913) [Metrics UI] Add integration tests for Metric Threshold Rule and refactor to fire correctly (elastic#109971) [DOCS] Updates Discover docs (elastic#110346) [RAC] Persistent timeline fields fix (elastic#110685) ...

* fix stringify circular ref crash and default columns on createTimeline * rollback reset buton fix to split PR * adding fields to the storage cleaning * tests fixed * test fix Co-authored-by: Sergi Massaneda <sergi.massaneda@elastic.co>

* [RAC] Persistent timeline fields fix (#110685) * fix stringify circular ref crash and default columns on createTimeline * rollback reset buton fix to split PR * adding fields to the storage cleaning * tests fixed * test fix * Skip cypress test Co-authored-by: Sergi Massaneda <sergi.massaneda@elastic.co> Co-authored-by: Christos Nasikas <christos.nasikas@elastic.co>

* fix stringify circular ref crash and default columns on createTimeline * rollback reset buton fix to split PR * adding fields to the storage cleaning * tests fixed * test fix # Conflicts: # x-pack/plugins/security_solution/public/timelines/store/timeline/defaults.ts # x-pack/plugins/timelines/public/store/t_grid/model.ts

* [RAC] Persistent timeline fields fix (#110685) * fix stringify circular ref crash and default columns on createTimeline * rollback reset buton fix to split PR * adding fields to the storage cleaning * tests fixed * test fix # Conflicts: # x-pack/plugins/security_solution/public/timelines/store/timeline/defaults.ts # x-pack/plugins/timelines/public/store/t_grid/model.ts * fix types and tests

arvindersingh-qasource-zz · 2021-09-16T10:30:45Z

Hi @semd

We have validated this ticket on 7.15 BC5 build and found that issue is now FIXED

Please find the below observations

Build Details

Version : 7.15.0 BC5
Commit : 0239ff6864dd9930cfe9bcd9a679272f2b7465c2
Build : 43957

Timelines on Dev tools for Chrome Browser

Alerts table on the Detections
Alerts table on the Rule Details page
Events and External alerts tables on the Host Details page
Events and External alerts tables on the Host Page
External alerts on the Network page

Thanks.

semd · 2021-09-16T12:05:28Z

Thank you for the accurate validation @arvindersingh-qasource,
This fix has been also backported to 7.14.2 two days ago, could you please check there is working as well? Thanks

arvindersingh-qasource-zz · 2021-09-16T12:34:34Z

Hi @semd

We have validated this ticket on 7.14.2 BC1 build and found that issue is now FIXED

Please find the below observations

Build Details

Version : 7.14.2 BC1
Commit : 6b971d6410ca4a99271f2abf9d3de68b7a3653a5
Build : 42959

Timelines on Dev tools for Chrome Browser

Alerts table on the Detections
Alerts table on the Rule Details page
Events and External alerts tables on the Host Details page
Events and External alerts tables on the Host Page
External alerts on the Network page

Thanks.

semd · 2021-09-16T12:36:18Z

@arvindersingh-qasource thank you!

@timestamp

…lastic#113090>, as proposed [here](elastic#113090 (comment)): - Configures the `Columns` popover to be consistent with `Discover` - Changes the `Hide column` action to `Remove column`, to be consistent with `Discover` - Persists updates to the `Columns` popover order in `local storage` - Restores the feature to persist column widths in `local storage` - We now pass `false` to the `allowHide` [EuiDataGrid API](https://elastic.github.io/eui/#/tabular-content/data-grid): ![allow_hide](https://user-images.githubusercontent.com/4459398/136114714-02f25b97-86af-47e5-9adc-1177d5a2c715.png) This makes all `EuiDataGrid`-based views in the Security Solution consistent with `Discover`'s use of the `EuiDataGrid` `Columns` popover. In `7.15`, the `Columns` popover includes the _hide column_ toggle, as shown in the screenshot below: ![alerts_columns_popover_7_15](https://user-images.githubusercontent.com/4459398/136112441-455ddbeb-dea3-4837-81ad-32d6c82c11fe.png) _Above: The `Columns` popover in the `7.15` `Alerts` table_ The `Columns` popover in `Discover`'s `EuiDataGrid`-based table does not display the hide column toggle, as shown the screenshot below: ![columns_popover_discover](https://user-images.githubusercontent.com/4459398/136112856-7e42c822-2260-4759-ac78-5bea63a171c7.png) _Above: The `EuiDataGrid` `Columns` popover in `Discover`, in `master`_ Passing `false` to the `allowHide` [EuiDataGrid API](https://elastic.github.io/eui/#/tabular-content/data-grid) API makes the `Columns` popover in all `EuiDataGrid`-based views in the Security Solution consistent with `Discover`, as illustrated by the screenshot below: ![alerts_columns_popover_no_hide](https://user-images.githubusercontent.com/4459398/136112980-d4219fbd-1443-4612-8cdb-b97bee8b97ef.png) _Above: The `Columns` popover is now consistent with `Discover`_ - The `Hide column` action shown in the `7.15` alerts table is changed to `Remove column`, making it consistent with `Discover`'s use of `EuiDataGrid` In `7.15`, the `Alerts` table has a `Hide column` action, as shown in the screenshot below: ![hide_column](https://user-images.githubusercontent.com/4459398/136115681-9e0da144-a981-4352-8092-9368d74cd153.png) _Above: The `Hide Column` action in the `7.15` `Alerts` table_ In `7.15`, clicking the `Hide Column` action shown in the screenshot above hides the column, but does not remove it. In `7.15`, columns may only be removed by un-checking them in the `Fields` browser, or by un-toggling them in the Alerts / Events details popover. Both of those methods require multiple clicks, and require uses to re-find the field in the modal or popover before it may be toggled for removal. In `Discover`, users don't hide columns. In `Discover`, users directly remove columns by clicking the `Remove column` action, shown in the screenshot below: ![discover_remove_column](https://user-images.githubusercontent.com/4459398/136114295-f018a561-f9ee-4ce4-a9c6-0fcd7f71e67b.png) _Above: The `Remove column` action in `Discover`'s use of `EuiDataGrid` in `master`_ All `EuiDataGrid`-based views in the Security Solution were made consistent with `Discover` by replacing the `Hide column` action with `Remove column`, per the screenshot below: ![remove_column_after](https://user-images.githubusercontent.com/4459398/137047582-3c4d6cb0-ac12-4c50-9c34-0c4ef5536550.png) _Above: The `Remove column` action in the Alerts table_ Note: the `Remove column` action shown above appears as the last item in the popover because it's specified via the `EuiDataGrid` `EuiDataGridColumnActions` > `additonal` API, which appends additonal actions to the end of popover, after the built-in actions: ![additional](https://user-images.githubusercontent.com/4459398/137047825-625002b3-5cd6-4b3e-87da-e76dbaf2a827.png) - Persist column order updates to `local storage` when users update the order of columns via the `Columns` popover The following PR <elastic#110685> restored partial support for persisting columns across page refreshes via `local storage`, but the Redux store was not updated when users sort columns via the `Columns` popover, an shown in the animated gif below: ![ordering_via_columns](https://user-images.githubusercontent.com/4459398/136119497-65f76f49-091c-4a45-b8d3-1e5ef80ccbb2.gif) _Above: Ordering via the `Columns` popover is not persisted to `local storage` in `7.15`_ This PR utilizes the `setVisibleColumns` [EuiDataGrid API](https://elastic.github.io/eui/#/tabular-content/data-grid) API as a callback to update Redux when the columns are sorted, which will in-turn update `local storage` to persist the new order across page refreshes: ![setVisibleColumns](https://user-images.githubusercontent.com/4459398/136117249-628bb147-a860-4ccf-811a-0e57a99296fb.png) In previous releases, resized column widths were peristed in `local storage` to persist across page refreshes, as documented in <elastic#110524> : ``` { "detections-page":{ "id":"detections-page", "activeTab":"query", "prevActiveTab":"query", "columns":[ { "category":"base", "columnHeaderType":"not-filtered", "description":"Date/time when the event originated. This is the date/time extracted from the event, typically representing when the event was generated by the source. If the event source has no original timestamp, this value is typically populated by the first time the event was received by the pipeline. Required field for all events.", "example":"2016-05-23T08:05:34.853Z", "id":"@timestamp", "type":"date", "aggregatable":true, "width":190 }, { "category":"cloud", "columnHeaderType":"not-filtered", "description":"The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.", "example":"666777888999", "id":"cloud.account.id", "type":"string", "aggregatable":true, "width":180 }, { "category":"cloud", "columnHeaderType":"not-filtered", "description":"Availability zone in which this host is running.", "example":"us-east-1c", "id":"cloud.availability_zone", "type":"string", "aggregatable":true, "width":180 }, // ... } ], // ... } } ``` _Above: column widths were persisted to `local storage` in previous release, (going at least back to `7.12`)_ In this PR, we utilize the `onColumnResize` [EuiDataGrid API](https://elastic.github.io/eui/#/tabular-content/data-grid) API as a callback to update Redux when the columns are sorted via the `Columns` popover. Updating Redux will in-turn update `local storage`, so resized columns widths will persist across page refreshes: ![onColumnResize](https://user-images.githubusercontent.com/4459398/136120062-3b0bebce-9c44-47fc-9956-48fe07a30f83.png) The Alerts page `Trend` chart and table were updated to include the following additional `Stack by` fields (CC @paulewing): ``` process.name file.name hash.sha256 ``` per the before / after screenshots below: ![alerts-trend-before](https://user-images.githubusercontent.com/4459398/137045011-7da4530b-0259-4fd4-b903-9eee6c26d02f.png) _Above: The Alerts `Trend` Stack by fields in `7.15` (before)_ ![alerts-trend-after](https://user-images.githubusercontent.com/4459398/137045023-d0ae987c-a474-4123-a05b-a6ad2fc52922.png) _Above: The Alerts `Trend` `Stack by` fields (after the addition of the `process.name`, `file.name`, and `hash.sha256` fields)_ CC: @monina-n @paulewing

@timestamp

…nd the Remove Column action (#114742) ## [Security Solution] Restores Alerts table local storage persistence and the Remove Column action This PR implements the following changes summarized below to address <#113090>, as proposed [here](#113090 (comment)): - Configures the `Columns` popover to be consistent with `Discover` - Changes the `Hide column` action to `Remove column`, to be consistent with `Discover` - Persists updates to the `Columns` popover order in `local storage` - Restores the feature to persist column widths in `local storage` ### Configures the `Columns` popover to be consistent with `Discover` - We now pass `false` to the `allowHide` [EuiDataGrid API](https://elastic.github.io/eui/#/tabular-content/data-grid): ![allow_hide](https://user-images.githubusercontent.com/4459398/136114714-02f25b97-86af-47e5-9adc-1177d5a2c715.png) This makes all `EuiDataGrid`-based views in the Security Solution consistent with `Discover`'s use of the `EuiDataGrid` `Columns` popover. In `7.15`, the `Columns` popover includes the _hide column_ toggle, as shown in the screenshot below: ![alerts_columns_popover_7_15](https://user-images.githubusercontent.com/4459398/136112441-455ddbeb-dea3-4837-81ad-32d6c82c11fe.png) _Above: The `Columns` popover in the `7.15` `Alerts` table_ The `Columns` popover in `Discover`'s `EuiDataGrid`-based table does not display the hide column toggle, as shown the screenshot below: ![columns_popover_discover](https://user-images.githubusercontent.com/4459398/136112856-7e42c822-2260-4759-ac78-5bea63a171c7.png) _Above: The `EuiDataGrid` `Columns` popover in `Discover`, in `master`_ Passing `false` to the `allowHide` [EuiDataGrid API](https://elastic.github.io/eui/#/tabular-content/data-grid) API makes the `Columns` popover in all `EuiDataGrid`-based views in the Security Solution consistent with `Discover`, as illustrated by the screenshot below: ![alerts_columns_popover_no_hide](https://user-images.githubusercontent.com/4459398/136112980-d4219fbd-1443-4612-8cdb-b97bee8b97ef.png) _Above: The `Columns` popover is now consistent with `Discover`_ ## Changes the `Hide column` action to `Remove column`, to be consistent with `Discover` - The `Hide column` action shown in the `7.15` alerts table is changed to `Remove column`, making it consistent with `Discover`'s use of `EuiDataGrid` In `7.15`, the `Alerts` table has a `Hide column` action, as shown in the screenshot below: ![hide_column](https://user-images.githubusercontent.com/4459398/136115681-9e0da144-a981-4352-8092-9368d74cd153.png) _Above: The `Hide Column` action in the `7.15` `Alerts` table_ In `7.15`, clicking the `Hide Column` action shown in the screenshot above hides the column, but does not remove it. In `7.15`, columns may only be removed by un-checking them in the `Fields` browser, or by un-toggling them in the Alerts / Events details popover. Both of those methods require multiple clicks, and require uses to re-find the field in the modal or popover before it may be toggled for removal. In `Discover`, users don't hide columns. In `Discover`, users directly remove columns by clicking the `Remove column` action, shown in the screenshot below: ![discover_remove_column](https://user-images.githubusercontent.com/4459398/136114295-f018a561-f9ee-4ce4-a9c6-0fcd7f71e67b.png) _Above: The `Remove column` action in `Discover`'s use of `EuiDataGrid` in `master`_ All `EuiDataGrid`-based views in the Security Solution were made consistent with `Discover` by replacing the `Hide column` action with `Remove column`, per the screenshot below: ![remove_column_after](https://user-images.githubusercontent.com/4459398/137047582-3c4d6cb0-ac12-4c50-9c34-0c4ef5536550.png) _Above: The `Remove column` action in the Alerts table_ Note: the `Remove column` action shown above appears as the last item in the popover because it's specified via the `EuiDataGrid` `EuiDataGridColumnActions` > `additonal` API, which appends additonal actions to the end of popover, after the built-in actions: ![additional](https://user-images.githubusercontent.com/4459398/137047825-625002b3-5cd6-4b3e-87da-e76dbaf2a827.png) ## Persists updates to the `Columns` popover order in `local storage` - Persist column order updates to `local storage` when users update the order of columns via the `Columns` popover The following PR <#110685> restored partial support for persisting columns across page refreshes via `local storage`, but the Redux store was not updated when users sort columns via the `Columns` popover, an shown in the animated gif below: ![ordering_via_columns](https://user-images.githubusercontent.com/4459398/136119497-65f76f49-091c-4a45-b8d3-1e5ef80ccbb2.gif) _Above: Ordering via the `Columns` popover is not persisted to `local storage` in `7.15`_ This PR utilizes the `setVisibleColumns` [EuiDataGrid API](https://elastic.github.io/eui/#/tabular-content/data-grid) API as a callback to update Redux when the columns are sorted, which will in-turn update `local storage` to persist the new order across page refreshes: ![setVisibleColumns](https://user-images.githubusercontent.com/4459398/136117249-628bb147-a860-4ccf-811a-0e57a99296fb.png) ## Restores the feature to persist column widths in `local storage` In previous releases, resized column widths were peristed in `local storage` to persist across page refreshes, as documented in <#110524> : ``` { "detections-page":{ "id":"detections-page", "activeTab":"query", "prevActiveTab":"query", "columns":[ { "category":"base", "columnHeaderType":"not-filtered", "description":"Date/time when the event originated. This is the date/time extracted from the event, typically representing when the event was generated by the source. If the event source has no original timestamp, this value is typically populated by the first time the event was received by the pipeline. Required field for all events.", "example":"2016-05-23T08:05:34.853Z", "id":"@timestamp", "type":"date", "aggregatable":true, "width":190 }, { "category":"cloud", "columnHeaderType":"not-filtered", "description":"The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.", "example":"666777888999", "id":"cloud.account.id", "type":"string", "aggregatable":true, "width":180 }, { "category":"cloud", "columnHeaderType":"not-filtered", "description":"Availability zone in which this host is running.", "example":"us-east-1c", "id":"cloud.availability_zone", "type":"string", "aggregatable":true, "width":180 }, // ... } ], // ... } } ``` _Above: column widths were persisted to `local storage` in previous release, (going at least back to `7.12`)_ In this PR, we utilize the `onColumnResize` [EuiDataGrid API](https://elastic.github.io/eui/#/tabular-content/data-grid) API as a callback to update Redux when the columns are sorted via the `Columns` popover. Updating Redux will in-turn update `local storage`, so resized columns widths will persist across page refreshes: ![onColumnResize](https://user-images.githubusercontent.com/4459398/136120062-3b0bebce-9c44-47fc-9956-48fe07a30f83.png) ### Other changes The Alerts page `Trend` chart and table were updated to include the following additional `Stack by` fields (CC @paulewing): ``` process.name file.name hash.sha256 ``` per the before / after screenshots below: ![alerts-trend-before](https://user-images.githubusercontent.com/4459398/137045011-7da4530b-0259-4fd4-b903-9eee6c26d02f.png) _Above: The Alerts `Trend` Stack by fields in `7.15` (before)_ ![alerts-trend-after](https://user-images.githubusercontent.com/4459398/137045023-d0ae987c-a474-4123-a05b-a6ad2fc52922.png) _Above: The Alerts `Trend` `Stack by` fields (after the addition of the `process.name`, `file.name`, and `hash.sha256` fields)_ CC: @monina-n @paulewing

@timestamp

…nd the Remove Column action (elastic#114742) ## [Security Solution] Restores Alerts table local storage persistence and the Remove Column action This PR implements the following changes summarized below to address <elastic#113090>, as proposed [here](elastic#113090 (comment)): - Configures the `Columns` popover to be consistent with `Discover` - Changes the `Hide column` action to `Remove column`, to be consistent with `Discover` - Persists updates to the `Columns` popover order in `local storage` - Restores the feature to persist column widths in `local storage` ### Configures the `Columns` popover to be consistent with `Discover` - We now pass `false` to the `allowHide` [EuiDataGrid API](https://elastic.github.io/eui/#/tabular-content/data-grid): ![allow_hide](https://user-images.githubusercontent.com/4459398/136114714-02f25b97-86af-47e5-9adc-1177d5a2c715.png) This makes all `EuiDataGrid`-based views in the Security Solution consistent with `Discover`'s use of the `EuiDataGrid` `Columns` popover. In `7.15`, the `Columns` popover includes the _hide column_ toggle, as shown in the screenshot below: ![alerts_columns_popover_7_15](https://user-images.githubusercontent.com/4459398/136112441-455ddbeb-dea3-4837-81ad-32d6c82c11fe.png) _Above: The `Columns` popover in the `7.15` `Alerts` table_ The `Columns` popover in `Discover`'s `EuiDataGrid`-based table does not display the hide column toggle, as shown the screenshot below: ![columns_popover_discover](https://user-images.githubusercontent.com/4459398/136112856-7e42c822-2260-4759-ac78-5bea63a171c7.png) _Above: The `EuiDataGrid` `Columns` popover in `Discover`, in `master`_ Passing `false` to the `allowHide` [EuiDataGrid API](https://elastic.github.io/eui/#/tabular-content/data-grid) API makes the `Columns` popover in all `EuiDataGrid`-based views in the Security Solution consistent with `Discover`, as illustrated by the screenshot below: ![alerts_columns_popover_no_hide](https://user-images.githubusercontent.com/4459398/136112980-d4219fbd-1443-4612-8cdb-b97bee8b97ef.png) _Above: The `Columns` popover is now consistent with `Discover`_ ## Changes the `Hide column` action to `Remove column`, to be consistent with `Discover` - The `Hide column` action shown in the `7.15` alerts table is changed to `Remove column`, making it consistent with `Discover`'s use of `EuiDataGrid` In `7.15`, the `Alerts` table has a `Hide column` action, as shown in the screenshot below: ![hide_column](https://user-images.githubusercontent.com/4459398/136115681-9e0da144-a981-4352-8092-9368d74cd153.png) _Above: The `Hide Column` action in the `7.15` `Alerts` table_ In `7.15`, clicking the `Hide Column` action shown in the screenshot above hides the column, but does not remove it. In `7.15`, columns may only be removed by un-checking them in the `Fields` browser, or by un-toggling them in the Alerts / Events details popover. Both of those methods require multiple clicks, and require uses to re-find the field in the modal or popover before it may be toggled for removal. In `Discover`, users don't hide columns. In `Discover`, users directly remove columns by clicking the `Remove column` action, shown in the screenshot below: ![discover_remove_column](https://user-images.githubusercontent.com/4459398/136114295-f018a561-f9ee-4ce4-a9c6-0fcd7f71e67b.png) _Above: The `Remove column` action in `Discover`'s use of `EuiDataGrid` in `master`_ All `EuiDataGrid`-based views in the Security Solution were made consistent with `Discover` by replacing the `Hide column` action with `Remove column`, per the screenshot below: ![remove_column_after](https://user-images.githubusercontent.com/4459398/137047582-3c4d6cb0-ac12-4c50-9c34-0c4ef5536550.png) _Above: The `Remove column` action in the Alerts table_ Note: the `Remove column` action shown above appears as the last item in the popover because it's specified via the `EuiDataGrid` `EuiDataGridColumnActions` > `additonal` API, which appends additonal actions to the end of popover, after the built-in actions: ![additional](https://user-images.githubusercontent.com/4459398/137047825-625002b3-5cd6-4b3e-87da-e76dbaf2a827.png) ## Persists updates to the `Columns` popover order in `local storage` - Persist column order updates to `local storage` when users update the order of columns via the `Columns` popover The following PR <elastic#110685> restored partial support for persisting columns across page refreshes via `local storage`, but the Redux store was not updated when users sort columns via the `Columns` popover, an shown in the animated gif below: ![ordering_via_columns](https://user-images.githubusercontent.com/4459398/136119497-65f76f49-091c-4a45-b8d3-1e5ef80ccbb2.gif) _Above: Ordering via the `Columns` popover is not persisted to `local storage` in `7.15`_ This PR utilizes the `setVisibleColumns` [EuiDataGrid API](https://elastic.github.io/eui/#/tabular-content/data-grid) API as a callback to update Redux when the columns are sorted, which will in-turn update `local storage` to persist the new order across page refreshes: ![setVisibleColumns](https://user-images.githubusercontent.com/4459398/136117249-628bb147-a860-4ccf-811a-0e57a99296fb.png) ## Restores the feature to persist column widths in `local storage` In previous releases, resized column widths were peristed in `local storage` to persist across page refreshes, as documented in <elastic#110524> : ``` { "detections-page":{ "id":"detections-page", "activeTab":"query", "prevActiveTab":"query", "columns":[ { "category":"base", "columnHeaderType":"not-filtered", "description":"Date/time when the event originated. This is the date/time extracted from the event, typically representing when the event was generated by the source. If the event source has no original timestamp, this value is typically populated by the first time the event was received by the pipeline. Required field for all events.", "example":"2016-05-23T08:05:34.853Z", "id":"@timestamp", "type":"date", "aggregatable":true, "width":190 }, { "category":"cloud", "columnHeaderType":"not-filtered", "description":"The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.", "example":"666777888999", "id":"cloud.account.id", "type":"string", "aggregatable":true, "width":180 }, { "category":"cloud", "columnHeaderType":"not-filtered", "description":"Availability zone in which this host is running.", "example":"us-east-1c", "id":"cloud.availability_zone", "type":"string", "aggregatable":true, "width":180 }, // ... } ], // ... } } ``` _Above: column widths were persisted to `local storage` in previous release, (going at least back to `7.12`)_ In this PR, we utilize the `onColumnResize` [EuiDataGrid API](https://elastic.github.io/eui/#/tabular-content/data-grid) API as a callback to update Redux when the columns are sorted via the `Columns` popover. Updating Redux will in-turn update `local storage`, so resized columns widths will persist across page refreshes: ![onColumnResize](https://user-images.githubusercontent.com/4459398/136120062-3b0bebce-9c44-47fc-9956-48fe07a30f83.png) ### Other changes The Alerts page `Trend` chart and table were updated to include the following additional `Stack by` fields (CC @paulewing): ``` process.name file.name hash.sha256 ``` per the before / after screenshots below: ![alerts-trend-before](https://user-images.githubusercontent.com/4459398/137045011-7da4530b-0259-4fd4-b903-9eee6c26d02f.png) _Above: The Alerts `Trend` Stack by fields in `7.15` (before)_ ![alerts-trend-after](https://user-images.githubusercontent.com/4459398/137045023-d0ae987c-a474-4123-a05b-a6ad2fc52922.png) _Above: The Alerts `Trend` `Stack by` fields (after the addition of the `process.name`, `file.name`, and `hash.sha256` fields)_ CC: @monina-n @paulewing

@timestamp

…nd the Remove Column action (#114742) (#115301) ## [Security Solution] Restores Alerts table local storage persistence and the Remove Column action This PR implements the following changes summarized below to address <#113090>, as proposed [here](#113090 (comment)): - Configures the `Columns` popover to be consistent with `Discover` - Changes the `Hide column` action to `Remove column`, to be consistent with `Discover` - Persists updates to the `Columns` popover order in `local storage` - Restores the feature to persist column widths in `local storage` ### Configures the `Columns` popover to be consistent with `Discover` - We now pass `false` to the `allowHide` [EuiDataGrid API](https://elastic.github.io/eui/#/tabular-content/data-grid): ![allow_hide](https://user-images.githubusercontent.com/4459398/136114714-02f25b97-86af-47e5-9adc-1177d5a2c715.png) This makes all `EuiDataGrid`-based views in the Security Solution consistent with `Discover`'s use of the `EuiDataGrid` `Columns` popover. In `7.15`, the `Columns` popover includes the _hide column_ toggle, as shown in the screenshot below: ![alerts_columns_popover_7_15](https://user-images.githubusercontent.com/4459398/136112441-455ddbeb-dea3-4837-81ad-32d6c82c11fe.png) _Above: The `Columns` popover in the `7.15` `Alerts` table_ The `Columns` popover in `Discover`'s `EuiDataGrid`-based table does not display the hide column toggle, as shown the screenshot below: ![columns_popover_discover](https://user-images.githubusercontent.com/4459398/136112856-7e42c822-2260-4759-ac78-5bea63a171c7.png) _Above: The `EuiDataGrid` `Columns` popover in `Discover`, in `master`_ Passing `false` to the `allowHide` [EuiDataGrid API](https://elastic.github.io/eui/#/tabular-content/data-grid) API makes the `Columns` popover in all `EuiDataGrid`-based views in the Security Solution consistent with `Discover`, as illustrated by the screenshot below: ![alerts_columns_popover_no_hide](https://user-images.githubusercontent.com/4459398/136112980-d4219fbd-1443-4612-8cdb-b97bee8b97ef.png) _Above: The `Columns` popover is now consistent with `Discover`_ ## Changes the `Hide column` action to `Remove column`, to be consistent with `Discover` - The `Hide column` action shown in the `7.15` alerts table is changed to `Remove column`, making it consistent with `Discover`'s use of `EuiDataGrid` In `7.15`, the `Alerts` table has a `Hide column` action, as shown in the screenshot below: ![hide_column](https://user-images.githubusercontent.com/4459398/136115681-9e0da144-a981-4352-8092-9368d74cd153.png) _Above: The `Hide Column` action in the `7.15` `Alerts` table_ In `7.15`, clicking the `Hide Column` action shown in the screenshot above hides the column, but does not remove it. In `7.15`, columns may only be removed by un-checking them in the `Fields` browser, or by un-toggling them in the Alerts / Events details popover. Both of those methods require multiple clicks, and require uses to re-find the field in the modal or popover before it may be toggled for removal. In `Discover`, users don't hide columns. In `Discover`, users directly remove columns by clicking the `Remove column` action, shown in the screenshot below: ![discover_remove_column](https://user-images.githubusercontent.com/4459398/136114295-f018a561-f9ee-4ce4-a9c6-0fcd7f71e67b.png) _Above: The `Remove column` action in `Discover`'s use of `EuiDataGrid` in `master`_ All `EuiDataGrid`-based views in the Security Solution were made consistent with `Discover` by replacing the `Hide column` action with `Remove column`, per the screenshot below: ![remove_column_after](https://user-images.githubusercontent.com/4459398/137047582-3c4d6cb0-ac12-4c50-9c34-0c4ef5536550.png) _Above: The `Remove column` action in the Alerts table_ Note: the `Remove column` action shown above appears as the last item in the popover because it's specified via the `EuiDataGrid` `EuiDataGridColumnActions` > `additonal` API, which appends additonal actions to the end of popover, after the built-in actions: ![additional](https://user-images.githubusercontent.com/4459398/137047825-625002b3-5cd6-4b3e-87da-e76dbaf2a827.png) ## Persists updates to the `Columns` popover order in `local storage` - Persist column order updates to `local storage` when users update the order of columns via the `Columns` popover The following PR <#110685> restored partial support for persisting columns across page refreshes via `local storage`, but the Redux store was not updated when users sort columns via the `Columns` popover, an shown in the animated gif below: ![ordering_via_columns](https://user-images.githubusercontent.com/4459398/136119497-65f76f49-091c-4a45-b8d3-1e5ef80ccbb2.gif) _Above: Ordering via the `Columns` popover is not persisted to `local storage` in `7.15`_ This PR utilizes the `setVisibleColumns` [EuiDataGrid API](https://elastic.github.io/eui/#/tabular-content/data-grid) API as a callback to update Redux when the columns are sorted, which will in-turn update `local storage` to persist the new order across page refreshes: ![setVisibleColumns](https://user-images.githubusercontent.com/4459398/136117249-628bb147-a860-4ccf-811a-0e57a99296fb.png) ## Restores the feature to persist column widths in `local storage` In previous releases, resized column widths were peristed in `local storage` to persist across page refreshes, as documented in <#110524> : ``` { "detections-page":{ "id":"detections-page", "activeTab":"query", "prevActiveTab":"query", "columns":[ { "category":"base", "columnHeaderType":"not-filtered", "description":"Date/time when the event originated. This is the date/time extracted from the event, typically representing when the event was generated by the source. If the event source has no original timestamp, this value is typically populated by the first time the event was received by the pipeline. Required field for all events.", "example":"2016-05-23T08:05:34.853Z", "id":"@timestamp", "type":"date", "aggregatable":true, "width":190 }, { "category":"cloud", "columnHeaderType":"not-filtered", "description":"The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.", "example":"666777888999", "id":"cloud.account.id", "type":"string", "aggregatable":true, "width":180 }, { "category":"cloud", "columnHeaderType":"not-filtered", "description":"Availability zone in which this host is running.", "example":"us-east-1c", "id":"cloud.availability_zone", "type":"string", "aggregatable":true, "width":180 }, // ... } ], // ... } } ``` _Above: column widths were persisted to `local storage` in previous release, (going at least back to `7.12`)_ In this PR, we utilize the `onColumnResize` [EuiDataGrid API](https://elastic.github.io/eui/#/tabular-content/data-grid) API as a callback to update Redux when the columns are sorted via the `Columns` popover. Updating Redux will in-turn update `local storage`, so resized columns widths will persist across page refreshes: ![onColumnResize](https://user-images.githubusercontent.com/4459398/136120062-3b0bebce-9c44-47fc-9956-48fe07a30f83.png) ### Other changes The Alerts page `Trend` chart and table were updated to include the following additional `Stack by` fields (CC @paulewing): ``` process.name file.name hash.sha256 ``` per the before / after screenshots below: ![alerts-trend-before](https://user-images.githubusercontent.com/4459398/137045011-7da4530b-0259-4fd4-b903-9eee6c26d02f.png) _Above: The Alerts `Trend` Stack by fields in `7.15` (before)_ ![alerts-trend-after](https://user-images.githubusercontent.com/4459398/137045023-d0ae987c-a474-4123-a05b-a6ad2fc52922.png) _Above: The Alerts `Trend` `Stack by` fields (after the addition of the `process.name`, `file.name`, and `hash.sha256` fields)_ CC: @monina-n @paulewing Co-authored-by: Andrew Goldstein <andrew-goldstein@users.noreply.github.com>

fix stringify circular ref crash and default columns on createTimeline

b4eef3e

semd requested a review from a team as a code owner August 31, 2021 18:08

semd self-assigned this Aug 31, 2021

semd requested a review from a team as a code owner August 31, 2021 18:08

rollback reset buton fix to split PR

0a2e8a9

semd changed the title ~~[RAC] Fix persistent fields and Reset button on timeline table~~ [RAC] Persistent timeline fields fix Aug 31, 2021

XavierM reviewed Sep 1, 2021

View reviewed changes

semd added 2 commits September 1, 2021 12:23

Merge remote-tracking branch 'upstream/master' into 110524_fields_res…

39b8775

…et_localstorage_fix

adding fields to the storage cleaning

b016f59

semd enabled auto-merge (squash) September 1, 2021 16:27

XavierM reviewed Sep 1, 2021

View reviewed changes

semd added 2 commits September 2, 2021 11:27

tests fixed

647b868

test fix

708fb9e

semd disabled auto-merge September 2, 2021 13:51

semd enabled auto-merge (squash) September 2, 2021 13:57

machadoum approved these changes Sep 2, 2021

View reviewed changes

semd merged commit 56669c0 into elastic:master Sep 2, 2021

kibanamachine mentioned this pull request Sep 2, 2021

[7.15] [RAC] Persistent timeline fields fix (#110685) #110995

Merged

kibanamachine mentioned this pull request Sep 2, 2021

[7.x] [RAC] Persistent timeline fields fix (#110685) #110996

Merged

semd mentioned this pull request Sep 14, 2021

[7.14] [RAC] Persistent timeline fields fix (#110685) #112150

Merged

semd added the v7.14.2 label Sep 16, 2021

andrew-goldstein mentioned this pull request Oct 6, 2021

[Security Solution] All hidden fields in Alert table become visible when a new field is added #113090

Closed

andrew-goldstein mentioned this pull request Oct 13, 2021

[Security Solution] Restores Alerts table local storage persistence and the Remove Column action #114742

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RAC] Persistent timeline fields fix #110685

[RAC] Persistent timeline fields fix #110685

semd commented Aug 31, 2021 •

edited

Loading

elasticmachine commented Aug 31, 2021

elasticmachine commented Aug 31, 2021

XavierM Sep 1, 2021

XavierM Sep 1, 2021

semd Sep 2, 2021

kibanamachine commented Sep 2, 2021

Standard Out

Stack Trace

machadoum left a comment

kibanamachine commented Sep 2, 2021

arvindersingh-qasource-zz commented Sep 16, 2021

semd commented Sep 16, 2021 •

edited

Loading

arvindersingh-qasource-zz commented Sep 16, 2021

semd commented Sep 16, 2021

[RAC] Persistent timeline fields fix #110685

[RAC] Persistent timeline fields fix #110685

Conversation

semd commented Aug 31, 2021 • edited Loading

Summary

elasticmachine commented Aug 31, 2021

elasticmachine commented Aug 31, 2021

XavierM Sep 1, 2021

Choose a reason for hiding this comment

XavierM Sep 1, 2021

Choose a reason for hiding this comment

semd Sep 2, 2021

Choose a reason for hiding this comment

kibanamachine commented Sep 2, 2021

💛 Build succeeded, but was flaky

Test Failures

Standard Out

Stack Trace

Metrics [docs]

Async chunks

History

machadoum left a comment

Choose a reason for hiding this comment

kibanamachine commented Sep 2, 2021

💚 Backport successful

arvindersingh-qasource-zz commented Sep 16, 2021

Please find the below observations

Build Details

Timelines on Dev tools for Chrome Browser

semd commented Sep 16, 2021 • edited Loading

arvindersingh-qasource-zz commented Sep 16, 2021

Please find the below observations

Build Details

Timelines on Dev tools for Chrome Browser

semd commented Sep 16, 2021

semd commented Aug 31, 2021 •

edited

Loading

semd commented Sep 16, 2021 •

edited

Loading