[Security Solution] Integrate Vanilla Unified Data Table in Timeline #176064

michaelolo24 · 2024-02-01T13:20:48Z

Fixes below issues

Details

Objective

The objective of this PR is to implement basic unified data table replacing the current timeline table. It is intended to behave exactly like discover table. Below are the functionalities of the timeline table that are out of scope of this PR and will be included in the follow up PR.

Before	After
after_timeline_unified.mov	before_timeline_unified.mov

This feature can be enabled with below feature flag:

xpack.securitySolution.enableExperimental:
  - unifiedComponentsInTimelineEnabled

Out of scope functionalities

Row Renderers - https://github.com/elastic/security-team/issues/8728
Notes / Pinned Events / Analyzer / Session View Link - https://github.com/elastic/security-team/issues/8727
Drag / Drop from table --> data providers.
Telemetry

Desk Testing Guide

Below are some areas which have changes and would warrant some desk testing.

One pre-requisite is to enable feature flag unifiedComponentsInTimelineEnabled which will replace traditional timeline table with the new unified timeline table.

Pagination
- If total number of events are > sample size ( default 500 ), table footer on last page should display the notification so that user can Load More on-demand.
- Page changes should be instant, because it is just client-side pagination where we download multiple pages ( 500 events ) at once.
Flyouts
- Event Detail ( Open / Close ) - should be ~~new~~ old Expandable flyout - ( New Expandable fl
- Host / User / IP details flyout ( Open / Close)
Unified List
- Add column by dragging & by clicking ⨁ control.
- Remove columns by clicking on column header & by clicking on ❌ control
Full screen mode
Last updated date
Columns Order
- Change columns order from table controls
Sort Order
- timestamp
- Any number column
- Any string column
Column Actions
- Move Right/Left
- Sort Asc/Desc
- Copy Column and Column Name
- Edit Data View Field with custom label
Table Control - Display Options
- Try different row heights and Sample sizes.

Current Observations of Unified table vis-a-vis discover UI + Issues

Checklist

Delete any items that are not applicable to this PR.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
Flaky Test Runner was used on any tests changed
Any UI touched in this PR is usable by keyboard only (learn more about keyboard accessibility)
Any UI touched in this PR does not create any new axe failures (run axe in browser: FF, Chrome)
If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
This renders correctly on smaller devices using a responsive layout. (You can test this in your browser)
This was checked for cross-browser compatibility

x-pack/plugins/security_solution/public/common/components/sourcerer/get_sourcerer_data_view.tsx

x-pack/plugins/security_solution/public/common/lib/cell_actions/field_value.tsx

...n/public/detections/components/alerts_table/timeline_actions/use_investigate_in_timeline.tsx

logeekal · 2024-02-15T08:19:10Z

...solution/public/timelines/components/timeline/unified_components/data_table/last_updated.tsx

+  updatedAt: number;
+}
+
+export const FixedWidthLastUpdatedContainer = React.memo<FixedWidthLastUpdatedContainerProps>(


Reminder for @logeekal : need to check why is this needed when there is a last updated component already.

@YulNaumenko do you remember?

Separated out and deduped last updated component. This is the final one with tests : https://github.com/elastic/kibana/pull/176064/files#diff-36743070b5744453188c3858a3869fa054832cba119926b371108f818cb01a77

Here is the commit : 4c8fe91

MadameSheema · 2024-03-28T08:50:39Z

.../detection_engine/rule_creation/indicator_match_rule_suppression_serverless_essentials.cy.ts

@@ -37,6 +37,12 @@ describe(
          { product_line: 'security', product_tier: 'essentials' },
          { product_line: 'endpoint', product_tier: 'essentials' },
        ],
+        // alertSuppressionForIndicatorMatchRuleEnabled feature flag is also enabled in a global config
+        kbnServerArgs: [


Why the feature flag has been added globally? We should add FF at the beginning of the spec file.

It must be a merge issue. Let me check.. I did not touch this file.

This was added as part of this PR. I think while resolving merge conflicts some issue happenned. I will just correct this in accordance with main. Additionally, I could not find this flag in global config.

@MadameSheema , I have restored this file according to main.

MadameSheema · 2024-03-28T09:18:27Z

...ity_solution_cypress/cypress/e2e/investigations/timelines/unified_components/query_tab.cy.ts

+    context('flyout', () => {
+      it('should be able to open/close details details/host/user flyout', () => {
+        cy.log('Event Details Flyout');
+        openEventDetailsFlyout(0);


the assertion you are doing in the openEventDetailsFlyout I'll add it here to make more visible what is happening, if not seems that we are doing a test without assertions :)

fixed here: 84fe0fb

MadameSheema · 2024-03-28T09:21:13Z

x-pack/test/security_solution_cypress/cypress/tasks/discover.ts

@@ -112,6 +112,10 @@ export const addFieldToTable = (fieldId: string) => {
  clearFieldSearch();
 };

+export const removeFieldFromTable = (fieldId: string) => {
+  cy.get(GET_DISCOVER_COLUMN_TOGGLE_BTN(fieldId)).first().trigger('click');


Why are we using .trigger('click') instead of click()?

fixed here: 84fe0fb

MadameSheema

Lots of thanks for addressing all the comments! :)

semd · 2024-03-28T15:12:28Z

x-pack/plugins/security_solution/public/timelines/components/open_timeline/helpers.ts

@@ -254,13 +251,21 @@ export const getTimelineStatus = (
 export const defaultTimelineToTimelineModel = (
  timeline: TimelineResult,
  duplicate: boolean,
-  timelineType?: TimelineType
+  timelineType?: TimelineType,
+  unifiedComponentsInTimelineEnabled?: boolean


The current approach forces many components to retrieve this value using useIsExperimentalFeatureEnabled to pass it to this function.

Wouldn't it be easier to use ExperimentalFeaturesService singleton inside this file, instead of receiving this value by parameter? It would simplify the cleaning when we remove this flag too.

@semd , I did not even know that it existed. I agree with you but I have three more upcoming PRs for this functionality. Do you mind, if I do that change in next PR ?

Yes sure, it's just a simplification, and also to reduce the scope of the changes. But if you keep it this way it's also fine, as you prefer 👍

Actually I will have to check the Singleton class API but I will make changes accordingly in the next PR.

I would be glad as well if I could remove this extra param from everywhere.

semd · 2024-03-28T17:18:00Z

securitySolution 16.6MB 17.0MB ⚠️ +459.0KB

I hope we go back to previous sizes when we clean the old version.

Thanks for doing these changes, looks great 💯

…ata-table-1

michaelolo24 · 2024-03-29T17:37:50Z

/ci

…-fix'

…ata-table-1

michaelolo24 · 2024-04-01T15:24:37Z

/ci

michaelolo24 · 2024-04-01T15:43:24Z

/ci

kibana-ci · 2024-04-01T16:53:15Z

💚 Build Succeeded

Buildkite Build
Commit: 2467a37

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`securitySolution`	5042	5287	+245

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`securitySolution`	122	124	+2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`cloudSecurityPosture`	401.3KB	401.3KB	+84.0B
`discover`	590.3KB	590.4KB	+166.0B
`logsExplorer`	1.1MB	1.1MB	+4.0B
`securitySolution`	16.6MB	17.0MB	⚠️ +459.0KB
`slo`	610.4KB	610.5KB	+144.0B
total			+459.4KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`securitySolution`	72.8KB	73.3KB	+518.0B

Unknown metric groups

API count

id	before	after	diff
`@kbn/unified-data-table`	143	144	+1
`securitySolution`	193	195	+2
total			+3

async chunk count

id	before	after	diff
`securitySolution`	57	66	+9

ESLint disabled line counts

id	before	after	diff
`securitySolution`	490	495	+5

References to deprecated APIs

id	before	after	diff
`securitySolution`	531	533	+2

Total ESLint disabled count

id	before	after	diff
`securitySolution`	566	571	+5

History

💔 Build #200188 failed 2df4376
💛 Build #200004 was flaky c312a7b
💚 Build #199920 succeeded 84fe0fb
💛 Build #199861 was flaky 472c173
💔 Build #199800 failed a4b1e9c

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @logeekal

PhilippeOberti

Here are a few observations:

in the PR description, we say that this PR fixes this ticket but looking at the tasks on that ticket, a lot of them aren't actually covered by this PR. We should maybe remove the link to the ticket so it doesn't confuse PR later?
PR description point 2: the expandable flyout isn't opening, even when using the expandableTimelineFlyoutEnabled flag. This is valid for the document details, host and user flyouts. For all of them, only the old flyout opens (but it does open on top of timeline instead of within the timeline modal along side the table so that's good)
there is a weird blinking happening when removing a column. Also I wonder why we fetch the data on removing columns, it's not like we need new data...
https://github.com/elastic/kibana/assets/17276605/5fb47ce8-a536-4ea3-b915-6217ab49bbd6
PR description point 3: I'm confused about that the + and x buttons are?
I'm wondering why we trigger a fetch for the data when we move a column around. It's not like we need any new information?
Same thing for the column sorting. Are we fetching different data that the 500 (or whatever selected in the table control display option) or just sorting the data in the frontend?
It seems that sorting by @timestamp isn't working. Other columns seem to be working fine
https://github.com/elastic/kibana/assets/17276605/fbca15a3-3212-4a97-bbb1-0ad38b510d84

PhilippeOberti

Approving for the Threat Hunting Investigations team, as all the points are raised will be tackled in a subsequent PR (see issue)

PhilippeOberti · 2024-04-02T12:36:50Z

@logeekal @michaelolo24 I think it would be nice to update the PR description to really reflect what the code in this PR does, and what it does now. I raised a few points in my last comment, but I think having the PR description as reflective of the code as possible will be beneficiary if we come back to this one day.

logeekal · 2024-04-02T12:54:07Z

PR description point 3: I'm confused about that the + and x buttons are?

Below is + and X icons in unified field list

## Summary This PR cleans up a bit of the code around timeline tabs. It de-dupes the layout components and some of the shared functionality between the tabs. I would also like to move the tabs from using the `connector` pattern to using `useSelector`, but that'll be done in a follow up PR. The commit history unfortunately pulls in a bit from [this pr](#176064), but the 2 commits for the actual files changed in this PR are as follows: 1. (No code changes, just moving files) Moving the tabs into a nested tabs folder: 89fa2d8 2. (Actual Code Changes) De-duping the shared components: c6eecdb 3. (No code changes, moving filed and renaming) Removed the `_content` parts of the folder names, and moved the `tabs_content` files into the `tabs` folder: ec3b959 --------- Co-authored-by: Jatin Kathuria <jatin.kathuria@elastic.co> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

michaelolo24 added 5 commits February 1, 2024 08:19

wip udt

78cfc74

use control columns completed

7eb5b4d

wip column

5dff62f

got table visible

72cfff5

move query tab to new file

258ed5d

michaelolo24 force-pushed the integrate-unified-data-table-1 branch from 2a6c95c to 258ed5d Compare February 1, 2024 13:21

michaelolo24 and others added 2 commits February 1, 2024 11:35

Continued refactor of table code

20921de

fix: add appropriate actions and unified table changes

dfde043

logeekal changed the title ~~Integrate unified data table 1~~ [Security Solution] Integrate Vanilla Unified Data Table in Timeline Feb 9, 2024

logeekal added 4 commits February 13, 2024 08:06

fix: column auto adjust width

b1556c7

fix: Field Formatting

769d5ce

fix: flyout issues + rename var

1a4b9a4

switch off feature flag by default

e87c5a9

logeekal reviewed Feb 15, 2024

View reviewed changes

x-pack/plugins/security_solution/public/common/components/sourcerer/get_sourcerer_data_view.tsx Outdated Show resolved Hide resolved

logeekal reviewed Feb 15, 2024

View reviewed changes

x-pack/plugins/security_solution/public/common/lib/cell_actions/field_value.tsx Outdated Show resolved Hide resolved

logeekal reviewed Feb 15, 2024

View reviewed changes

...n/public/detections/components/alerts_table/timeline_actions/use_investigate_in_timeline.tsx Show resolved Hide resolved

logeekal reviewed Feb 15, 2024

View reviewed changes

logeekal added 3 commits February 15, 2024 09:28

fix: housekeeping

aed026b

fix: cypress + unit tests

a5c7a81

Merged main -> current branch

c2e3719

logeekal self-assigned this Feb 20, 2024

logeekal added Team:Threat Hunting:Investigations Security Solution Investigations Team 8.14 candidate release_note:skip Skip the PR/issue when compiling release notes labels Feb 20, 2024

fix: types

6fb3bef

logeekal added the backport:skip This commit does not require backporting label Feb 20, 2024

logeekal added 2 commits February 21, 2024 06:48

fix: alert acount

9595983

fix: some tests + remove unnecessary changes

d42cb2e

logeekal marked this pull request as ready for review February 21, 2024 06:52

logeekal requested a review from a team as a code owner February 21, 2024 06:52

update timeout time

472c173

MadameSheema reviewed Mar 28, 2024

View reviewed changes

logeekal added 2 commits March 28, 2024 10:01

Merge branch 'main' into integrate-unified-data-table-1

112a343

fix: merge issues

7360c8c

MadameSheema reviewed Mar 28, 2024

View reviewed changes

logeekal added 2 commits March 28, 2024 10:49

fix: review feedback cypress

84fe0fb

fix: add brokenInServerlessQA tag

c312a7b

MadameSheema approved these changes Mar 28, 2024

View reviewed changes

logeekal enabled auto-merge (squash) March 28, 2024 14:44

semd reviewed Mar 28, 2024

View reviewed changes

semd approved these changes Mar 28, 2024

View reviewed changes

Merge remote-tracking branch 'upstream/main' into integrate-unified-d…

8e840b2

…ata-table-1

kibanamachine and others added 3 commits March 29, 2024 18:18

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

2df4376

…-fix'

update timeline helpers tests

4aa05e9

Merge remote-tracking branch 'upstream/main' into integrate-unified-d…

2467a37

…ata-table-1

PhilippeOberti reviewed Apr 2, 2024

View reviewed changes

PhilippeOberti approved these changes Apr 2, 2024

View reviewed changes

logeekal merged commit 883beee into elastic:main Apr 2, 2024
34 checks passed

michaelolo24 mentioned this pull request Apr 2, 2024

[Investigations] - Timeline tabs file cleanup #179832

Merged

This was referenced Apr 23, 2024

[Security Solution] Alert Detail flyout for Linux Session Alert overlaps Investigation Timeline. #180679

Closed

[Security Solution][Alert details] - remove old flyout unnecessary z-index change #181480

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Security Solution] Integrate Vanilla Unified Data Table in Timeline #176064

[Security Solution] Integrate Vanilla Unified Data Table in Timeline #176064

michaelolo24 commented Feb 1, 2024 •

edited by logeekal

Loading

logeekal Feb 15, 2024 •

edited

Loading

michaelolo24 Mar 8, 2024

logeekal Mar 11, 2024

logeekal Mar 11, 2024

MadameSheema Mar 28, 2024

logeekal Mar 28, 2024

logeekal Mar 28, 2024 •

edited

Loading

logeekal Mar 28, 2024

MadameSheema Mar 28, 2024

logeekal Mar 28, 2024

MadameSheema Mar 28, 2024

logeekal Mar 28, 2024

MadameSheema left a comment

semd Mar 28, 2024 •

edited

Loading

logeekal Mar 28, 2024

semd Mar 28, 2024

logeekal Mar 28, 2024

semd commented Mar 28, 2024

michaelolo24 commented Mar 29, 2024

michaelolo24 commented Apr 1, 2024

michaelolo24 commented Apr 1, 2024

kibana-ci commented Apr 1, 2024

API count

async chunk count

ESLint disabled line counts

References to deprecated APIs

Total ESLint disabled count

PhilippeOberti left a comment •

edited

Loading

PhilippeOberti left a comment

PhilippeOberti commented Apr 2, 2024

logeekal commented Apr 2, 2024

[Security Solution] Integrate Vanilla Unified Data Table in Timeline #176064

[Security Solution] Integrate Vanilla Unified Data Table in Timeline #176064

Conversation

michaelolo24 commented Feb 1, 2024 • edited by logeekal Loading

Fixes below issues

Details

Objective

Out of scope functionalities

Desk Testing Guide

Current Observations of Unified table vis-a-vis discover UI + Issues

Checklist

logeekal Feb 15, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

logeekal Mar 28, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MadameSheema left a comment

Choose a reason for hiding this comment

semd Mar 28, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

semd commented Mar 28, 2024

michaelolo24 commented Mar 29, 2024

michaelolo24 commented Apr 1, 2024

michaelolo24 commented Apr 1, 2024

kibana-ci commented Apr 1, 2024

💚 Build Succeeded

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Page load bundle

API count

async chunk count

ESLint disabled line counts

References to deprecated APIs

Total ESLint disabled count

History

PhilippeOberti left a comment • edited Loading

Choose a reason for hiding this comment

PhilippeOberti left a comment

Choose a reason for hiding this comment

PhilippeOberti commented Apr 2, 2024

logeekal commented Apr 2, 2024

michaelolo24 commented Feb 1, 2024 •

edited by logeekal

Loading

logeekal Feb 15, 2024 •

edited

Loading

logeekal Mar 28, 2024 •

edited

Loading

semd Mar 28, 2024 •

edited

Loading

PhilippeOberti left a comment •

edited

Loading