Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Security Solution][SIEM migrations] Onboarding UI improvements #204320

Merged
merged 83 commits into from
Dec 17, 2024
Merged

[Security Solution][SIEM migrations] Onboarding UI improvements #204320

merged 83 commits into from
Dec 17, 2024
+1,852 −640

Conversation

semd
Copy link
Contributor

@semd semd commented Dec 14, 2024 •
edited by kibanamachine
Loading

Summary

Part of: https://github.com/elastic/security-team/issues/10667

Improvements

  • Implementation of the Onboarding card to create migrations using the flyout
  • Migration complete summary panel implemented
  • Migration ready panel improved to detect missing resources
  • Migration processing improved
  • Migration missing resources panel implemented
  • All migration panels and refactored to be reusable by translation table using the
  • RuleMigrationDataInputWrapper implemented to reuse the Flyout from the translation table
  • Request poll interval increased from 5 to 10 seconds due to event loop usage.

Note

This feature needs siemMigrationsEnabled experimental flag enabled to work.

Screenshots

Lookups input

Lookups

Translation "complete" panel
Translation summary

Translation "created" panel (w/ and w/o missing macros)
Ready panels

Translation processing (preparing)
preparing panel

Translation processing (translating)
translating panel

semd and others added 30 commits November 22, 2024 15:53
@semd
add support for topics on the onboarding page
60fbe6f
@semd
reorganize directories inside components
3985477
@semd
solve conflict
03b5034
@semd
Merge remote-tracking branch 'upstream/main' into 10667/siem_migratio…
376f2d6 
…ns/onboarding_ui
@semd
Merge remote-tracking branch 'upstream' into 10667/siem_migrations/on…
318d79e 
…boarding_ui
@semd
resources schema id
8557994
@semd
update and add tests
da3938c
@semd
resources type fix
132c67b
@semd
add connector selection and auto-resume stopped migrations within the…
94c56c8 
… request polling
@semd
Merge remote-tracking branch 'origin/10667/siem_migrations/onboarding…
a0702c9 
…_ui' into 10667/siem_migrations/onboarding_ui_flyout
@elasticmachine
Merge branch 'main' into 10667/siem_migrations/onboarding_ui
1d1f1da
@semd
initial implementation
cee50d7
@semd
extract common styles into single places
5a9a06a
@semd
format code
f2beaf3
@semd
Merge branch '10667/siem_migrations/onboarding_ui' into 10667/siem_mi…
9f305c9 
…grations/onboarding_ui_flyout
@semd
conflict solve
4ce992d
@semd
cleaning
8b60935
@semd
solve conflict
f09344b
@semd
upload panel styles
ef6c17d
@semd
flyout scafolding
5cb839d
@semd
solve conflicts
ecadb11
@semd
add upload input
d79dc47
@semd
basic workflow working
a486424
@semd
Merge remote-tracking branch 'upstream/main' into 10667/siem_migratio…
0ce93d9 
…ns/onboarding_ui_flyout
@semd
no connector selected callout
b02a569
@semd
solve conflicts
216a6dc
@kibanamachine
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
0d6ae1f 
…-fix'
@semd
add langsmith trace options
6e63ffa
@semd
Merge remote-tracking branch 'refs/remotes/origin/10667/siem_migratio…
86210fc 
…ns/onboarding_ui_flyout' into 10667/siem_migrations/onboarding_ui_flyout
@semd
lint fixes
ac6e675
@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
securitySolution 6410 6430 +20

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
securitySolution 14.7MB 14.8MB +16.0KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
securitySolution 88.0KB 88.0KB +1.0B
Unknown metric groups

async chunk count

id before after diff
securitySolution 105 106 +1

References to deprecated APIs

id before after diff
securitySolution 491 489 -2

History

cc @semd

e40pud
e40pud reviewed Dec 17, 2024
View reviewed changes
...migrations/rules/task/agent/sub_graphs/translate_rule/nodes/translate_rule/translate_rule.ts Outdated
@@ -63,9 +63,9 @@ export const getTranslateRuleNode = ({
};
};

const getTranslationResult = (esqlQuery: string): SiemMigrationRuleTranslationResult => {
const getTranslationResult = (esqlQuery: string): RuleTranslationResult => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this method is equivalent of another function in x-pack/plugins/security_solution/server/lib/siem_migrations/rules/task/agent/sub_graphs/translate_rule/nodes/ecs_mapping/ecs_mapping.ts. Maybe we could re-use this code

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, noted. I am creating a list of things to do on the agent side, we can do it in a separate PR.

e40pud
e40pud approved these changes Dec 17, 2024
View reviewed changes
Copy link
Contributor

@e40pud e40pud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested these changes locally - everything looks great!

angorayc
angorayc approved these changes Dec 17, 2024
View reviewed changes
Copy link
Contributor

@angorayc angorayc left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

  1. It cannot be unmarked after marking the lookup as empty
  2. Would be nice to indicate that the lookup file is a CSV (as previous uploaded files are JSON)
  3. Delete an added migration.
Screenshot 2024-12-17 at 12 08 25

...utions/security/plugins/security_solution/public/common/components/panel_text/panel_text.tsx Outdated Show resolved Hide resolved
...boarding/components/onboarding_body/cards/siem_migrations/ai_connector/ai_connector_card.tsx
const connectors = checkCompleteMetadata?.connectors;
const canExecuteConnectors = checkCompleteMetadata?.canExecuteConnectors;
const canCreateConnectors = checkCompleteMetadata?.canCreateConnectors;
if (!checkCompleteMetadata) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@semd
Copy link
Contributor Author

semd commented Dec 17, 2024

@angorayc Thanks a lot for the review, about the questions:

  • It cannot be unmarked after marking the lookup as empty

After marking a lookup as empty, users can still copy the name and upload the file, if it's provided it's unmarked:
lookup_uploaded

  • Would be nice to indicate that the lookup file is a CSV (as previous uploaded files are JSON)

We assume it's a CSV for now, but we'll need to support both formats for the MVP (8.18/9.0). I will add specific parsers in the following PRs

  • Delete an added migration.

That won't be possible in the MVP.

@semd semd enabled auto-merge (squash) December 17, 2024 13:02
@semd semd merged commit 303eee8 into elastic:main Dec 17, 2024
8 checks passed
@elasticmachine
Copy link
Contributor

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
securitySolution 6413 6432 +19

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
securitySolution 14.7MB 14.8MB +15.0KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
securitySolution 88.0KB 88.0KB +1.0B
Unknown metric groups

async chunk count

id before after diff
securitySolution 105 106 +1

References to deprecated APIs

id before after diff
securitySolution 491 489 -2

History

cc @semd

@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/12375309176

@kibanamachine
Copy link
Contributor

💔 All backports failed

Status Branch Result
❌ 8.x Backport failed because of merge conflicts

You might need to backport the following PRs to 8.x:
- Update data archives for SO integration tests (#204561)
- SKA: Relocate script v5.1 (#204563)
- [ML] Transforms: Support wildcards in the alerting rule flyout (#204226)
- [Search] Deprecate canDeployEntSearch config value (#204272)
- [Fleet] Display next steps and actions in agentless integrations flyout (#203824)
- [devx] Create script to stage files by CODEOWNER (#203940)
- [ai][assistant] Refactor o11y to use new Assistant logo and beacon (#204289)
- [Security Solution] Preview navigation in document details flyout (#204292)
- Fix generation of dynamic mapping for object with specific subfield (#204104)
- Sustainable Kibana Architecture: Move modules owned by @elastic/kibana-cloud-security-posture (#202862)

Manual backport

To create the backport manually run:

node scripts/backport --pr 204320

Questions ?

Please refer to the Backport tool documentation

@kibanamachine kibanamachine mentioned this pull request Dec 17, 2024
Merged
4 tasks
@semd semd mentioned this pull request Dec 18, 2024
Open
@semd
Copy link
Contributor Author

semd commented Dec 18, 2024

💚 All backports created successfully

Status Branch Result
✅ 8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

semd added a commit to semd/kibana that referenced this pull request Dec 18, 2024
@semd
[Security Solution][SIEM migrations] Onboarding UI improvements (elas…
7cdbdca 
…tic#204320)

## Summary

Part of: elastic/security-team#10667

#### Improvements

- Implementation of the Onboarding card to create migrations using the
flyout
- Migration complete summary panel implemented
- Migration ready panel improved to detect missing resources
- Migration processing improved
- Migration missing resources panel implemented
- All migration panels and refactored to be reusable by translation
table using the
- `RuleMigrationDataInputWrapper` implemented to reuse the Flyout from
the translation table
- Request poll interval increased from 5 to 10 seconds due to event loop
usage.

> [!NOTE]
> This feature needs `siemMigrationsEnabled` experimental flag enabled
to work.

## Screenshots

Lookups input

![Lookups](https://github.com/user-attachments/assets/73f91e10-7252-44d1-ab0d-89880c78a2b3)

Translation "complete" panel
![Translation
summary](https://github.com/user-attachments/assets/6fbb451d-c7b3-4a23-a2df-083c91948cbd)

Translation "created" panel (w/ and w/o missing macros)
![Ready
panels](https://github.com/user-attachments/assets/f8334af2-ccc1-473c-8548-772a9d656aba)

Translation processing (preparing)
![preparing
panel](https://github.com/user-attachments/assets/0156caba-c6c9-43c1-881a-8bf631f3a8ab)

Translation processing (translating)
![translating
panel](https://github.com/user-attachments/assets/db523e4b-4858-482f-bfe9-1e36f715fa20)

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit 303eee8)
@kibanamachine kibanamachine mentioned this pull request Dec 18, 2024
Merged
@kibanamachine kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Dec 19, 2024
@kibanamachine
Copy link
Contributor

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

JoseLuisGJ pushed a commit to JoseLuisGJ/kibana that referenced this pull request Dec 19, 2024
@semd @elasticmachine
@kibanamachine @JoseLuisGJ
[Security Solution][SIEM migrations] Onboarding UI improvements (elas…
c9ea9e5 
…tic#204320)

## Summary

Part of: elastic/security-team#10667

#### Improvements

- Implementation of the Onboarding card to create migrations using the
flyout
- Migration complete summary panel implemented
- Migration ready panel improved to detect missing resources
- Migration processing improved
- Migration missing resources panel implemented
- All migration panels and refactored to be reusable by translation
table using the
- `RuleMigrationDataInputWrapper` implemented to reuse the Flyout from
the translation table
- Request poll interval increased from 5 to 10 seconds due to event loop
usage.


> [!NOTE]  
> This feature needs `siemMigrationsEnabled` experimental flag enabled
to work.

## Screenshots

Lookups input


![Lookups](https://github.com/user-attachments/assets/73f91e10-7252-44d1-ab0d-89880c78a2b3)

Translation "complete" panel
![Translation
summary](https://github.com/user-attachments/assets/6fbb451d-c7b3-4a23-a2df-083c91948cbd)

Translation "created" panel (w/ and w/o missing macros)
![Ready
panels](https://github.com/user-attachments/assets/f8334af2-ccc1-473c-8548-772a9d656aba)

Translation processing (preparing)
![preparing
panel](https://github.com/user-attachments/assets/0156caba-c6c9-43c1-881a-8bf631f3a8ab)

Translation processing (translating)
![translating
panel](https://github.com/user-attachments/assets/db523e4b-4858-482f-bfe9-1e36f715fa20)

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
@kibanamachine
Copy link
Contributor

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

5 similar comments
@kibanamachine
Copy link
Contributor

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

@kibanamachine
Copy link
Contributor

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

@kibanamachine
Copy link
Contributor

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

@kibanamachine
Copy link
Contributor

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

@kibanamachine
Copy link
Contributor

Looks like this PR has a backport PR but it still hasn't been merged. Please merge it ASAP to keep the branches relatively in sync.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@angorayc angorayc angorayc approved these changes

@e40pud e40pud e40pud approved these changes

Assignees

@semd semd

Labels
backport missing Added to PRs automatically when the are determined to be missing a backport. backport:version Backport to applied version labels release_note:skip Skip the PR/issue when compiling release notes Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Threat Hunting Security Solution Threat Hunting Team v8.18.0 v9.0.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@semd @elasticmachine @kibanamachine @e40pud @angorayc