-
Notifications
You must be signed in to change notification settings - Fork 8.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ESQL] Add chart switch #177790
[ESQL] Add chart switch #177790
Conversation
f723bd7
to
c053c05
Compare
c053c05
to
dd373d3
Compare
Pinging @elastic/kibana-visualizations (Team:Visualizations) |
...hart_expressions/expression_legacy_metric/common/expression_functions/metric_vis_function.ts
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code review only. Will submit the final local check tomorrow 🚀
packages/kbn-visualization-ui-components/components/chart_switch_trigger.tsx
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@mbondyra this is fantastic 😍
I just wonder if we can hide the legacy metric from the suggestions but keep it on the switch. It works slightly different from the new metric, It will display all rows (while the new metric displays only the first) so in the case you don't run an aggregation but just render the numeric column the display is a bit weird.
I want to be hidden by the suggestions because I want to advertise in this section the charts that make sense. This is why I had hid it in the first place.
So ideally let's hide it from suggestions but allow the selection on the chart switcher. If this is difficult ping me to discuss async!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So far I've found only one bug here related to this PR:
- if I define a chart with 2 layers with different metrics, changing the chart type of the second layer will migrate the first layer configuration to the new chart type rather than the second one.
As for a general problem, but it is not specific to this PR necessarily, I see that unsaved changed are then persisted when clicking on the Edit in Lens editor
link with no easy way to get back to the original state.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for putting this together, @mbondyra! I'm excited to see this first step towards getting rid of the dueling visualization selectors in the Lens app. Leaving you a few comments below for your first review. Let me know if you have any questions or need any design/CSS support. Thanks!
-
Currently the resizable border between the ES|QL editor and chart configuration visually appears to be 2px in its idle/default state. In reality, there are just two 1px borders stacked on top of each other (one from the editor footer and one from the resizable button). Could you remove the bottom border on the editor footer to prevent this from looking like 2px?
-
The configuration toolbar currently scrolls with any overflowing layer panel(s). Would it be possible to restructure the markup or CSS in a way that would limit the scrolling container to the layer panel(s) only, thus allowing the toolbar to always be visible to the user?
-
It appears that the odd jumpy animation when closing the configuration accordion has returned. Not sure if it is directly due to this PR, but would be nice to fix if possible.
-
The padding reductions made to the accordion contents feels odd. Now the accordion contents are no longer properly aligned with other content in the flyout. If this was done to address the fact that excessive padding was previously appearing to the right of the scrollbar, I think we should try an alternative approach. The reason this padding was appearing to the right was because a parent element of the scrolling container was receiving that padding. Instead, applying the padding to the scrolling container itself (or an element within it) should correctly address the issue without adversely affecting the alignment with other flyout content. Thoughts?
-
In reading the Google doc comments, perhaps changing the current
warning
icon to thedot
icon (similar to how we do for incompatible quick functions in Lens) would make it appear less scary and distracting. Similarly, it might also be nice to reduce the "Technical preview" badge to a simplebeaker
icon with accompanying tooltip. And perhaps both can be moved to be adjacent to text (thus avoiding the jumping of icons on focus). Thoughts? I've placed a quick example mockup below. -
Also per the Google doc comments, I agree that it's a bit problematic that deleting a required dimensions doesn't properly inform the user. Would it be possible to show the standard Lens "Requires field" validation message in red as we typically do for users that outright deleted required dimensions?
packages/kbn-visualization-ui-components/components/chart_switch_trigger.tsx
Outdated
Show resolved
Hide resolved
x-pack/plugins/lens/public/app_plugin/shared/edit_on_the_fly/lens_configuration_flyout.tsx
Outdated
Show resolved
Hide resolved
@@ -0,0 +1,78 @@ | |||
/* |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@MichaelMarcialis thank you for your comments! I'll keep the responses here so we can take advantage of @markov00 proposal to keep the context in one place. First issue posted by you:
The configuration toolbar currently scrolls with any overflowing layer panel(s). Would it be possible to restructure the markup or CSS in a way that would limit the scrolling container to the layer panel(s) only, thus allowing the toolbar to always be visible to the user?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure it's a way to go. Here's how it looks right now in my Macbook 16'. The scrolling area is already extremely small for ESQL charts and making it even smaller makes it not even fit one layer configuration. I think it makes it worse for the user and I'd prefer to keep scroll as it is. Whether we decide to change it or not, I can add an issue and keep it separate from this PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's a fair point. Yeah, let's split it off as a separate issue to tackle, since it will likely require more design thought and possible layout shuffling (beyond the scope of this PR). Thanks!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done here: #178234
@@ -0,0 +1,78 @@ | |||
/* | |||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In reading the Google doc comments, perhaps changing the current
warning
icon to thedot
icon (similar to how we do for incompatible quick functions in Lens) would make it appear less scary and distracting. Similarly, it might also be nice to reduce the "Technical preview" badge to a simplebeaker
icon with accompanying tooltip. And perhaps both can be moved to be adjacent to text (thus avoiding the jumping of icons on focus). Thoughts? I've placed a quick example mockup below.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I love the beaker idea! Regarding the other one, but shouldn't be just a little scary though? We don't offer any undo functionality and user can loose a big chunk of their configuration with clicking this button. I am happy to implement it though, it's just my thoughts.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I submitted an issue here: #178232
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for splitting off as a separate issue. And I do share your concern about making a potentially destructive action less scary while we continue to operate without undo functionality. On the other hand, the warning
icon redundancy was so overwhelming in some scenarios, and something that has been bugging me for a while. Perhaps a good interim solution (until we have an undo) is to hit the user with a confirmation modal that they can optionally choose to "Never show again", similar to how we do when deleting layers? That way we ensure the user knows they are taking a destructive action without making the visualization selector overly noisy. Thoughts?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like it, let's do it! I think we can also make the warning more specific. For example.
- We can know which columns will be persisted
- Which layers will be persisted
Maybe we could use this information while thinking about this new design? I will create a small POC (outside of this PR) and link it to the issue
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I really like this proposal 😍
@@ -0,0 +1,78 @@ | |||
/* | |||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | |||
* or more contributor license agreements. Licensed under the Elastic License |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also per the Google doc comments, I agree that it's a bit problematic that deleting a required dimensions doesn't properly inform the user. Would it be possible to show the standard Lens "Requires field" validation message in red as we typically do for users that outright deleted required dimensions?
We still do it, we just don't do it when the visualization is completely cleared out. Do you think we should change it? It's possible, but I'd do it separately 🙏🏼
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I think it'd be a better user experience to show that sort of "required" error on the appropriate dimensions even when the entire configuration is cleared out. I'm fine with splitting it out as a separate issue as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done: #178237
packages/kbn-visualization-ui-components/components/chart_switch_trigger.tsx
Show resolved
Hide resolved
8a3b9e6
to
16db2b5
Compare
x-pack/plugins/lens/public/app_plugin/shared/edit_on_the_fly/layer_configuration_section.tsx
Outdated
Show resolved
Hide resolved
x-pack/plugins/lens/public/visualizations/legacy_metric/metric_suggestions.ts
Show resolved
Hide resolved
430a808
to
800a58b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great!! Thanx Marta, this PR has made my day! It is the beginning of many great enhancements and a great feature for ES|QL inline editing 😍
I tested everything and I can't find any regression
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for making those changes and creating separate issues for things that are beyond the scope of this PR, @mbondyra. It looks great. I have two final comments below, but nothing worth holding you up over. Approving now, assuming you're able to address (or split off as a separate issue).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
-
It looks like the jumpy accordion animation wasn't resolved, as I'm still seeing it in local testing. Possible to give it another look?
-
There are currently some issues present in the flyout when the user has greatly increased the height of the ES|QL editor and/or they have reduced the height of their browser's viewport:
- There are overflow issues present with the accordion arrows (see screenshot below).
- Additionally, open accordions may also be rendered so short in height that they would be unusable in some extreme scenarios. Should we attempt to prevent open accordions from getting that small (via a min-height or some other method)? Also, for the sake of responsiveness, at very small viewport heights, should we remove the accordions' existing inner content scrolling styles (allowing them to grow as tall as the height of their contents) and move the scrolling styles to a parent element of the accordions? This one may be a can of worms, so feel free to split off as a separate PR if you wish.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you Michael! As you say, it is a can of worms – I fixed the jump before but I broke another thing and then I thought I fixed it again, but it's not fixed for all the cases and broke the second thing you mention... I will submit another issue to make sure it's tested in separation 🙏🏼
💚 Build Succeeded
Metrics [docs]Module Count
Public APIs missing comments
Async chunks
Page load bundle
Unknown metric groupsAPI count
History
To update your PR or re-run it, just comment with: |
) - Resolves #167887 ## Summary On Discover page user can see a visualization for data view and ES|QL modes. For ES|QL mode it's also allowed to customize the visualization. This PR allows to save such customization together with a saved search. In more details, various types of Lens visualization can be shown on Discover page: - If in the default (data view) mode, Unified Histogram shows a "formBased" histogram (`type: UnifiedHistogramSuggestionType.histogramForDataView` in this PR) - If in the ES|QL mode, 2 scenarios are possible (so far only these are customizable): - If Lens has suggestions for the current query, Unified Histogram shows one of them (`type: UnifiedHistogramSuggestionType.lensSuggestion` in this PR) Example query: `from kibana_sample_data_logs | stats avg(bytes) by message.keyword` - If Lens suggestion list is empty, Unified Histogram shows a "textBased" histogram (`type: UnifiedHistogramSuggestionType.histogramForESQL` in this PR). Example query: `from kibana_sample_data_logs | limit 10` The main flow is that Unified Histogram first picks a suggestion (one of `UnifiedHistogramSuggestionType` type), then calculates lens attributes which are necessary to build Lens embeddable. With a saved search we are saving those calculated lens attributes under `savedSearch.visContext`. For handling this logic, I refactored `useLensSuggestion`, `getLensAttributes` into `LensVisService`. Restoring a saved customization adds complexity to the flow as it should pick now not just any available suggestion but the suggestion which matches to the previously saved lens attributes. Changes to the current query, time range, time field etc can make the current vis context incompatible and we have to drop the vis customization. This PR already includes this logic of invalidating the stored lens attributes if they are not compatible any more. New vis context will override the previous one when user presses Save for the current search. Until then, we try to restore the customization from the previously saved vis context (for example when the query changes back to the compatible one). What can invalidate the saved vis context and drop the user's customization: - data view id - data view time field name - query/filters - time range if it has a different time interval - text based columns affect what lens suggestions are available Flow of creating a new search: ![1](https://github.com/elastic/kibana/assets/1415710/9274d895-cedb-454a-9a9d-3b0cf600d801) Flow of editing a saved search: ![2](https://github.com/elastic/kibana/assets/1415710/086ce4a0-f679-4d96-892b-631bcfee7ee3) <details> <summary>Previous details</summary> - Previous approach #174373 (saving current suggestion instead of lens attributes) - Previous approach #174783 (saving lens attributes but it's based on existing hooks) But I was stuck with how to make "Unsaved changes" badge work well when user tries to revert changes. For testing in ES|QL mode I use `from kibana_sample_data_logs | limit 10` as query, customize color of a lens histogram, and save it with a saved search. Next I check 2 cases: 1. edit query limit `from kibana_sample_data_logs | limit 100`, see that vis customization gets reset which is expected, press "Revert changes" in the "Unsaved changes" badge => notice that reset did not work 2. edit only histogram color, press "Revert changes" in the "Unsaved changes" badge => notice that reset did not work Here are some nuances with the state management I am seeing which together do not allow to successfully revert unsaved changes: - For ES|QL histogram lens attributes include a modified query `from kibana_sample_data_logs | limit 10 | EVAL timestamp=DATE_TRUNC(30 second, @timestamp) | stats results = count(*) by timestamp | rename timestamp as "@timestamp every 30 second"` which means that not only changes to the original query but also a different time interval invalidates the saved lens attributes. - In ES|QL mode, `query` prop update is delayed for `UnifiedHistogramContainer` component until Discover finishes the documents fetch https://github.com/elastic/kibana/blob/fc2ec957fe78900967da26c80817aea8a0bd2c65/src/plugins/discover/public/application/main/components/layout/use_discover_histogram.ts#L346 which means that Discover should make a request on revert changes. And It's not happening for (2) as it does not make sense for Discover to trigger refetch if only `visContext` changes so we should find another way. With (1) there is another problem that Discover `visContext` state gets hijacked by lens attributes invalidation logic (as query is not sync yet to UnifiedHistogram) before fetch is completed or get [a chance to be fired](https://github.com/elastic/kibana/blob/6038f92b1fcaeedf635a0eab68fd9cdadd1103d3/src/plugins/discover/public/application/main/hooks/utils/build_state_subscribe.ts#L51-L54). I tried delaying `externalVisContext` prop update too (to keep in sync with `query` update) but it does not help https://github.com/elastic/kibana/blob/fc2ec957fe78900967da26c80817aea8a0bd2c65/src/plugins/discover/public/application/main/components/layout/use_discover_histogram.ts#L437 - Unified Histogram should signal to Discover to start a refetch when current suggestion changes https://github.com/elastic/kibana/blob/fc2ec957fe78900967da26c80817aea8a0bd2c65/src/plugins/discover/public/application/main/components/layout/use_discover_histogram.ts#L289 - for some reason this logic is required for "Revert changes" to work as it triggers the refetch. I would expect Discover on its own to notice the change in query and refetch data but it does not seem to be the case. </details> <details> <summary>Other challenges</summary> - [ ] Since we are starting to save lens attributes inside a saved search object (similar to how Dashboard saves lens vis by value), we should integrate Lens migrations into saved search migrations logic. I made a quick PoC how it could look like here jughosta@4529711 This showed that adding Lens plugin as a dependency to saved search plugin causes lots of circular deps in Kibana. To resolve that I am suggesting to spit saved search plugin into 2 plugins #174939 - not the best solution but it seems impossible to split lens plugins instead. Updates here: - [x] revert the code regarding migrations and saved search plugin split - [x] create a github issue to handle client side migrations once their API is available #179151 - [x] Discover syncs app state with URL which means that the new `visContext` (large lens attributes object) ends up in the URL too. We should exclude `visContext` from URL sync as it can make the URL too long. Updates here: we are not using appState for this any more - [x] Changes from #171081 would need to be refactored and integrated into the new `LensVisService`. - [x] Refactor after #177790 - [x] Handle a case when no chart is available for current ES|QL query - [ ] For ES|QL histogram lens attributes include a modified query `from kibana_sample_data_logs | limit 10 | EVAL timestamp=DATE_TRUNC(30 second, @timestamp) | stats results = count(*) by timestamp | rename timestamp as "@timestamp every 30 second"` which means that not only changes to the original query but also a different time range can reset the customization of lens vis as it gets a different time interval based on current time range - New update from Stratoula: - [ ] would it help to persist response of `onApplyCb` instead of lens attributes? <= the shape does not seem to be different and it works as it is so I'm keeping lens attributes - [x] use new `getLensAttributes` from #174677 </details> 10x flaky test https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/5578 ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] This was checked for [cross-browser compatibility](https://www.elastic.co/support/matrix#matrix_browsers) --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Matthias Wilhelm <matthias.wilhelm@elastic.co> Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
Summary
Solves the first part of #173222
Adds chart switcher to ESQL inline Lens editor only (only on Dashbaord):
Very slight design (paddings) changes
Added padding to not cut the icon buttons when hovered: