[ES|QL] Bypass no data views screen #174316

clintandrewhall · 2024-01-05T03:34:15Z

Derived from #173068
Further addresses #169873

Summary

We're adding a link to the No Data Prompt to send a person to explore their data using ES|QL if there exists any ingested data but no data view.
This PR populates the query bar with the first available index (some special handling for logs index. 2a9edec)
All consumers of the prompt/no data view are updated in this PR.
~~There's an issue where, if you're in Discover, clicking the link won't refresh the page. .~~ This is fixed on Discover side by reinitializing the state container when user clicks the "try es|ql" link and URL state updates.
There is an issue that you can save the es|ql chart from Discover, but Dashboard's empty screen blocks the navigation because data views don't exist [ES|QL] Bypass no data views screen #174316 (comment). This is fixed by allowing the dashboard to work without the default data view. Hopefully, this won't lead to major issues
~~ES|QL panels can't be created without the default data views~~ this is fixed by trying to fallback to an ad-hoc dataview, plan to move that code to the utils introduced here [ES|QL] Use same adhoc dataviews for queries with the same index pattern #174736
fix circular deps
Add functional tests

Visuals

esql.empty.prompt.demo.mov

Storybook Stories

Can access ES|QL

Cannot access (e.g. preview is unavailable - not implemented)

cc: @stratoula @mattkime @thomasneirynck

stratoula · 2024-01-05T07:36:42Z

Thanx Clint! Looks nice. I feel though that navigating the users to the show functions is not the correct path forward.
I am informed that I dont have a dataview but I do have data. The component navigates me to Discover to start exploring. I find much more useful to have a query that will actually help me to explore my data. from index | limit 10.

I am doing exactly the same on the global search so you can check my implementation there.

src/plugins/discover/common/esql_locator.ts

MichaelMarcialis · 2024-01-05T19:46:04Z

@clintandrewhall: Regarding design support, I recall @andreadelrio sharing some design concepts for this a week or so ago, so I have a feeling she has something in mind for how to present these ES|QL links. I believe she's on PTO today, but will be back next week.

@andreadelrio: Hopefully I'm remembering correctly, but let me know if that's not the case, and I can happily lend some design assistance here as needed.

src/plugins/discover/common/esql_locator.ts

src/plugins/discover/public/application/main/discover_main_route.tsx

andreadelrio · 2024-01-10T01:20:41Z

Hi all, sending a couple of ideas of how we can handle this as discussed before

No illustrations

With illustrations

Notes:

The illustration for the ES|QL card was found in a blog post. You can consider it a placeholder.
All copy needs to be reviewed by one of our UX writers.

cc @clintandrewhall @ninoslavmiskovic @stratoula

clintandrewhall · 2024-01-11T01:08:30Z

@rshen91 @stratoula @andreadelrio I think we should ship the locator and related logic in this PR and hold off on the UX, rather than continue to rebase this until the UX is decided.

We can iterate in code on the different proposals using Storybook, and ship the changes with only Shared UX and design really being involved, (from a PR approval perspective, that is).

Thoughts?

stratoula · 2024-01-11T06:41:08Z

@clintandrewhall makes sense and I am fine with that. This means that is going to look like this right?

stratoula · 2024-01-11T07:23:01Z

@clintandrewhall fixed the logic here 2a9edec

You can remove the dataviewSpec, it works without it 🎸

clintandrewhall · 2024-01-11T12:43:06Z

@clintandrewhall makes sense and I am fine with that. This means that is going to look like this right?

Actually, I'm suggesting we don't ship and UI change at all in this PR, just provide the locator and related logic. We'll leave any UX implementation to the next PR.

stratoula · 2024-01-11T12:46:18Z

@clintandrewhall got it! I am fine with this approach too 👍

ninoslavmiskovic · 2024-02-01T18:58:39Z

I like that @nreese !

lukasolson

LGTM, one minor point below. Also, would be nice if someone else on the team can approve the state changes (resetStateContainer), but I personally don't see an issue.

lukasolson · 2024-02-01T20:27:29Z

src/plugins/discover/common/esql_locator.ts

+    const { discoverAppLocator, getIndices } = this.deps;
+
+    const indexName = await getIndexForESQLQuery({ dataViews: { getIndices } });
+    const esql = `from ${indexName ?? '*'} | limit 10`;


Not sure if this should be fixed in this PR, but the indexName can contain special characters (like |). Is there some sanitization that we can add here?

Really? I get this error when I try to create one:

Invalid index name [my|index|1], must not contain the following characters ['\',' ','"','<','*','?','>','|',',','/']

stratoula · 2024-02-02T06:55:54Z

/ci

stratoula · 2024-02-02T07:24:46Z

@Dosant amazing job! You are super fast! This is a very important improvement imho. Being able to query your data or even create visualizations with a dataview is huge! I am ready to approve, just waiting for the presentation team to chime in.

Would it make more sense to have this open a new dashboard with a new inline lens panel editor open so users can just start creating an ES|QL lens chart directly in dashboard?

This would be a very good idea. We will need ui actions here I think, it is not possible from a locator (or is it and I am missing something?)

I was hoping that the @elastic/kibana-presentation team can take the dashboard behavior as a follow up. I was thinking that we can navigate to a dashboard as Nathan says (and having the ES|QL panel open would be also fantastic) and then have only active the panels that do not require a dataview. These must be ES|QL, text, image, vega and maps. Now every time you click to Lens you get this no data view that I dont necessarily like. Lens should be inactive, works only with dataviews for now etc. But I think this is a bigger project and out of the scope of this PR. We can discuss further as an analyst XP team!

I think for now as a first iteration and as this is something that multiple people have complained about it makes sense. It navigates to Discover which is the best application we have to learn ES|QL. So it is ok, can this be better for dashboards? Absolutely!!

With that being said, I am fine with that if the rest of the gang is. @Dosant I think we need to prioritize the redesign of this component. There are 2 things I don't like:

ES|QL should be promoted as equivalent to dataviews
We only have docs for dataviews while I feel a link to ES|QL docs would be awesome here.

Also can we remove the ES|QL button when you specifically are in the dataviews management app?

Dosant · 2024-02-02T11:37:34Z

@nreese, @ninoslavmiskovic: Dashboard presents users with screen below. Clicking "Try ES|QL" takes users to discover. Would it make more sense to have this open a new dashboard with a new inline lens panel editor open so users can just start creating a ES|QL lens chart directly in dashboard?

@stratoula: I was thinking that we can navigate to a dashboard as Nathan says (and having the ES|QL panel open would be also fantastic) and then have only active the panels that do not require a dataview.
I think for now as a first iteration and as this is something that multiple people have complained about it makes sense. It navigates to Discover which is the best application we have to learn ES|QL. So it is ok, can this be better for dashboards? Absolutely!!

@nreese, @ninoslavmiskovic, agree with @stratoula, I am afraid to extend the scope of this slipped pr further with the ff coming up. But the suggestion is totally how I'd like it to work. I'd like to create an issue and will see if this is something shared-ux can also do

@stratoula: @Dosant I think we need to prioritize the redesign of this component. There are 2 things I don't like:

ES|QL should be promoted as equivalent to dataviews

We only have docs for dataviews while I feel a link to ES|QL docs would be awesome here.

I'll create an issue for the @andreadelrio's proposal #174316 (comment). Not sure if I will follow up with it straight away, but will see.

As of the docs,
@ryankeairns, @andreadelrio, maybe you'd have a suggestion how can we low-effort add an ES|QL docs link to the design in this PR?

…t_esql # Conflicts: # packages/kbn-esql-utils/index.ts

…bana into shared_ux/prompt_esql

…t_esql

nreese · 2024-02-02T14:42:28Z

@nreese, @ninoslavmiskovic, agree with @stratoula, I am afraid to extend the scope of this slipped pr further with the ff coming up. But the suggestion is totally how I'd like it to work. I'd like to create an issue and will see if this is something shared-ux can also do

agree. I have created #176159 to track

nreese

kibana-presentation changes LGTM
code review and tested in chrome

stratoula

Visualizations team changes LGTM! Thanx for this PR Anton. I did a brief testing again and everything works as expected. :)

…t_esql

kibana-ci · 2024-02-05T14:08:18Z

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`dashboard`	424	429	+5
`dataViewManagement`	190	195	+5
`discover`	709	715	+6
`kibanaOverview`	120	125	+5
`lens`	1252	1257	+5
`visualizations`	397	402	+5
total			+31

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/deeplinks-analytics`	5	6	+1
`@kbn/es-query`	201	202	+1
`@kbn/esql-utils`	11	13	+2
`@kbn/shared-ux-prompt-no-data-views`	11	14	+3
`@kbn/shared-ux-prompt-no-data-views-mocks`	16	23	+7
`discover`	95	104	+9
total			+23

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`dashboard`	383.7KB	385.7KB	+2.0KB
`dataViewManagement`	136.3KB	137.9KB	+1.6KB
`discover`	570.1KB	571.7KB	+1.6KB
`kibanaOverview`	50.1KB	51.9KB	+1.8KB
`lens`	1.4MB	1.4MB	+2.2KB
`visualizations`	271.3KB	273.1KB	+1.8KB
total			+11.0KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
`@kbn/shared-ux-prompt-no-data-views`	0	1	+1
`@kbn/shared-ux-prompt-no-data-views-types`	5	6	+1
`share`	10	11	+1
total			+3

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`discover`	31.5KB	33.0KB	+1.5KB
`kbnUiSharedDeps-srcJs`	2.9MB	2.9MB	+110.0B
total			+1.6KB

Unknown metric groups

API count

id	before	after	diff
`@kbn/deeplinks-analytics`	5	6	+1
`@kbn/es-query`	261	263	+2
`@kbn/esql-utils`	11	14	+3
`@kbn/shared-ux-page-analytics-no-data`	18	19	+1
`@kbn/shared-ux-prompt-no-data-views`	26	34	+8
`@kbn/shared-ux-prompt-no-data-views-mocks`	17	24	+7
`discover`	141	151	+10
total			+32

History

💔 Build #191304 failed 757d820
💔 Build #191294 failed 641acdc
💚 Build #191068 succeeded 790c3c4
💔 Build #191056 failed 672e8c9
💔 Build #191013 failed 379bc71
💚 Build #190949 succeeded c2ab32a

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @andreadelrio @Dosant

> Derived from elastic#173068 > Further addresses elastic#169873 ## Summary - We're adding a link to the No Data Prompt to send a person to explore their data using ES|QL if there exists any ingested data but no data view. - This PR populates the query bar with the first available index (some special handling for logs index. elastic@2a9edec) - All consumers of the prompt/no data view are updated in this PR. - [x] ~There's an issue where, if you're in Discover, clicking the link won't refresh the page. .~ This is fixed on Discover side by reinitializing the state container when user clicks the "try es|ql" link and URL state updates. - [x] ~There is an issue that you can save the es|ql chart from Discover, but Dashboard's empty screen blocks the navigation because data views don't exist elastic#174316 (comment). This is fixed by allowing the dashboard to work without the default data view. Hopefully, this won't lead to major issues - [x] ~ES|QL panels can't be created without the default data views~ this is fixed by trying to fallback to an ad-hoc dataview, plan to move that code to the utils introduced here elastic#174736 - [x] fix circular deps - [x] Add functional tests ## Visuals https://github.com/elastic/kibana/assets/7784120/af3592c1-f4c8-43bb-a128-3268b7761367 ### Storybook Stories #### Can access ES|QL ![Screenshot 2024-01-31 at 17 05 47](https://github.com/elastic/kibana/assets/7784120/370d0351-198e-4dc3-b22e-86f497ad4df5) #### Cannot access (e.g. preview is unavailable - _not implemented_) ![Screenshot 2024-01-31 at 17 05 59](https://github.com/elastic/kibana/assets/7784120/c2bf52ab-9fa8-4f25-9e5d-512d4f4342fa) --------- Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co> Co-authored-by: Rachel Shen <rshen@elastic.co> Co-authored-by: Anton Dosov <anton.dosov@elastic.co>

clintandrewhall assigned rshen91 Jan 5, 2024

clintandrewhall changed the title ~~[No Data Prompt] ES|QL direct link~~ [shared-ux] ES|QL direct link Jan 5, 2024

clintandrewhall commented Jan 5, 2024

View reviewed changes

src/plugins/discover/common/esql_locator.ts Outdated Show resolved Hide resolved

clintandrewhall commented Jan 5, 2024

View reviewed changes

src/plugins/discover/common/esql_locator.ts Show resolved Hide resolved

rshen91 reviewed Jan 5, 2024

View reviewed changes

src/plugins/discover/public/application/main/discover_main_route.tsx Show resolved Hide resolved

clintandrewhall added 2 commits January 9, 2024 12:00

[No Data Prompt] ES|QL direct link

8dec605

Convert handler to hook; simplify services; set appropriate index

b75c9ee

clintandrewhall force-pushed the shared_ux/prompt_esql branch 2 times, most recently from 6d48c06 to b75c9ee Compare January 9, 2024 17:06

clintandrewhall and others added 3 commits January 9, 2024 12:07

Merge branch 'main' into shared_ux/prompt_esql

b590f50

Fix query logic

e8204f0

Address feedback

8ff3b72

andreadelrio self-assigned this Jan 10, 2024

Fix locator

2a9edec

lukasolson approved these changes Feb 1, 2024

View reviewed changes

Merge branch 'main' into shared_ux/prompt_esql

c2ab32a

Dosant added 8 commits February 2, 2024 12:44

Merge branch 'main' of github.com:elastic/kibana into shared_ux/promp…

2a52aea

…t_esql # Conflicts: # packages/kbn-esql-utils/index.ts

fix merge

af27b8e

don’t show “try es|ql” on data view managament page

ef4d45f

Merge branch 'shared_ux/prompt_esql' of github.com:clintandrewhall/ki…

379bc71

…bana into shared_ux/prompt_esql

don’t show “try es|ql” on data view managament page

57c363b

Merge branch 'main' of github.com:elastic/kibana into shared_ux/promp…

45117b8

…t_esql

remove unwanted change

672e8c9

add ftr

790c3c4

nreese mentioned this pull request Feb 2, 2024

[dashboard][ES|QL] Update no data views ES|QL action to open new dashboard with new inline lens panel #176159

Closed

nreese approved these changes Feb 2, 2024

View reviewed changes

stratoula approved these changes Feb 2, 2024

View reviewed changes

Dosant added 4 commits February 5, 2024 11:15

Merge branch 'main' of github.com:elastic/kibana into shared_ux/promp…

5dd1f76

…t_esql

add esql doc link

641acdc

fix link

757d820

fix dashboard esql link

60df991

Dosant merged commit 450f9f6 into elastic:main Feb 5, 2024
20 checks passed

kibanamachine added the backport:skip This commit does not require backporting label Feb 5, 2024

Dosant mentioned this pull request Feb 6, 2024

[ES|QL] Present ES|QL as an equal to data views on the "no data views" screen #176291

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ES|QL] Bypass no data views screen #174316

[ES|QL] Bypass no data views screen #174316

clintandrewhall commented Jan 5, 2024 •

edited by Dosant

Loading

stratoula commented Jan 5, 2024

MichaelMarcialis commented Jan 5, 2024

andreadelrio commented Jan 10, 2024

clintandrewhall commented Jan 11, 2024

stratoula commented Jan 11, 2024

stratoula commented Jan 11, 2024 •

edited

Loading

clintandrewhall commented Jan 11, 2024

stratoula commented Jan 11, 2024

ninoslavmiskovic commented Feb 1, 2024

lukasolson left a comment

lukasolson Feb 1, 2024

stratoula Feb 2, 2024

stratoula commented Feb 2, 2024

stratoula commented Feb 2, 2024 •

edited

Loading

Dosant commented Feb 2, 2024

nreese commented Feb 2, 2024

nreese left a comment

stratoula left a comment

kibana-ci commented Feb 5, 2024

API count

[ES|QL] Bypass no data views screen #174316

[ES|QL] Bypass no data views screen #174316

Conversation

clintandrewhall commented Jan 5, 2024 • edited by Dosant Loading

Summary

Visuals

Storybook Stories

Can access ES|QL

Cannot access (e.g. preview is unavailable - not implemented)

stratoula commented Jan 5, 2024

MichaelMarcialis commented Jan 5, 2024

andreadelrio commented Jan 10, 2024

clintandrewhall commented Jan 11, 2024

stratoula commented Jan 11, 2024

stratoula commented Jan 11, 2024 • edited Loading

clintandrewhall commented Jan 11, 2024

stratoula commented Jan 11, 2024

ninoslavmiskovic commented Feb 1, 2024

lukasolson left a comment

Choose a reason for hiding this comment

lukasolson Feb 1, 2024

Choose a reason for hiding this comment

stratoula Feb 2, 2024

Choose a reason for hiding this comment

stratoula commented Feb 2, 2024

stratoula commented Feb 2, 2024 • edited Loading

Dosant commented Feb 2, 2024

nreese commented Feb 2, 2024

nreese left a comment

Choose a reason for hiding this comment

stratoula left a comment

Choose a reason for hiding this comment

kibana-ci commented Feb 5, 2024

💚 Build Succeeded

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Public APIs missing exports

Page load bundle

API count

History

clintandrewhall commented Jan 5, 2024 •

edited by Dosant

Loading

stratoula commented Jan 11, 2024 •

edited

Loading

stratoula commented Feb 2, 2024 •

edited

Loading