[APM] Serverless onboarding

[gbamparop]

Dependency on https://elasticco.atlassian.net/browse/CP-2403 for getting the managed service url.

The UX for APM serverless onboarding will be very similar to the existing tutorial.

As the onboarding experience is likely to change soon (see design explorations), the work that will be done as part of this issue will be focused on the necessary changes for the serverless offering.

• The section for APM Server / Fleet will be removed
• Hide the RUM agent tab
• The Elastic Agent policy selector will be removed
• The APM managed service url will be displayed in the configuration settings and code snippets. We will need a way to access this in the UI (see thread) (internal)
• An API key will be created with a similar approach as logs onboarding
• References to secret tokens in the configuration settings and code snippets will be replaced by api keys (example of Node.js agent config)
• Required assets need to be installed (index templates, ingest pipelines, DLM).

Note: The following screenshot is for reference only and will be discussed further with @boriskirov.

The existing onboarding flow can be found at /app/home#/tutorial/apm.

https://github.com/elastic/logs-dev/issues/17

• Add Custom Tutorials - [APM] Serverless Onboarding with Custom Tutorials #158228
• Add Agent Status check for Serverless (It was not present for cloud, so initially not added for Serverless) - Add Agent status action #159227
• Add E2E tests - Will be delivered as part of the same PR as above Add Agent status action #159227
• Remove default values for Service Name and Environment - [APM] remove default service name and environment #159901

[elasticmachine]

Pinging @elastic/apm-ui (Team:APM)

[achyutjhunjhunwala]

@gbamparop @sqren @dgieselaar I did the analysis for this change.
We have 2 options -

1. Currently the way tutorials work is they expect 3 set of instructions. We pass 2 of them
   • On Premise
   • Elastic Cloud
   • On Prem Elastic Cloud (For this we don't provide any instructions)

For these 2 configs, we provide our set of instructions (steps and tabs) which get loaded on the UI. We can ask Tutorials team to provide us one more config for Serverless. This would make our lives much simpler as we can add our new instructions and they would load just like they load for the other 2.

2. We need to create our own Tutorials Page (literally cloning everything they have got with our customisations). Then when user lands on app/home#/tutorial/apm we check if its on serverless mode and force Redirect the user to our custom page. Not sure how good this would be

[gbamparop]

This would make our lives much simpler as we can add our new instructions and they would load just like they load for the other 2.

As we will probably be planning to move the new onboarding flow outside of the tutorial framework, having another set of instructions might be an overhead for now, but would be happy to hear other opinions. Also, will this support our needs for creating API keys and adding them in the config table / code snippets?

2. we check if its on serverless mode and force Redirect the user to our custom page. Not sure how good this would be

Instead of redirecting the users from the tutorial, can we select the appropriate link before the no data screen and related components render?

[sorenlouv]

We can ask Tutorials team to provide us one more config for Serverless

There is no tutorials team. Anything we need changed in the home plugin we should do ourselves.

[achyutjhunjhunwala]

@gbamparop I am yet to analyse the part of API Keys creation. I will check the button logic which they have to see how that works in case i can find something.

Regarding the second point, i am not clear if i understands it correctly. The way i am reaching tutorials is from Kibana Home -> Add Integrations -> APM. The moment i click on APM, that would render the Tutorial.

@sqren We can do the change to load a new type of Instructions, but i don't understand yet how is the environment variable set. Somehow the environment variable telling that if its a serverless deployment or not has to be set by Kibana Platform somewhere.

[sorenlouv]

Regarding the second point, i am not clear if i understands it correctly. The way i am reaching tutorials is from Kibana Home -> Add Integrations -> APM. The moment i click on APM, that would render the Tutorial.

I think this is because we have registered the APM tutorial:

kibana/x-pack/plugins/apm/server/plugin.ts

Lines 150 to 156 in deafe4a

	plugins.home?.tutorials.registerTutorial(
	tutorialProvider({
	apmConfig: currentConfig,
	apmIndices: indices,
	cloud: plugins.cloud,
	isFleetPluginEnabled: !isEmpty(resourcePlugins.fleet),
	})

We can make that conditional, so we don't do that on serverless. I think we instead should register a custom integration like enterprise_search does it (note the uiInternalPath):

kibana/x-pack/plugins/enterprise_search/server/integrations.ts

Lines 365 to 385 in deafe4a

	if (config.hasWebCrawler) {
	customIntegrations.registerCustomIntegration({
	id: 'web_crawler',
	title: i18n.translate('xpack.enterpriseSearch.integrations.webCrawlerName', {
	defaultMessage: 'Web crawler',
	}),
	description: i18n.translate('xpack.enterpriseSearch.integrations.webCrawlerDescription', {
	defaultMessage: 'Add search to your website with the Enterprise Search web crawler.',
	}),
	categories: ['enterprise_search', 'app_search', 'web', 'elastic_stack', 'crawler'],
	uiInternalPath: '/app/enterprise_search/content/search_indices/new_index/crawler',
	icons: [
	{
	type: 'eui',
	src: 'logoEnterpriseSearch',
	},
	],
	shipper: 'enterprise_search',
	isBeta: false,
	});
	}

[achyutjhunjhunwala]

Capturing the discussions here

1. We will start with the assumption that serverlessFlag is available from the ymlfile for now
2. Instead of using the Home App, we will use Custom Integrations to register APM app for serverless.
3. Custom new route to be added on the frontend, which would replicate the code to display the current existing onboarding view.
4. When no data is present, APM -> Service page currently shows a button to APM Integrations which is managed by Fleet, this will change to our new UI

[achyutjhunjhunwala]

@boriskirov During the APM UI Weekly call, 2 important feedbacks were provided which could be interesting for the new Onboarding Flow (Out of scope for this ticket.)

1. The Service Name and Environment variables are redundant and not mandatory, which means instead of showing default value of my-service-name and my-environment, we can introduce them as input field at the table level and then if they provide some value, we can update the code snippet at the bottom.

2. The table currently displays Configuration Key and Value. The configuration key is mostly for the code. But our agent also supports Environment names, so we can also display configuration option using Environment names in the table.

[achyutjhunjhunwala]

@chrisdistasio @akhileshpok The Tutorials display an additional step for onPrem setup called Agent Status which is not present during Cloud setup.
I am hoping that even in Serverless we will not be having this status check ?

cc @sqren @gbamparop

[chrisdistasio]

hi @achyutjhunjhunwala thanks for bringing that to attention...what does it show (im actually unfamiliar with that button)?

[sorenlouv]

I am hoping that even in Serverless we will not be having this status check ?

That step is also useful on serverless so I think we should keep it.

what does it show (im actually unfamiliar with that button)?

Clicking the button will check for the existence of APM data in ES. This way users can verify that the agent is correctly set up and data is being ingested.

I don't think it's a lot of work to implement the agent check. Mostly it's a question about design.

[achyutjhunjhunwala]

@sqren Do you know why we don't have this for cloud then ?

[chrisdistasio]

thanks @sqren . this is valuable feedback for customers. this is desirable to have if we can make it work for serverless. i would classify it as nice to have for mvp (since it doesnt exist today for cloud). if it's light work to add it, we should; if it requires a lot more investigation or is technically constrained, we can reconsider for post-mvp.

@achyutjhunjhunwala i wonder if direct access to es on cloud is prevented in cloud?

this is similar user interaction feedback that exists in the new k8s and logs onboarding where user is show "waiting for data" progress until data is confirmed.

[sorenlouv]

@sqren Do you know why we don't have this for cloud then ?

That's new to me. I thought we had the agent check both on prem and on cloud.

[achyutjhunjhunwala]

@chrisdistasio @sqren Thank you for quickly clarifying the doubts.

Chris - I will analyse how big of a change it is and the comment if we can add it or needs to wait. Regarding the direct access to ES on cloud, i will have to check how Home App was doing it.

[achyutjhunjhunwala]

@sqren @gbamparop I analysed the effort for adding the Agent Status button.
We will have to implement a API on our side to fire the ES query and then call it from the UI. This PR itself is quite huge and i would not like to over burden it with API changes. Is it fine if i create a follow up ticket and then work on it, to keep this PR smaller.

There is one more way, which is calling the Home App API directly, but i would refrain from doing that, rather have our own API which lives inside our own solution rather than an external plugin.

[gbamparop]

thanks @sqren . this is valuable feedback for customers. this is desirable to have if we can make it work for serverless. i would classify it as nice to have for mvp (since it doesnt exist today for cloud). if it's light work to add it, we should; if it requires a lot more investigation or is technically constrained, we can reconsider for post-mvp.

@achyutjhunjhunwala i wonder if direct access to es on cloud is prevented in cloud?

this is similar user interaction feedback that exists in the new k8s and logs onboarding where user is show "waiting for data" progress until data is confirmed.

Thanks for looking into this @achyutjhunjhunwala, I think it's ok to have a follow-up on this.

[achyutjhunjhunwala]

Testing Index Templates installed

Path - app/management/data/index_management/templates

[achyutjhunjhunwala]

Testing Ingest Pipelines

Path - app/management/ingest/ingest_pipelines

Screen.Recording.2023-06-21.at.12.20.30.mov

[achyutjhunjhunwala]

Testing Create API key without Access

Screen.Recording.2023-06-21.at.14.55.02.mov

[achyutjhunjhunwala]

Testing ILM Policies

On Serverless Cluster, ILM policies for APM seems to be missing

Regular Cloud Cluster they are present

[achyutjhunjhunwala]

Testing Serverless Onboarding Experience -

Screen.Recording.2023-06-21.at.11.59.53.mov

1. Navigation from Cloud to Landing Page to Onboarding ✅
2. All Tabs load properly with Managed Service URL ✅
3. Check Agent Status - With No data ✅
4. Check Agent Status - With Data ✅ link
5. Create API Key - With Access ✅
6. Create API Key - With No Access ✅ link
7. Index Templates Installed ✅ link
8. Ingest Pipelines ✅ link
9. DLM 🔴 DLM and ILM is not expected in Serverless
10. No RUM ✅
11. No Policy Selector ✅
12. No APM Server block ✅