Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Dataset Quality] Add fix it flow for field limit #195561

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
b34b9f1
Fix refresh table breaking for summary
achyutjhunjhunwala Jun 21, 2024
d1fdaeb
Add timeout to find
achyutjhunjhunwala Jun 21, 2024
ac496c4
Merge branch 'elastic:main' into main
achyutjhunjhunwala Jun 21, 2024
12cad21
Merge branch 'elastic:main' into main
achyutjhunjhunwala Jun 21, 2024
db7d63b
Revert "Add timeout to find"
achyutjhunjhunwala Jun 21, 2024
c3a2774
Revert "Fix refresh table breaking for summary"
achyutjhunjhunwala Jun 21, 2024
9ba8128
Merge branch 'elastic:main' into main
achyutjhunjhunwala Jun 24, 2024
425cdb4
Merge branch 'elastic:main' into main
achyutjhunjhunwala Jun 25, 2024
a5ccc32
Merge branch 'elastic:main' into main
achyutjhunjhunwala Jun 26, 2024
1ab7806
Merge branch 'elastic:main' into main
achyutjhunjhunwala Jul 1, 2024
abad280
Merge branch 'elastic:main' into main
achyutjhunjhunwala Jul 3, 2024
14a3620
Merge branch 'elastic:main' into main
achyutjhunjhunwala Jul 5, 2024
0dd0eb1
Merge branch 'elastic:main' into main
achyutjhunjhunwala Jul 9, 2024
2dd64cc
Merge branch 'elastic:main' into main
achyutjhunjhunwala Jul 22, 2024
3833283
Merge branch 'elastic:main' into main
achyutjhunjhunwala Jul 23, 2024
a1a1ce1
Merge branch 'elastic:main' into main
achyutjhunjhunwala Jul 25, 2024
6278a51
Merge branch 'elastic:main' into main
achyutjhunjhunwala Aug 9, 2024
14161a3
Merge branch 'elastic:main' into main
achyutjhunjhunwala Aug 13, 2024
2df6d10
Merge branch 'elastic:main' into main
achyutjhunjhunwala Aug 13, 2024
99d8084
Merge branch 'elastic:main' into main
achyutjhunjhunwala Aug 13, 2024
28e789c
Merge branch 'elastic:main' into main
achyutjhunjhunwala Aug 14, 2024
59fc802
Merge branch 'elastic:main' into main
achyutjhunjhunwala Aug 27, 2024
dec1cae
Merge branch 'elastic:main' into main
achyutjhunjhunwala Aug 27, 2024
33c0c60
Merge branch 'elastic:main' into main
achyutjhunjhunwala Sep 4, 2024
6f5c1c6
Merge branch 'elastic:main' into main
achyutjhunjhunwala Sep 5, 2024
fbf7d08
Merge branch 'elastic:main' into main
achyutjhunjhunwala Sep 6, 2024
55d1cff
Merge branch 'elastic:main' into main
achyutjhunjhunwala Sep 17, 2024
73ae28d
Merge branch 'elastic:main' into main
achyutjhunjhunwala Sep 26, 2024
b475502
Merge branch 'elastic:main' into main
achyutjhunjhunwala Oct 3, 2024
53e9a19
Merge branch 'elastic:main' into main
achyutjhunjhunwala Oct 7, 2024
19015ad
Add code to implement Fix It flow for Field Limits
achyutjhunjhunwala Oct 9, 2024
7c7e3e0
Merge branch 'main' into add-fix-it-flow-for-field-limit
achyutjhunjhunwala Oct 9, 2024
a5f32c6
Add code logic to show a warning message for ignore_malformed
achyutjhunjhunwala Oct 9, 2024
ebc986c
Remove unwanted tests
achyutjhunjhunwala Oct 9, 2024
c8f2a1c
Merge branch 'main' into add-fix-it-flow-for-field-limit
achyutjhunjhunwala Oct 9, 2024
fd3df01
Fix text for situation where previously field_limit existed
achyutjhunjhunwala Oct 10, 2024
6875a73
Update x-pack/plugins/observability_solution/dataset_quality/common/t…
achyutjhunjhunwala Oct 10, 2024
f1ba3e7
Fix checktype issues
achyutjhunjhunwala Oct 10, 2024
9fd87ff
Fix pipeline logic for integrations and non integrations
achyutjhunjhunwala Oct 10, 2024
2727ad0
Merge branch 'main' into add-fix-it-flow-for-field-limit
achyutjhunjhunwala Oct 15, 2024
88b908c
Fix certain DOM level review comments
achyutjhunjhunwala Oct 15, 2024
6c90898
Fix locator for Management plugin
achyutjhunjhunwala Oct 15, 2024
c4f2304
Fix logic to move fix it flow to the server side
achyutjhunjhunwala Oct 16, 2024
54c07c7
Fix more review comments
achyutjhunjhunwala Oct 16, 2024
14f8dad
Fix message callout and state machine
achyutjhunjhunwala Oct 16, 2024
8887f70
Fix checking of response from settings API
achyutjhunjhunwala Oct 16, 2024
cd3fa22
Migrate and rewrote Deployment Agnostic Tests for Settings
achyutjhunjhunwala Oct 17, 2024
413373d
Add Deployment Agnostic Tests for rollover API
achyutjhunjhunwala Oct 17, 2024
e76d91e
Add Deployment Agnostic Tests for Update field Limit API
achyutjhunjhunwala Oct 17, 2024
f416124
Merge branch 'main' into add-fix-it-flow-for-field-limit
achyutjhunjhunwala Oct 17, 2024
a9baa83
Fix checktypes and lint issues
achyutjhunjhunwala Oct 17, 2024
73662db
Fix ingest pipeline design
achyutjhunjhunwala Oct 18, 2024
d04fac5
Add stateful ftr tests
achyutjhunjhunwala Oct 21, 2024
ed33c6a
Merge branch 'main' into add-fix-it-flow-for-field-limit
achyutjhunjhunwala Oct 21, 2024
2c548f8
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Oct 21, 2024
14dee9f
Fix tests for Logs DB and MKI issue
achyutjhunjhunwala Oct 21, 2024
ab3855e
Add serverless FTR tests
achyutjhunjhunwala Oct 22, 2024
de5db88
Fix Editor role permissions and tests
achyutjhunjhunwala Oct 22, 2024
5f3c371
Fix Dima's review comments
achyutjhunjhunwala Oct 22, 2024
71f0621
Simplify type definition for limits
achyutjhunjhunwala Oct 23, 2024
9e80364
Fix agnostic tests
achyutjhunjhunwala Oct 23, 2024
2f7bc4c
Fix locator changes
achyutjhunjhunwala Oct 23, 2024
6715bcf
Fix locator logic from success callout
achyutjhunjhunwala Oct 23, 2024
7e2c165
Add tests for locators
achyutjhunjhunwala Oct 23, 2024
5f8d1f9
Fix a somewhat flaky test
achyutjhunjhunwala Oct 23, 2024
9a01c07
Merge branch 'main' into add-fix-it-flow-for-field-limit
achyutjhunjhunwala Oct 23, 2024
94772d0
Fix stateful tests
achyutjhunjhunwala Oct 24, 2024
e77dc96
Fix checktype issues
achyutjhunjhunwala Oct 24, 2024
22d4080
Fix more checktype issues
achyutjhunjhunwala Oct 24, 2024
53d6564
Fix checktypes for DA tests
achyutjhunjhunwala Oct 24, 2024
aba6ffe
Refactored state machine to be more elegant
achyutjhunjhunwala Oct 24, 2024
20db933
Merge branch 'main' into add-fix-it-flow-for-field-limit
achyutjhunjhunwala Oct 24, 2024
ea2b161
Remove unused variable
achyutjhunjhunwala Oct 24, 2024
3192cc7
Fix checktypes and linting issues
achyutjhunjhunwala Oct 24, 2024
2e79d84
fix ct issues for tests
achyutjhunjhunwala Oct 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,21 @@ import type { LocatorPublic } from '@kbn/share-plugin/public';
import { ExtensionsSetup } from './services/extensions_service';
import { PublicApiServiceSetup } from './services/public_api_service';

export interface IndexManagementLocatorParams extends SerializableRecord {
page: 'data_streams_details';
dataStreamName?: string;
}
export type IndexManagementLocatorParams = SerializableRecord &
(
| {
page: 'data_streams_details';
dataStreamName?: string;
}
| {
page: 'index_template';
indexTemplate: string;
}
| {
page: 'component_template';
componentTemplate: string;
}
);

export type IndexManagementLocator = LocatorPublic<IndexManagementLocatorParams>;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import { Section } from '../../../common/constants';
import type { IndexDetailsTabId } from '../../../common/constants';
import { ExtensionsService } from '../../services/extensions_service';
import { IndexDetailsSection } from '../../../common/constants';

export const getTemplateListLink = () => `/templates`;

export const getTemplateDetailsLink = (name: string, isLegacy?: boolean) => {
Expand Down Expand Up @@ -81,6 +82,11 @@ export const getComponentTemplatesLink = (usedByTemplateName?: string) => {
}
return url;
};

export const getComponentTemplateDetailLink = (name: string) => {
return `/component_templates/${encodeURIComponent(name)}`;
};

export const navigateToIndexDetailsPage = (
indexName: string,
indicesListURLParams: string,
Expand Down
22 changes: 22 additions & 0 deletions x-pack/plugins/index_management/public/locator.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,4 +34,26 @@ describe('Index Management URL locator', () => {
});
expect(path).toBe('/data/index_management/data_streams/test');
});

test('locator returns the correct url for index_template', async () => {
const indexTemplateName = 'test@custom';
const { path } = await locator.getLocation({
page: 'index_template',
indexTemplate: indexTemplateName,
});
expect(path).toBe(
encodeURI(`/data/index_management/templates/${encodeURIComponent(indexTemplateName)}`)
);
});

test('locator returns the correct url for component_template', async () => {
const componentTemplateName = 'log@custom';
const { path } = await locator.getLocation({
page: 'component_template',
componentTemplate: componentTemplateName,
});
expect(path).toBe(
`/data/index_management/component_templates/${encodeURIComponent(componentTemplateName)}`
);
});
});
18 changes: 17 additions & 1 deletion x-pack/plugins/index_management/public/locator.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,11 @@
import { ManagementAppLocator } from '@kbn/management-plugin/common';
import { LocatorDefinition } from '@kbn/share-plugin/public';
import { IndexManagementLocatorParams } from '@kbn/index-management-shared-types';
import { getDataStreamDetailsLink } from './application/services/routing';
import {
getComponentTemplateDetailLink,
getDataStreamDetailsLink,
getTemplateDetailsLink,
} from './application/services/routing';
import { PLUGIN } from '../common/constants';

export const INDEX_MANAGEMENT_LOCATOR_ID = 'INDEX_MANAGEMENT_LOCATOR_ID';
Expand Down Expand Up @@ -37,6 +41,18 @@ export class IndexManagementLocatorDefinition
path: location.path + getDataStreamDetailsLink(params.dataStreamName!),
};
}
case 'index_template': {
return {
...location,
path: location.path + getTemplateDetailsLink(params.indexTemplate),
};
}
case 'component_template': {
return {
...location,
path: location.path + getComponentTemplateDetailLink(params.componentTemplate),
};
}
}
};
}
Original file line number Diff line number Diff line change
Expand Up @@ -134,22 +134,39 @@ export const degradedFieldAnalysisRt = rt.intersection([
type: rt.string,
ignore_above: rt.number,
}),
defaultPipeline: rt.string,
}),
]);

export type DegradedFieldAnalysis = rt.TypeOf<typeof degradedFieldAnalysisRt>;

export const dataStreamSettingsRt = rt.intersection([
export const updateFieldLimitResponseRt = rt.intersection([
rt.type({
lastBackingIndexName: rt.string,
isComponentTemplateUpdated: rt.union([rt.boolean, rt.undefined]),
isLatestBackingIndexUpdated: rt.union([rt.boolean, rt.undefined]),
customComponentTemplateName: rt.string,
}),
rt.partial({
createdOn: rt.union([rt.null, rt.number]), // rt.null is needed because `createdOn` is not available on Serverless
integration: rt.string,
datasetUserPrivileges: datasetUserPrivilegesRt,
error: rt.string,
}),
]);

export type UpdateFieldLimitResponse = rt.TypeOf<typeof updateFieldLimitResponseRt>;

export const dataStreamRolloverResponseRt = rt.type({
acknowledged: rt.boolean,
});

export type DataStreamRolloverResponse = rt.TypeOf<typeof dataStreamRolloverResponseRt>;

export const dataStreamSettingsRt = rt.partial({
lastBackingIndexName: rt.string,
indexTemplate: rt.string,
createdOn: rt.union([rt.null, rt.number]), // rt.null is needed because `createdOn` is not available on Serverless
integration: rt.string,
datasetUserPrivileges: datasetUserPrivilegesRt,
});

export type DataStreamSettings = rt.TypeOf<typeof dataStreamSettingsRt>;

export const dataStreamDetailsRt = rt.partial({
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,8 @@ export interface AnalyzeDegradedFieldsParams {
lastBackingIndex: string;
degradedField: string;
}

export interface UpdateFieldLimitParams {
dataStream: string;
newFieldLimit: number;
}
Original file line number Diff line number Diff line change
Expand Up @@ -500,3 +500,182 @@ export const degradedFieldMessageIssueDoesNotExistInLatestIndex = i18n.translate
'This issue was detected in an older version of the dataset, but not in the most recent version.',
}
);

export const possibleMitigationTitle = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigationTitle',
{
defaultMessage: 'Possible mitigation',
}
);

export const increaseFieldMappingLimitTitle = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.increaseFieldMappingLimitTitle',
{
defaultMessage: 'Increase field mapping limit',
}
);

export const fieldLimitMitigationDescriptionText = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.fieldLimitMitigationDescription',
{
defaultMessage:
'The field mapping limit sets the maximum number of fields in an index. When exceeded, additional fields are ignored. To prevent this, increase your field mapping limit.',
}
);

export const fieldLimitMitigationConsiderationText = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.fieldLimitMitigationConsiderations',
{
defaultMessage: 'Before changing the field limit, consider the following:',
}
);

export const fieldLimitMitigationConsiderationText1 = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.fieldLimitMitigationConsiderations1',
{
defaultMessage: 'Increasing the field limit could slow cluster performance.',
}
);

export const fieldLimitMitigationConsiderationText2 = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.fieldLimitMitigationConsiderations2',
{
defaultMessage: 'Increasing the field limit also resolves field limit issues for other fields.',
}
);

export const fieldLimitMitigationConsiderationText3 = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.fieldLimitMitigationConsiderations3',
{
defaultMessage:
'This change applies to the [name] component template and affects all namespaces in the template.',
}
);

export const fieldLimitMitigationConsiderationText4 = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.fieldLimitMitigationConsiderations4',
{
defaultMessage:
'You need to roll over affected data streams to apply mapping changes to component templates.',
}
);

export const fieldLimitMitigationCurrentLimitLabelText = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.fieldLimitMitigationCurrentLimitLabelText',
{
defaultMessage: 'Current limit',
}
);

export const fieldLimitMitigationNewLimitButtonText = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.fieldLimitMitigationNewLimitButtonText',
{
defaultMessage: 'New limit',
}
);

export const fieldLimitMitigationNewLimitPlaceholderText = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.fieldLimitMitigationNewLimitPlaceholderText',
{
defaultMessage: 'New field limit',
}
);

export const fieldLimitMitigationApplyButtonText = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.fieldLimitMitigationApplyButtonText',
{
defaultMessage: 'Apply',
}
);

export const otherMitigationsLoadingAriaText = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.otherMitigationsLoadingText',
{
defaultMessage: 'Loading possible mitigations',
}
);

export const otherMitigationsCustomComponentTemplate = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.otherMitigationsCustomComponentTemplate',
{
defaultMessage: 'Add or edit custom component template',
}
);

export const otherMitigationsCustomIngestPipeline = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.otherMitigationsCustomIngestPipeline',
{
defaultMessage: 'Add or edit custom ingest pipeline',
}
);

export const fieldLimitMitigationOfficialDocumentation = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.fieldLimitMitigationOfficialDocumentation',
{
defaultMessage: 'Documentation',
}
);

export const fieldLimitMitigationSuccessMessage = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.fieldLimitMitigationSuccessMessage',
{
defaultMessage: 'New limit set!',
}
);

export const fieldLimitMitigationSuccessComponentTemplateLinkText = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.fieldLimitMitigationSuccessComponentTemplateLinkText',
{
defaultMessage: 'See component template',
}
);

export const fieldLimitMitigationPartiallyFailedMessage = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.fieldLimitMitigationPartiallyFailedMessage',
{
defaultMessage: 'Changes not applied to new data',
}
);

export const fieldLimitMitigationFailedMessage = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.fieldLimitMitigationFailedMessage',
{
defaultMessage: 'Changes not applied',
}
);

export const fieldLimitMitigationFailedMessageDescription = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.fieldLimitMitigationFailedMessageDescription',
{
defaultMessage: 'Failed to set new limit',
}
);

export const fieldLimitMitigationPartiallyFailedMessageDescription = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.fieldLimitMitigationPartiallyFailedMessageDescription',
{
defaultMessage:
'The component template was successfully updated with the new field limit, but the changes were not applied to the most recent backing index. Perform a rollover to apply your changes to new data.',
}
);

export const fieldLimitMitigationRolloverButton = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.fieldLimitMitigationRolloverButton',
{
defaultMessage: 'Rollover',
}
);

export const manualMitigationCustomPipelineCopyPipelineNameAriaText = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.copyPipelineNameAriaText',
{
defaultMessage: 'Copy pipeline name',
}
);

export const manualMitigationCustomPipelineCreateEditPipelineLink = i18n.translate(
'xpack.datasetQuality.details.degradedField.possibleMitigation.createEditPipelineLink',
{
defaultMessage: 'create or edit the pipeline',
}
);
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0; you may not use this file except in compliance with the Elastic License
* 2.0.
*/

/*
* There are index templates like this metrics-apm.service_transaction.10m@template which exists.
* Hence this @ needs to be removed to derive the custom component template name.
*/
export function getComponentTemplatePrefixFromIndexTemplate(indexTemplate: string) {
if (indexTemplate.includes('@')) {
return indexTemplate.split('@')[0];
}

return indexTemplate;
}
Loading