Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrations v2: don't auto-create indices + FTR/esArchiver support #85778

Merged
merged 33 commits into from
Feb 1, 2021
Merged

Migrations v2: don't auto-create indices + FTR/esArchiver support #85778

Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
c91b296
Migrations V2 on by default
rudolf Dec 17, 2020
c82a595
esArchiver delete migrations v2 indices
rudolf Jan 21, 2021
cebb9c0
Fix saved_objects_management api_integration tests
rudolf Jan 21, 2021
a9a9355
Try to fix v2 migrations for pre-release builds
rudolf Jan 21, 2021
aba78bd
esArchiver delete auto-created v2 migration indices like .kibana_8.0.0
rudolf Jan 21, 2021
a1a1567
Try to fix v2 migrations for pre-release builds
rudolf Jan 22, 2021
3d76664
Use require_alias to prevent auto-created saved objects index
rudolf Jan 25, 2021
2a991a0
Wrap SO routes until core logs all internal errors
rudolf Jan 25, 2021
2edc8ce
Fix api_integration tests requiring an empty kibana index
rudolf Jan 26, 2021
be152a5
Delete corrupt saved object from lens archives
rudolf Jan 26, 2021
f971b3e
Update docs
rudolf Jan 26, 2021
46a3f35
Merge branch 'master' into so-migrations-on-by-default
rudolf Jan 26, 2021
30cf7fc
Fix ui_settings tests
rudolf Jan 26, 2021
4e71702
Fix core jest tests
rudolf Jan 26, 2021
bb8dc12
Fix type errors
rudolf Jan 26, 2021
b806398
Fix accessibility tests
rudolf Jan 26, 2021
117e992
Fix plugin functional tests
rudolf Jan 26, 2021
7898f0c
Fix api_integration tests after merging in master
rudolf Jan 26, 2021
55b1461
Fix plugin functional tests #2
rudolf Jan 27, 2021
0387fa3
EsArchiver: Don't reset ui settings after the .kibana index was deleted
rudolf Jan 27, 2021
03b4f1b
Fix functional management/visualize tests
rudolf Jan 27, 2021
b82616d
Fix oss security functional tests
rudolf Jan 27, 2021
329b9c6
EsArchiver clean task manager indices to fix alerting api integration…
rudolf Jan 27, 2021
9baf383
migrationsv2 correctly handle unknown saved object type mappings
rudolf Jan 28, 2021
34c5d87
Revert "Try to fix v2 migrations for pre-release builds"
rudolf Jan 28, 2021
bb4f238
Revert "Try to fix v2 migrations for pre-release builds"
rudolf Jan 28, 2021
07809d5
Merge branch 'master' into so-migrations-on-by-default
rudolf Jan 28, 2021
4cbda7f
Re-enable v2 migrations in tests after merging in master
rudolf Jan 28, 2021
1b2dbd9
Try to fix async dashboard functional test
rudolf Jan 29, 2021
e95c441
Merge branch 'master' into so-migrations-on-by-default
rudolf Jan 29, 2021
9cdd1d6
Restore UiSettings defaults after emptyKibanaIndex()
rudolf Jan 29, 2021
6f8972b
Merge branch 'master' into so-migrations-on-by-default
rudolf Feb 1, 2021
0da675d
Review feedback: rename test to match behaviour
rudolf Feb 1, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions ...na-plugin-core-server.savedobjectserrorhelpers.createindexaliasnotfounderror.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [SavedObjectsErrorHelpers](./kibana-plugin-core-server.savedobjectserrorhelpers.md) &gt; [createIndexAliasNotFoundError](./kibana-plugin-core-server.savedobjectserrorhelpers.createindexaliasnotfounderror.md)

## SavedObjectsErrorHelpers.createIndexAliasNotFoundError() method

<b>Signature:</b>

```typescript
static createIndexAliasNotFoundError(alias: string): DecoratedError;
```

## Parameters

| Parameter | Type | Description |
| --- | --- | --- |
| alias | <code>string</code> | |

<b>Returns:</b>

`DecoratedError`

23 changes: 23 additions & 0 deletions ...-plugin-core-server.savedobjectserrorhelpers.decorateindexaliasnotfounderror.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [SavedObjectsErrorHelpers](./kibana-plugin-core-server.savedobjectserrorhelpers.md) &gt; [decorateIndexAliasNotFoundError](./kibana-plugin-core-server.savedobjectserrorhelpers.decorateindexaliasnotfounderror.md)

## SavedObjectsErrorHelpers.decorateIndexAliasNotFoundError() method

<b>Signature:</b>

```typescript
static decorateIndexAliasNotFoundError(error: Error, alias: string): DecoratedError;
```

## Parameters

| Parameter | Type | Description |
| --- | --- | --- |
| error | <code>Error</code> | |
| alias | <code>string</code> | |

<b>Returns:</b>

`DecoratedError`

22 changes: 22 additions & 0 deletions ...ore/server/kibana-plugin-core-server.savedobjectserrorhelpers.isgeneralerror.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
<!-- Do not edit this file. It is automatically generated by API Documenter. -->

[Home](./index.md) &gt; [kibana-plugin-core-server](./kibana-plugin-core-server.md) &gt; [SavedObjectsErrorHelpers](./kibana-plugin-core-server.savedobjectserrorhelpers.md) &gt; [isGeneralError](./kibana-plugin-core-server.savedobjectserrorhelpers.isgeneralerror.md)

## SavedObjectsErrorHelpers.isGeneralError() method

<b>Signature:</b>

```typescript
static isGeneralError(error: Error | DecoratedError): boolean;
```

## Parameters

| Parameter | Type | Description |
| --- | --- | --- |
| error | <code>Error &#124; DecoratedError</code> | |

<b>Returns:</b>

`boolean`

3 changes: 3 additions & 0 deletions docs/development/core/server/kibana-plugin-core-server.savedobjectserrorhelpers.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ export declare class SavedObjectsErrorHelpers
| [createBadRequestError(reason)](./kibana-plugin-core-server.savedobjectserrorhelpers.createbadrequesterror.md) | <code>static</code> | |
| [createConflictError(type, id, reason)](./kibana-plugin-core-server.savedobjectserrorhelpers.createconflicterror.md) | <code>static</code> | |
| [createGenericNotFoundError(type, id)](./kibana-plugin-core-server.savedobjectserrorhelpers.creategenericnotfounderror.md) | <code>static</code> | |
| [createIndexAliasNotFoundError(alias)](./kibana-plugin-core-server.savedobjectserrorhelpers.createindexaliasnotfounderror.md) | <code>static</code> | |
| [createInvalidVersionError(versionInput)](./kibana-plugin-core-server.savedobjectserrorhelpers.createinvalidversionerror.md) | <code>static</code> | |
| [createTooManyRequestsError(type, id)](./kibana-plugin-core-server.savedobjectserrorhelpers.createtoomanyrequestserror.md) | <code>static</code> | |
| [createUnsupportedTypeError(type)](./kibana-plugin-core-server.savedobjectserrorhelpers.createunsupportedtypeerror.md) | <code>static</code> | |
Expand All @@ -27,6 +28,7 @@ export declare class SavedObjectsErrorHelpers
| [decorateEsUnavailableError(error, reason)](./kibana-plugin-core-server.savedobjectserrorhelpers.decorateesunavailableerror.md) | <code>static</code> | |
| [decorateForbiddenError(error, reason)](./kibana-plugin-core-server.savedobjectserrorhelpers.decorateforbiddenerror.md) | <code>static</code> | |
| [decorateGeneralError(error, reason)](./kibana-plugin-core-server.savedobjectserrorhelpers.decorategeneralerror.md) | <code>static</code> | |
| [decorateIndexAliasNotFoundError(error, alias)](./kibana-plugin-core-server.savedobjectserrorhelpers.decorateindexaliasnotfounderror.md) | <code>static</code> | |
| [decorateNotAuthorizedError(error, reason)](./kibana-plugin-core-server.savedobjectserrorhelpers.decoratenotauthorizederror.md) | <code>static</code> | |
| [decorateRequestEntityTooLargeError(error, reason)](./kibana-plugin-core-server.savedobjectserrorhelpers.decoraterequestentitytoolargeerror.md) | <code>static</code> | |
| [decorateTooManyRequestsError(error, reason)](./kibana-plugin-core-server.savedobjectserrorhelpers.decoratetoomanyrequestserror.md) | <code>static</code> | |
Expand All @@ -35,6 +37,7 @@ export declare class SavedObjectsErrorHelpers
| [isEsCannotExecuteScriptError(error)](./kibana-plugin-core-server.savedobjectserrorhelpers.isescannotexecutescripterror.md) | <code>static</code> | |
| [isEsUnavailableError(error)](./kibana-plugin-core-server.savedobjectserrorhelpers.isesunavailableerror.md) | <code>static</code> | |
| [isForbiddenError(error)](./kibana-plugin-core-server.savedobjectserrorhelpers.isforbiddenerror.md) | <code>static</code> | |
| [isGeneralError(error)](./kibana-plugin-core-server.savedobjectserrorhelpers.isgeneralerror.md) | <code>static</code> | |
| [isInvalidVersionError(error)](./kibana-plugin-core-server.savedobjectserrorhelpers.isinvalidversionerror.md) | <code>static</code> | |
| [isNotAuthorizedError(error)](./kibana-plugin-core-server.savedobjectserrorhelpers.isnotauthorizederror.md) | <code>static</code> | |
| [isNotFoundError(error)](./kibana-plugin-core-server.savedobjectserrorhelpers.isnotfounderror.md) | <code>static</code> | |
Expand Down
3 changes: 2 additions & 1 deletion packages/kbn-es-archiver/src/actions/empty_kibana_index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,5 +25,6 @@ export async function emptyKibanaIndexAction({

await cleanKibanaIndices({ client, stats, log, kibanaPluginIds });
await migrateKibanaIndex({ client, kbnClient });
return stats;
stats.createdIndex('.kibana');
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An existing bug. By adding it to the created stats, the extendEsArchiver will restore any UiSettings defaults after a test calls emptyKibanaIndex()

return stats.toJSON();
}
2 changes: 1 addition & 1 deletion packages/kbn-es-archiver/src/es_archiver.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ export class EsArchiver {
* @return Promise
*/
async emptyKibanaIndex() {
await emptyKibanaIndexAction({
return await emptyKibanaIndexAction({
client: this.client,
log: this.log,
kbnClient: this.kbnClient,
Expand Down
8 changes: 5 additions & 3 deletions packages/kbn-es-archiver/src/lib/indices/kibana_index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,9 @@ export async function migrateKibanaIndex({
*/
async function fetchKibanaIndices(client: Client) {
const kibanaIndices = await client.cat.indices({ index: '.kibana*', format: 'json' });
const isKibanaIndex = (index: string) => /^\.kibana(:?_\d*)?$/.test(index);
const isKibanaIndex = (index: string) =>
/^\.kibana(:?_\d*)?$/.test(index) ||
/^\.kibana(_task_manager)?_(pre)?\d+\.\d+\.\d+/.test(index);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

esArchiver previously didn't handle the task manager index

return kibanaIndices.map((x: { index: string }) => x.index).filter(isKibanaIndex);
}

Expand All @@ -103,7 +105,7 @@ export async function cleanKibanaIndices({

while (true) {
const resp = await client.deleteByQuery({
index: `.kibana`,
index: `.kibana,.kibana_task_manager`,
body: {
query: {
bool: {
Expand All @@ -115,7 +117,7 @@ export async function cleanKibanaIndices({
},
},
},
ignore: [409],
ignore: [404, 409],
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some tests don't load the task manager index, so we need to ignore it if this index doesn't exist

});

if (resp.total !== resp.deleted) {
Expand Down
Loading