Skip to content

Commit

Permalink
[6.x] [CCR] Follower index CRUD (#27936) (#30148)
Browse files Browse the repository at this point in the history
* [CCR] Follower index CRUD (#27936)

* [CCR] Refactor redux for Auto-follow pattern detail panel (#27491)

* [CCR] Refactor redux for Auto-follow pattern detail panel

* [CCR] Small refactor

* [CCR] Change to present tense

* [CCR] Display auto-follow pattern name even if it does not exist

* [CCR] Use href to edit auto-follow pattern + remove middelware to update "pattern" query params

* [CCR] Fix navigation back bug + set 2 ids for detail and edit an auto-follow pattern

* [CCR] Replace api middleware with redux-thunk action

* [CCR] Show detail footer close button even when cluster is not valid

* [CCR] Add endpoints for fetching and creating follower indices (#27646)

* Add GET /follower_indices endpoint with deserialization logic and tests.

* Add POST /follower_indices endpoint with serialization logic and tests.

* [CCR]  Add unit tests for RemoteClusterForm, RemoteClusterList, and RemoteClusterTable (#27647)

* Use componentDidUpdate instead of getDerivedStateFromProps.

* Add unit tests for RemoteClusterForm, RemoteClusterList, and RemoteClusterTable.

* Add jest mock for eui `makeId()` utility and get deterministic aria IDs for snapshots

* Update snapshot for Remote Cluster list test

* [CCR] Follower indices table and detail panel (#27804)

* Store for follower indices

* Initial work for follower indices table and detail panel

* Fix load auto-follow stats load as middleware was removed

* [CCR] Create follower index UI form (#27864)

* Initial setup Follower Index form

* Working form without client validation

* Add client side validation for follower index

* Add client validation to check if index already exist

* Improve error message when leader index does not exist

* Remove update method for follower index

* Clear api error on field change

* Fix i18n error

* Update snapshots

* [CCR] Add pause, resume, and unfollow actions for follower indices (#28305)

* Add pause and resume follower index routes

* Add unfollow route

* Add api methods for new routes

* Adjust routes to have bulk capabilities, add corresponding actions

* Refresh list after pausing/resuming, remove items after unfollowing

* First pass at UI for pause and unfollow (and resume, but that is not visible due to ES stats response)

* Handle additional conditions needed for unfollowing leader index, add placeholder code to deduce paused status

* PR feedback

* [CCR] Advanced settings UI for follower indices (#28267)

* Add client side validation of advanced settings form
* Move form entry row to separate component
* Add server side serialization for advanced settings
* Ignore advanced settings input when that section is hidden.
  - Cache and restore input when the section is shown again.

* [CCR] Show remote cluster validation in CCR forms, and add edit follower index (#28778)

* [CCR] Advanced settings component

* Remove preview active on toggle settings

* Add client side validation of advanced settings form

* Move form entry row to separate component

* Add title to panel

* Add i18n translation of advanced settings

* Update Follower index form with toggle for advanced settings

* Add server side serialisation for advanced settings

* Make code review changes

* Fix test: mock constant dependency

* Add section to edit follower index

* Show confirm modal before updating follower index

* Add edit icon in table + update server endpoint to pause / resume

* [CCR] Show remote cluster validation in follower index form & auto-follow pattern form

* PR feedback, cleanup form sizes, add redirect to edit remote cluster

* Fix routing, remove unused code, adjust auto follow pattern edit loading error page

* Adjust error messages and make remote cluster not found edit page the same

* Fix functionality as result of merge

* Fix validation, reorder actions, fix tests, and address feedback

* PR feedback and fix validation pt 2

* Adjust remote cluster validation

* Fix i18n

* Fix api error not showing on add follower form

* [CCR]  Integrate new follower index info ES API (#29153)

* Integrate new follower index info ES API

* Collate data from follower stats and info apis when retrieving all followers and single follower
* Add follower settings info to detail panel
* Add paused/active UI state
* Surface follower default settings to UI

* Adjust tests

* Address PR feedback

* Update snapshots

* [CCR] Surface license errors in-app and refine permissions error UI. (#29228)

* Fix camelcasing bug in XPackInfo.
* Silently swallow API error when checking for index name availability.
* Fix typo in followerIndexForm fatal error.
* Add permissions check before allowing user to access the app.

* Refine wording of CCR permission denied page, to specify cluster privileges. (#29533)

* [CCR] Improve form error handling and general UX (#29419)

* Remove unnecessary eslint disable-line

* [CCR] Implement Advanced Settings design feedback (#29543)

* Use EuiSwitch to toggle advanced settings in Create Follower Index form.
* Move 'optional' from each Advanced Setting field to the section heading.
* Change Advanced Settings switch label and description to emphasize that you can customize them or use the defaults.
* Prepopulate Advanced Settings fields with default values.
* When editing a follower index, check if advanced settings have been edited and open them if so.
* Add 'Reset to default' button below advanced settings fields if their values are different than the default.
* Remove 'Default' copy from Advanced Settings descriptions.
* Simplify toggleAdvancedSettings function, add comments, and fix React console error.

* [CCR] Follower index list fixes from design feedback (#29484)

* Delete remote cluster settings before updating

* Fix detail panel z-index

* Remove default descriptor from follower index detail panel setting values

* Follower index confirm action copy adjustments

* Change z-index styling to use EUI vars

* [CCR] Improve remote clusters test coverage (#29487)

* Add Jest test for RemoteClusterForm validation state.
* Extract validation functions out of RemoteClusterForm and add unit tests.
  - Return null instead of undefined from validators.
* Add unit tests for different types of remote clusters in RemoteClusterTable.
* Add unit test for RemoteClusterList empty prompt.
* Add tests verifying behavior for row link, row delete button, and detail panel delete button.
  - Use getRouterLinkProps to assign onClick and href to edit buttons in row and detail panel.

* [CCR] Adjust spacing around descriptions in list views, link to transport port docs, etc (#29789)

* Adjust spacing around description around descriptions in list views so that it's even on top and bottom.
* Add link to transport port docs from Remote Cluster form.
* Move 'View in Index Management' link from the detail panel body into the footer.

* Re-order follower index form sections: remote cluster, leader index, follower index. (#29885)

* Fix deep-linking to follower index after creating/updating it. (#29865)

* [CCR] Copy edits (#29676)

* Use 'Resume/pause data replication' in context menu and row actions.
* Update copy of 'Update' confirm modal for a paused follower index.
* Update copy of 'Update' confirm modal for an active follower index.
* Update copy of 'Pause data replication' confirm modal.
* Update copy of 'Resume data replication' confirm modal.
* Update copy for permissions check.
* Update copy of table empty state.
* Update copy around tables.
* Update form copy.
* Update copy for RemoteClustersFormField callouts.
* Convert 'data replication' -> 'replication'.
* Update copy for Unfollow confirm modal.
* Update copy for form API error and Auto-follow Patterns table.
* Update form save button labels to be 'Create' and 'Update'.
* Move API errors to bottom of form, into same position as sync validation errors. Remove spacer from SectionError implementation.

* [CCR] Open index after unfollowing leader (#29887)

* Open index after unfollowing leader, fix some variable names

* Fix typo

* Add comment

* [CCR] IE and Screen reader accesibility (#29731)

* Fix api endpoit for auto-follow stats

* Prevent letter wrapping in IE for the Remote cluster "connection" table column

* Move inline style to CSS class to fix IE flex bug

* [CCR] Add callout to paused follower index detail panel (#30013)

* Add callout to paused follower index detail panel

* Update copy

* Skip call to ccr stats api if follower index is paused (#30027)

* [CCR] Add integration tests for follower indices (#30064)

* [CCR] Add integration tests for follower indices

* Import advanced settings value from app constants

* Disable flaky follower indices API integration tests.
  • Loading branch information
jen-huang authored Feb 6, 2019
1 parent cde31bf commit 8557222
Show file tree
Hide file tree
Showing 129 changed files with 8,865 additions and 1,213 deletions.
7 changes: 6 additions & 1 deletion src/ui/public/indices/constants/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,4 +19,9 @@

import { INDEX_PATTERN_ILLEGAL_CHARACTERS_VISIBLE } from 'ui/index_patterns';

export const INDEX_ILLEGAL_CHARACTERS_VISIBLE = INDEX_PATTERN_ILLEGAL_CHARACTERS_VISIBLE.concat(',');
export const INDEX_ILLEGAL_CHARACTERS_VISIBLE = [ ...INDEX_PATTERN_ILLEGAL_CHARACTERS_VISIBLE ];

// Insert the comma into the middle, so it doesn't look as if it has grammatical meaning when
// these characters are rendered in the UI.
const insertionIndex = Math.floor(INDEX_PATTERN_ILLEGAL_CHARACTERS_VISIBLE.length / 2);
INDEX_ILLEGAL_CHARACTERS_VISIBLE.splice(insertionIndex, 0, ',');
Original file line number Diff line number Diff line change
Expand Up @@ -8,3 +8,4 @@ export const BASE_PATH = '/management/elasticsearch/cross_cluster_replication';
export const BASE_PATH_REMOTE_CLUSTERS = '/management/elasticsearch/remote_clusters';
export const API_BASE_PATH = '/api/cross_cluster_replication';
export const API_REMOTE_CLUSTERS_BASE_PATH = '/api/remote_clusters';
export const API_INDEX_MANAGEMENT_BASE_PATH = '/api/index_management';
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@
export * from './plugin';
export * from './base_path';
export * from './app';
export * from './settings';
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

export const FOLLOWER_INDEX_ADVANCED_SETTINGS = {
maxReadRequestOperationCount: 5120,
maxOutstandingReadRequests: 12,
maxReadRequestSize: '32mb',
maxWriteRequestOperationCount: 5120,
maxWriteRequestSize: '9223372036854775807b',
maxOutstandingWriteRequests: 9,
maxWriteBufferCount: 2147483647,
maxWriteBufferSize: '512mb',
maxRetryDelay: '500ms',
readPollTimeout: '1m',
};
12 changes: 12 additions & 0 deletions x-pack/plugins/cross_cluster_replication/common/services/utils.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,3 +15,15 @@ export const wait = (time = 1000) => (data) => {
setTimeout(() => resolve(data), time);
});
};

/**
* Utility to remove empty fields ("") from a request body
*/
export const removeEmptyFields = (body) => (
Object.entries(body).reduce((acc, [key, value]) => {
if (value !== '') {
acc[key] = value;
}
return acc;
}, {})
);
159 changes: 159 additions & 0 deletions x-pack/plugins/cross_cluster_replication/fixtures/follower_index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,159 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

const Chance = require('chance'); // eslint-disable-line import/no-extraneous-dependencies
const chance = new Chance();

export const getFollowerIndexStatsMock = (
name = chance.string(),
shards = [{
id: chance.string(),
remoteCluster: chance.string(),
leaderIndex: chance.string(),
leaderGlobalCheckpoint: chance.integer(),
leaderMaxSequenceNum: chance.integer(),
followerGlobalCheckpoint: chance.integer(),
followerMaxSequenceNum: chance.integer(),
lastRequestedSequenceNum: chance.integer(),
outstandingReadRequestsCount: chance.integer(),
outstandingWriteRequestsCount: chance.integer(),
writeBufferOperationsCount: chance.integer(),
writeBufferSizeBytes: chance.integer(),
followerMappingVersion: chance.integer(),
followerSettingsVersion: chance.integer(),
totalReadTimeMs: chance.integer(),
totalReadRemoteExecTimeMs: chance.integer(),
successfulReadRequestCount: chance.integer(),
failedReadRequestsCount: chance.integer(),
operationsReadCount: chance.integer(),
bytesReadCount: chance.integer(),
totalWriteTimeMs: chance.integer(),
successfulWriteRequestsCount: chance.integer(),
failedWriteRequestsCount: chance.integer(),
operationsWrittenCount: chance.integer(),
readExceptions: [ chance.string() ],
timeSinceLastReadMs: chance.integer(),
}]
) => {
const serializeShard = ({
id,
remoteCluster,
leaderIndex,
leaderGlobalCheckpoint,
leaderMaxSequenceNum,
followerGlobalCheckpoint,
followerMaxSequenceNum,
lastRequestedSequenceNum,
outstandingReadRequestsCount,
outstandingWriteRequestsCount,
writeBufferOperationsCount,
writeBufferSizeBytes,
followerMappingVersion,
followerSettingsVersion,
totalReadTimeMs,
totalReadRemoteExecTimeMs,
successfulReadRequestCount,
failedReadRequestsCount,
operationsReadCount,
bytesReadCount,
totalWriteTimeMs,
successfulWriteRequestsCount,
failedWriteRequestsCount,
operationsWrittenCount,
readExceptions,
timeSinceLastReadMs,
}) => ({
shard_id: id,
remote_cluster: remoteCluster,
leader_index: leaderIndex,
leader_global_checkpoint: leaderGlobalCheckpoint,
leader_max_seq_no: leaderMaxSequenceNum,
follower_global_checkpoint: followerGlobalCheckpoint,
follower_max_seq_no: followerMaxSequenceNum,
last_requested_seq_no: lastRequestedSequenceNum,
outstanding_read_requests: outstandingReadRequestsCount,
outstanding_write_requests: outstandingWriteRequestsCount,
write_buffer_operation_count: writeBufferOperationsCount,
write_buffer_size_in_bytes: writeBufferSizeBytes,
follower_mapping_version: followerMappingVersion,
follower_settings_version: followerSettingsVersion,
total_read_time_millis: totalReadTimeMs,
total_read_remote_exec_time_millis: totalReadRemoteExecTimeMs,
successful_read_requests: successfulReadRequestCount,
failed_read_requests: failedReadRequestsCount,
operations_read: operationsReadCount,
bytes_read: bytesReadCount,
total_write_time_millis: totalWriteTimeMs,
successful_write_requests: successfulWriteRequestsCount,
failed_write_requests: failedWriteRequestsCount,
operations_written: operationsWrittenCount,
read_exceptions: readExceptions,
time_since_last_read_millis: timeSinceLastReadMs,
});

return {
index: name,
shards: shards.map(serializeShard),
};
};

export const getFollowerIndexListStatsMock = (total = 3, names) => {
const list = {
follow_stats: {
indices: [],
},
};

for(let i = 0; i < total; i++) {
list.follow_stats.indices.push(getFollowerIndexStatsMock(names[i]));
}

return list;
};

export const getFollowerIndexInfoMock = (
name = chance.string(),
status = chance.string(),
parameters = {
maxReadRequestOperationCount: chance.string(),
maxOutstandingReadRequests: chance.string(),
maxReadRequestSize: chance.string(),
maxWriteRequestOperationCount: chance.string(),
maxWriteRequestSize: chance.string(),
maxOutstandingWriteRequests: chance.string(),
maxWriteBufferCount: chance.string(),
maxWriteBufferSize: chance.string(),
maxRetryDelay: chance.string(),
readPollTimeout: chance.string(),
}
) => {
return {
follower_index: name,
status,
max_read_request_operation_count: parameters.maxReadRequestOperationCount,
max_outstanding_read_requests: parameters.maxOutstandingReadRequests,
max_read_request_size: parameters.maxReadRequestSize,
max_write_request_operation_count: parameters.maxWriteRequestOperationCount,
max_write_request_size: parameters.maxWriteRequestSize,
max_outstanding_write_requests: parameters.maxOutstandingWriteRequests,
max_write_buffer_count: parameters.maxWriteBufferCount,
max_write_buffer_size: parameters.maxWriteBufferSize,
max_retry_delay: parameters.maxRetryDelay,
read_poll_timeout: parameters.readPollTimeout,
};
};

export const getFollowerIndexListInfoMock = (total = 3) => {
const list = {
follower_indices: [],
};

for(let i = 0; i < total; i++) {
list.follower_indices.push(getFollowerIndexInfoMock());
}

return list;
};
7 changes: 7 additions & 0 deletions x-pack/plugins/cross_cluster_replication/fixtures/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,10 @@ export {
} from './auto_follow_pattern';

export { esErrors } from './es_errors';

export {
getFollowerIndexStatsMock,
getFollowerIndexListStatsMock,
getFollowerIndexInfoMock,
getFollowerIndexListInfoMock,
} from './follower_index';
7 changes: 7 additions & 0 deletions x-pack/plugins/cross_cluster_replication/public/app/_app.scss
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,10 @@
.ccrFollowerIndicesHelpText {
transform: translateY(-3px);
}

/**
* 1. Prevent context menu popover appearing above confirmation modal
*/
.ccrFollowerIndicesDetailPanel {
z-index: $euiZMask - 1; /* 1 */
}
Loading

0 comments on commit 8557222

Please sign in to comment.