Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up search service #53766

Merged
merged 65 commits into from
Jan 21, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
115fa6b
deprecate msearch
Dec 23, 2019
87ae717
Missing export
Dec 23, 2019
0593782
adjust tests, revert loading method of esaggs/boot
Dec 23, 2019
81ef64d
getInjectedMetadata
Dec 23, 2019
0063499
Fix jest tests
Dec 24, 2019
8e999d1
update default strategy abort test
Dec 24, 2019
fc9c714
Merge remote-tracking branch 'upstream/master' into newplatform/data/…
Jan 2, 2020
dfcb01b
notice update
Jan 2, 2020
2d8c3a3
Allow running discover errors test independently
Jan 2, 2020
c665b75
Remove batchSearches
Jan 2, 2020
726e9e3
Detect painless script error
Jan 2, 2020
4eb0b8d
don't show notifications for aborted requests
Jan 2, 2020
6950144
Merge remote-tracking branch 'upstream/master' into newplatform/data/…
Jan 5, 2020
f2510bb
Fix jest tests
Jan 5, 2020
d68de49
Restore loader indicator
Jan 5, 2020
74fc277
Decreace loading count on error
Jan 5, 2020
27e95df
update search test
Jan 5, 2020
ea0213f
Trigger digest after fetching fresh index patterns
Jan 5, 2020
0aa065c
Merge remote-tracking branch 'upstream/master' into newplatform/data/…
Jan 6, 2020
056ef7d
Revert isEqual
Jan 6, 2020
d568a0b
accurate revert
Jan 6, 2020
7f013a0
Merge remote-tracking branch 'upstream/master' into newplatform/data/…
Jan 7, 2020
cf51262
Return full error details to client from search endpoint
Jan 7, 2020
563b27e
Re-throw AbortError from http when user aborts request.
Jan 7, 2020
9a9ddcb
fix typo
Jan 7, 2020
99f0382
typo
Jan 7, 2020
5ad68ea
Adjust routes jest test
Jan 7, 2020
d93f307
Restore msearch using a separate es connection
Jan 8, 2020
c68fcd0
typescript fixes
Jan 8, 2020
d0005b7
set http service mock
Jan 8, 2020
d803363
Move es client to dat aplugin, for follow up PR
Jan 9, 2020
21c84fb
Merge remote-tracking branch 'upstream/master' into newplatform/data/…
Jan 9, 2020
917b91a
Add karma mock
Jan 9, 2020
b093823
Merge remote-tracking branch 'upstream/master' into newplatform/data/…
Jan 9, 2020
c2f375a
krma mock
Jan 9, 2020
9604030
fix tests
Jan 9, 2020
9ff4373
ts
Jan 9, 2020
95ebf09
Pass in version dynamically
Jan 9, 2020
14e2842
add headers to esClient host
Jan 9, 2020
f0ec6a9
Merge remote-tracking branch 'upstream/master' into newplatform/data/…
Jan 12, 2020
1e1c470
Restored fetch soon test
Jan 12, 2020
ca9dea0
Cleanup search params
Jan 12, 2020
30b9478
Cleanup search params test
Jan 12, 2020
0ae8750
Merge remote-tracking branch 'upstream/master' into newplatform/data/…
Jan 13, 2020
8057d9a
Merge remote-tracking branch 'upstream/master' into newplatform/data/…
Jan 13, 2020
e849b2c
Revert "Cleanup search params"
Jan 13, 2020
3ff6d3c
Revert "Cleanup search params test"
Jan 13, 2020
60393a2
Revert code to use old es client until #44302 is resolved
Jan 13, 2020
f94aaa7
Revert changes to getPainlessError
Jan 13, 2020
defd787
Fix jest test
Jan 13, 2020
7de985b
Merge branch 'master' into newplatform/data/search-msearch
elasticmachine Jan 14, 2020
f9abb2d
Merge remote-tracking branch 'upstream/master' into newplatform/data/…
Jan 15, 2020
4f0b90c
Merge remote-tracking branch 'upstream/master' into newplatform/data/…
Jan 15, 2020
be0a9c6
Refactor esClient to trigger loadingIndicator
Jan 15, 2020
9ad7561
Merge branch 'newplatform/data/search-msearch' of github.com:lizozom/…
Jan 15, 2020
aa0506a
fixing tests
Jan 15, 2020
45f5f78
Merge remote-tracking branch 'upstream/master' into newplatform/data/…
Jan 16, 2020
2ade3a6
use esClient from searchService
Jan 16, 2020
6b80ecf
git remove comment
Jan 16, 2020
0e8c21c
fix jest
Jan 16, 2020
487c64e
Merge remote-tracking branch 'upstream/master' into newplatform/data/…
Jan 17, 2020
7ac79fd
Merge remote-tracking branch 'upstream/master' into newplatform/data/…
Jan 19, 2020
2677b3b
Merge branch 'master' into newplatform/data/search-msearch
elasticmachine Jan 20, 2020
dd5d32d
Merge remote-tracking branch 'upstream/master' into newplatform/data/…
Jan 21, 2020
faddcd3
Merge branch 'newplatform/data/search-msearch' of github.com:lizozom/…
Jan 21, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion src/core/public/http/fetch.ts
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,11 @@ export class Fetch {
try {
response = await window.fetch(request);
} catch (err) {
throw new HttpFetchError(err.message, request);
if (err.name === 'AbortError') {
lizozom marked this conversation as resolved.
Show resolved Hide resolved
throw err;
} else {
throw new HttpFetchError(err.message, request);
}
}

const contentType = response.headers.get('Content-Type') || '';
Expand Down
19 changes: 17 additions & 2 deletions src/legacy/core_plugins/data/public/plugin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,24 @@
import { CoreSetup, CoreStart, Plugin } from 'kibana/public';
import { SearchService, SearchStart } from './search';
import { DataPublicPluginStart } from '../../../../plugins/data/public';
import { ExpressionsSetup } from '../../../../plugins/expressions/public';

import {
setFieldFormats,
setNotifications,
setIndexPatterns,
setQueryService,
setSearchService,
setUiSettings,
setInjectedMetadata,
setHttp,
// eslint-disable-next-line @kbn/eslint/no-restricted-paths
} from '../../../../plugins/data/public/services';

export interface DataPluginSetupDependencies {
expressions: ExpressionsSetup;
}

export interface DataPluginStartDependencies {
data: DataPublicPluginStart;
}
Expand All @@ -54,18 +63,24 @@ export interface DataStart {
* or static code.
*/

export class DataPlugin implements Plugin<void, DataStart, {}, DataPluginStartDependencies> {
export class DataPlugin
implements Plugin<void, DataStart, DataPluginSetupDependencies, DataPluginStartDependencies> {
private readonly search = new SearchService();

public setup(core: CoreSetup) {}
public setup(core: CoreSetup) {
setInjectedMetadata(core.injectedMetadata);
}

public start(core: CoreStart, { data }: DataPluginStartDependencies): DataStart {
// This is required for when Angular code uses Field and FieldList.
setFieldFormats(data.fieldFormats);
setQueryService(data.query);
setSearchService(data.search);
setIndexPatterns(data.indexPatterns);
setFieldFormats(data.fieldFormats);
setNotifications(core.notifications);
setUiSettings(core.uiSettings);
setHttp(core.http);

return {
search: this.search.start(core),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@

import { set } from 'lodash';
// @ts-ignore
import { createFilter } from '../../../core_plugins/visualizations/public';
import { FormattedData } from './adapters';
import { createFilter } from '../../../../visualizations/public';
import { FormattedData } from '../../../../../../plugins/inspector/public';

interface Column {
id: string;
Expand Down
10 changes: 3 additions & 7 deletions src/legacy/core_plugins/data/public/search/expressions/esaggs.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,21 +34,17 @@ import {
getTime,
FilterManager,
} from '../../../../../../plugins/data/public';
import {
SearchSource,
ISearchSource,
getRequestInspectorStats,
getResponseInspectorStats,
} from '../../../../../ui/public/courier';

import { buildTabularInspectorData } from '../../../../../ui/public/inspector/build_tabular_inspector_data';
import { buildTabularInspectorData } from './build_tabular_inspector_data';
import { calculateObjectHash } from '../../../../visualizations/public';
// @ts-ignore
import { tabifyAggResponse } from '../../../../../ui/public/agg_response/tabify/tabify';
import { PersistedState } from '../../../../../ui/public/persisted_state';
import { Adapters } from '../../../../../../plugins/inspector/public';
// eslint-disable-next-line @kbn/eslint/no-restricted-paths
import { getQueryService, getIndexPatterns } from '../../../../../../plugins/data/public/services';
import { ISearchSource, getRequestInspectorStats, getResponseInspectorStats } from '../..';
import { SearchSource } from '../search_source';

export interface RequestHandlerParams {
searchSource: ISearchSource;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ describe('callClient', () => {

test('Passes the additional arguments it is given to the search strategy', () => {
const searchRequests = [{ _searchStrategyId: 0 }];
const args = { es: {}, config: {}, esShardTimeout: 0 } as FetchHandlers;
const args = { searchService: {}, config: {}, esShardTimeout: 0 } as FetchHandlers;

callClient(searchRequests, [], args);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ import { SearchRequest } from '../types';
export function callClient(
searchRequests: SearchRequest[],
requestsOptions: FetchOptions[] = [],
{ es, config, esShardTimeout }: FetchHandlers
fetchHandlers: FetchHandlers
) {
// Correlate the options with the request that they're associated with
const requestOptionEntries: Array<[
Expand All @@ -53,9 +53,7 @@ export function callClient(
// then an error would have been thrown above
const { searching, abort } = searchStrategy!.search({
searchRequests: requests,
es,
config,
esShardTimeout,
...fetchHandlers,
});

requests.forEach((request, i) => {
Expand Down
10 changes: 5 additions & 5 deletions src/legacy/core_plugins/data/public/search/fetch/fetch_soon.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@ import { SearchRequest, SearchResponse } from '../types';
export async function fetchSoon(
request: SearchRequest,
options: FetchOptions,
{ es, config, esShardTimeout }: FetchHandlers
fetchHandlers: FetchHandlers
) {
const msToDelay = config.get('courier:batchSearches') ? 50 : 0;
return delayedFetch(request, options, { es, config, esShardTimeout }, msToDelay);
const msToDelay = fetchHandlers.config.get('courier:batchSearches') ? 50 : 0;
return delayedFetch(request, options, fetchHandlers, msToDelay);
}

/**
Expand Down Expand Up @@ -64,7 +64,7 @@ let fetchInProgress: Promise<SearchResponse> | null = null;
async function delayedFetch(
request: SearchRequest,
options: FetchOptions,
{ es, config, esShardTimeout }: FetchHandlers,
fetchHandlers: FetchHandlers,
ms: number
) {
const i = requestsToFetch.length;
Expand All @@ -73,7 +73,7 @@ async function delayedFetch(
const responses = await (fetchInProgress =
fetchInProgress ||
delay(() => {
const response = callClient(requestsToFetch, requestOptions, { es, config, esShardTimeout });
const response = callClient(requestsToFetch, requestOptions, fetchHandlers);
requestsToFetch = [];
requestOptions = [];
fetchInProgress = null;
Expand Down
3 changes: 2 additions & 1 deletion src/legacy/core_plugins/data/public/search/fetch/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
* under the License.
*/

import { ISearchStart } from 'src/plugins/data/public';
import { IUiSettingsClient } from '../../../../../../core/public';
import { SearchRequest, SearchResponse } from '../types';

Expand All @@ -35,7 +36,7 @@ export interface FetchOptions {
}

export interface FetchHandlers {
es: ApiCaller;
searchService: ISearchStart;
config: IUiSettingsClient;
esShardTimeout: number;
}
Original file line number Diff line number Diff line change
Expand Up @@ -19,19 +19,34 @@

import { SearchSource } from '../search_source';
import { IndexPattern } from '../../../../../../plugins/data/public';

jest.mock('ui/new_platform');
import {
setSearchService,
setUiSettings,
setInjectedMetadata,
// eslint-disable-next-line @kbn/eslint/no-restricted-paths
} from '../../../../../../plugins/data/public/services';

import {
injectedMetadataServiceMock,
uiSettingsServiceMock,
} from '../../../../../../core/public/mocks';

setUiSettings(uiSettingsServiceMock.createStartContract());
setInjectedMetadata(injectedMetadataServiceMock.createSetupContract());
setSearchService({
search: jest.fn(),
__LEGACY: {
esClient: {
search: jest.fn(),
msearch: jest.fn(),
},
},
});

jest.mock('../fetch', () => ({
fetchSoon: jest.fn().mockResolvedValue({}),
}));

jest.mock('ui/chrome', () => ({
dangerouslyGetActiveInjector: () => ({
get: jest.fn(),
}),
}));

const getComputedFields = () => ({
storedFields: [],
scriptFields: [],
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -70,19 +70,21 @@
*/

import _ from 'lodash';
import { npSetup } from 'ui/new_platform';
import chrome from 'ui/chrome';
import { normalizeSortRequest } from './normalize_sort_request';
import { fetchSoon } from '../fetch';
import { fieldWildcardFilter } from '../../../../../../plugins/kibana_utils/public';
import { getHighlightRequest, esFilters, esQuery } from '../../../../../../plugins/data/public';
import { RequestFailure } from '../fetch/errors';
import { filterDocvalueFields } from './filter_docvalue_fields';
import { SearchSourceOptions, SearchSourceFields, SearchRequest } from './types';
import { FetchOptions, ApiCaller } from '../fetch/types';
import { FetchOptions } from '../fetch/types';

const esShardTimeout = npSetup.core.injectedMetadata.getInjectedVar('esShardTimeout') as number;
const config = npSetup.core.uiSettings;
import {
getSearchService,
getUiSettings,
getInjectedMetadata,
// eslint-disable-next-line @kbn/eslint/no-restricted-paths
lizozom marked this conversation as resolved.
Show resolved Hide resolved
} from '../../../../../../plugins/data/public/services';
stacey-gammon marked this conversation as resolved.
Show resolved Hide resolved

export type ISearchSource = Pick<SearchSource, keyof SearchSource>;

Expand Down Expand Up @@ -192,21 +194,23 @@ export class SearchSource {
* @async
*/
async fetch(options: FetchOptions = {}) {
const $injector = await chrome.dangerouslyGetActiveInjector();
const es = $injector.get('es') as ApiCaller;

await this.requestIsStarting(options);

const searchRequest = await this.flatten();
this.history = [searchRequest];

const esShardTimeout = getInjectedMetadata().getInjectedVar('esShardTimeout') as number;
lizozom marked this conversation as resolved.
Show resolved Hide resolved
const response = await fetchSoon(
searchRequest,
{
...(this.searchStrategyId && { searchStrategyId: this.searchStrategyId }),
...options,
},
{ es, config, esShardTimeout }
{
searchService: getSearchService(),
config: getUiSettings(),
esShardTimeout,
}
);

if (response.error) {
Expand Down Expand Up @@ -313,7 +317,11 @@ export class SearchSource {
case 'source':
return addToBody('_source', val);
case 'sort':
const sort = normalizeSortRequest(val, this.getField('index'), config.get('sort:options'));
const sort = normalizeSortRequest(
val,
this.getField('index'),
getUiSettings().get('sort:options')
);
return addToBody(key, sort);
default:
return addToBody(key, val);
Expand Down Expand Up @@ -359,7 +367,7 @@ export class SearchSource {

if (body._source) {
// exclude source fields for this index pattern specified by the user
const filter = fieldWildcardFilter(body._source.excludes, config.get('metaFields'));
const filter = fieldWildcardFilter(body._source.excludes, getUiSettings().get('metaFields'));
body.docvalue_fields = body.docvalue_fields.filter((docvalueField: any) =>
filter(docvalueField.field)
);
Expand All @@ -377,11 +385,11 @@ export class SearchSource {
_.set(body, '_source.includes', remainingFields);
}

const esQueryConfigs = esQuery.getEsQueryConfig(config);
const esQueryConfigs = esQuery.getEsQueryConfig(getUiSettings());
body.query = esQuery.buildEsQuery(index, query, filters, esQueryConfigs);

if (highlightAll && body.query) {
body.highlight = getHighlightRequest(body.query, config.get('doc_table:highlight'));
body.highlight = getHighlightRequest(body.query, getUiSettings().get('doc_table:highlight'));
delete searchRequest.highlightAll;
}

Expand Down
Loading