[Ingest] cleanup names and files/paths to confirm to a standard format (

elastic#41773)

* [Maps] Rename modules for clarity (elastic#41608)

* [Docs] Add simple phrase highlighting to Logs UI (elastic#41610)

* [Docs] Add simple phrase highlighting to Logs UI

* Fix heading level

* [DOCS] Timelion cleanup (elastic#41381)

* [Canvas] Removes doc links from backticks. (elastic#41601)

* Upgrade EUI to 12.4.0 (elastic#41577)

* eui 12.4.0

* styled-components ts conflict

* update combobox test service to always open on open call

* Revert "update combobox test service to always open on open call"

This reverts commit 43074e6.

* scroll combobox into view

* scroll before filter

* Move CSP config default values to csp module (elastic#41676)

This gives us a little more control over the default values of these
configurations to help ensure (though not guarantee) that any changes
here can be audited by the security team.

* Remove notifications plugin (elastic#41674)

The notifications functionality has been replaced by the features of the
actions plugin. This notifications plugin was never actually used by
end-user facing features of Kibana.

* [Logs UI] Make column configurations reorderable (elastic#41035)

* [Logs UI] Make column configurations reorderable

* Improve typing aand memoize callback

* Guard against index bounds and rename reorderLogColumns

* Fix useCallback memoization

* Add functional test for reordering log columns

* Use browser.keys instead of Key in functional test

* [Maps] populate _id in tooltip (elastic#41684)

* [ML] Data Frames - search bar on list page (elastic#41415)

* add search and filter to df list table

* add mode filter to list table

* adds id + description search

* type fix

* ensure search syntax is valid

* ensure types are correct

* retain filter on refresh

* fix progress bar jump

* [DOCS] Changed Visual Builder to TSVB (elastic#39539)

* [DOCS] Changed Visual Builder to TSVB

* Reorg of interface changes

* Content reorg

* Updated image

* Added task content

* Content conslidation

* Final clean up

* Comments from Gail

* [DOCS] Adds missing Timelion link (elastic#41709)

* [Infra UI] Fix section mapping bug in node detail page (elastic#41641)

* [Infra UI] Fix section mapping bug in node detail page

* Fixing filter to match TSVB

* Adding an enum for the InfraMetricsQueryType

* removing unnecessary change

* Change id to InfraMetric to make less error prone

* Fixing type in Metrics Explorer

* [Infra UI] Add UI to customize Metrics Explorer chart style (elastic#41022)

* Add UI to customize Metrics Explorer chart style

* Re-order chart options form

* Adding chart options to TSVB link

* Rename line series to series chart

* Fixing chart context menu tests

* Adding test for calculate domain

* Ensure caclulateDomain returns numbers

* fixing typo

* Bump backport to 4.6.1 (elastic#41720)

* hide top nav menu in full screen mode of maps and dashboard (elastic#41672)

* hide top nav menu in full screen mode of maps and dashboard

* Fixed dashboard full screen mode and added full screen mode test to maps

* improve typing (elastic#41563)

* [Code] test colorize before load a file (elastic#41645)

* [Code] handle status when repo is not exists (elastic#41643)

fix can't switch to head on root path

* [Code] implement filtree api by using isogit (elastic#41558)

* remove obsolete http secutiry settings (elastic#41569)

* core doesn't use Record<string, unknown> for public API (elastic#41448)

* core contracts don't use unknown to support type assignment

limitations of https://github.com/Microsoft/TypeScript/wiki/Breaking-Changes#-k-string-unknown--is-no-longer-a-wildcard-assignment-target

* regenereate docs

* remove type over-write

* Narrow type of PluginDeps to an object (elastic#40846)

* Narrow type of PluginDeps to an object

* re-generate docs

* [APM] Fix "Show trace logs" link (elastic#41570)

* [APM] Fix "Show trace logs" link

* Add type for infra link items; escape url.domain param for uptime link

* Comment out flakey test steps (elastic#41743)

This comments out the problematic portions of the functional test, which caused elastic#41717 to occur.

* [ML] Fixes model memory limit for metricbeat system module jobs (elastic#41747)

* [i18n] fix i18nrc parsing (elastic#41741)

* check for translations in file

* update template

* Add ownership of new platform security plugin to security team. (elastic#41748)

* Clean up top nav \ search bar \ query bar directives (elastic#41636)

* Move timepicker (to be deprecated) into old kbn_top_nav

* Deleted search-bar and query-bar directives!

* moved search bar to kibana_react (it's a generic react component, not a service)

* translations

* Moved superDatePicker directive to kbn_top_nav (to be deprecated)
Deleted unused react_component directives call-out and tool-bar-search-box

* TS test fix

* Delete relative options

* [ML] Use NavigationMenu without angularjs wrapper. (elastic#41343)

Follow up to elastic#40830 and elastic#41054 in preparation for single metric viewer migration.
The previous PR introduced the navigation menu as a React component. This PR moves dependencies down from the angularjs wrapper directive directly to the React component so the component can also be used stand-alone without the angularjs wrapper.
For simple angularjs based HTML templates this stand-alone usage is also part of this PR. Unfortunately the PR turned out to be quite big due to: Most page react components had to be wrapped in another <Fragment> to allow the addition of <NavigationMenu> thus leading to large diffs for the components. All component code inside the <Fragment> was not touched though.

* [telemetry] Analytics Package (elastic#41113)

* kbn-analytics

* kbn-analytics

* expose provider

* add logger

* performance and stats reporters

* finalize ui stats metric

* functional tests

* remove readme file for now

* update readme

* add types file into to tsconfigs

* Update packages/kbn-analytics/src/report.ts

Co-Authored-By: Josh Dover <me@joshdover.com>

* fix typechecks

* use enum instead of strings for metric types

* getUiStatsReporter -> createUiStatsReporter

* fix special typo in README

* remove unused stop method

* fix tests

* default debug to false

* use chrome.getInjected

* add METRIC_TYPE to jest module mocks

* mock create fn

* handle enabled:false

* init ui_metric in test setup env

* regenerator runtime

* transform-regenerator

* update lock file

* update babel configs

* runtime dep

* add regenerator

* babel configs

* use env-preset

* merge conflicts

* fix workpad telemetry tests

* regeneratorRuntime attempt to fix number 30000

* env targets

* remove module config

* try again

* try without regenerator

* use kbn/babel-preset/webpack_preset only

* runtime

* just use typescript

* update tsconfig

* Caches trackers by app value for infra useTrackMetric hook

* replace all occurences of placeholder in drilldown url template (elastic#41673)

* cleanup names and files/paths to confirm to a standard format

* tack down WIP code

* remove things not or not yet needed

* Added Flexmonster Pivot Table to known plugins list (elastic#41655)

* Add Flexmonster Pivot Table to known plugins list

* Update docs/plugins/known-plugins.asciidoc

Co-Authored-By: Larry Gregory <lgregorydev@gmail.com>

* Fix typo (elastic#41705)

* turn on filtering tests (elastic#41202)

* turn on filtering tests

* run x-pack-firefoxSmoke ciGroup 40 times, run dashboard tests 20 times per job

* Revert "run x-pack-firefoxSmoke ciGroup 40 times, run dashboard tests 20 times per job"

This reverts commit 5ef02cc53ba5085c3f74431cb6ef20be2d876cde.

* GoodBye Notifier (elastic#41663)

* Begin notifier removal

* Remove remaining notifier traces

* Remove dead translations

* Remove Angular from config listener

* Import angular-sanitize explicitly in map

* Revert "lock es snapshot to avoid failing CI" (elastic#41539)

This reverts commit 4eca0f3.

* fix more types, define SO

* [SIEM] - Fix Jest test errors and warnings (elastic#41712)

* Fixes elastic#41787 (elastic#41791)

* [DOCS] Puts Spaces content on single page (elastic#41536)

* [DOCS] Puts Spaces content on single page

* [DOCS] Incorporates review comments

* [DOCS] Incorporated review comments

* [SIEM] - Hosts and Network Tables from LoadMore to Paginated (elastic#41532)

* [DOCS] Updates Console doc (elastic#41371)

* [DOCS] Updates Console doc

* [DOCS] Incorporates comments on Console docs

* [DOCS] Updated Console images

* Fixed unused variables. Added a few methods to the SO adapter

Co-authored-by: Nicolas Chaulet <n.chaulet@gmail.com>

* fix type

* Revert "Merge branch 'master' of github.com:elastic/kibana into feature-fleet"

This reverts commit 997490f, reversing
changes made to db5fc8f.

* removed blank test file

* Fix file path

* add i18n

x-pack/.i18nrc.json

@@ -12,11 +12,13 @@
     "xpack.dashboardMode": "legacy/plugins/dashboard_mode",
     "xpack.features": "plugins/features",
     "xpack.fileUpload": "legacy/plugins/file_upload",
+    "xpack.fleet": "legacy/plugins/fleet",
     "xpack.graph": "legacy/plugins/graph",
     "xpack.grokDebugger": "legacy/plugins/grokdebugger",
     "xpack.idxMgmt": "legacy/plugins/index_management",
     "xpack.indexLifecycleMgmt": "legacy/plugins/index_lifecycle_management",
     "xpack.infra": "legacy/plugins/infra",
+    "xpack.ingest": "legacy/plugins/ingest",
     "xpack.kueryAutocomplete": "legacy/plugins/kuery_autocomplete",
     "xpack.lens": "legacy/plugins/lens",
     "xpack.licensing": "plugins/licensing",

x-pack/legacy/plugins/fleet/common/types/helpers.ts

@@ -5,3 +5,5 @@
  */
 
 export type FlatObject<T> = { [Key in keyof T]: string };
+export type RendererResult = React.ReactElement<any> | null;
+export type RendererFunction<RenderArgs, Result = RendererResult> = (args: RenderArgs) => Result;

x-pack/legacy/plugins/fleet/index.ts

@@ -5,7 +5,7 @@
  */
 import * as Joi from 'joi';
 import { resolve } from 'path';
-import { i18n } from '@kbn/i18n/src';
+import { i18n } from '@kbn/i18n';
 import { PLUGIN } from './common/constants';
 import { CONFIG_PREFIX } from './common/constants/plugin';
 import { initServerWithKibana } from './server/kibana.index';

x-pack/legacy/plugins/fleet/public/hooks/with_kuery_autocompletion.tsx

@@ -9,7 +9,7 @@ import React from 'react';
 import { AutocompleteSuggestion } from 'ui/autocomplete_providers';
 
 import { FrontendLibs } from '../lib/types';
-import { RendererFunction } from '../utils/typed_react';
+import { RendererFunction } from '../../common/types/helpers';
 
 interface WithKueryAutocompletionLifecycleProps {
   libs: FrontendLibs;

x-pack/legacy/plugins/fleet/public/hooks/with_url_state.tsx

@@ -7,8 +7,7 @@
 import { parse, stringify } from 'querystring';
 import React from 'react';
 import { withRouter } from 'react-router-dom';
-import { FlatObject } from '../frontend_types';
-import { RendererFunction } from '../utils/typed_react';
+import { FlatObject, RendererFunction } from '../../common/types/helpers';
 
 type StateCallback<T> = (previousState: T) => T;
 

x-pack/legacy/plugins/fleet/public/lib/elasticsearch.ts

@@ -48,7 +48,7 @@ export class ElasticsearchLib {
         });
       }
 
-      return hiddenFieldsCheck.reduce((isvalid, field) => {
+      return hiddenFieldsCheck.reduce((isvalid: boolean, field) => {
         if (!isvalid) {
           return false;
         }

x-pack/legacy/plugins/fleet/public/routes.tsx

@@ -7,7 +7,6 @@
 import { get } from 'lodash';
 import React, { Component } from 'react';
 import { Redirect, Route, Switch } from 'react-router-dom';
-import { REQUIRED_ROLES } from '../common/constants/security';
 import { Loading } from './components/loading';
 import { ChildRoutes } from './components/navigation/child_routes';
 import { URLStateProps, WithURLState } from './hooks/with_url_state';
@@ -59,19 +58,6 @@ export class AppRoutes extends Component<RouterProps, RouterState> {
             />
           )}
 
-          {/* Make sure the user has correct permissions */}
-          {!this.props.libs.framework.currentUserHasOneOfRoles(
-            REQUIRED_ROLES.concat(this.props.libs.framework.info.settings.defaultUserRoles)
-          ) && (
-            <Route
-              render={props =>
-                !props.location.pathname.includes('/error') ? (
-                  <Redirect to="/error/no_access" />
-                ) : null
-              }
-            />
-          )}
-
           {/* This app does not make use of a homepage. The mainpage is overview/enrolled_agents */}
           <Route path="/" exact={true} render={() => <Redirect to="/overview/enrolled_agents" />} />
         </Switch>
@@ -80,7 +66,7 @@ export class AppRoutes extends Component<RouterProps, RouterState> {
         <WithURLState>
           {(URLProps: URLStateProps) => (
             <ChildRoutes
-              routes={routeMap}
+              routes={[]}
               {...URLProps}
               {...{
                 libs: this.props.libs,

x-pack/legacy/plugins/ingest/server/kibana.index.ts

@@ -4,12 +4,9 @@
  * you may not use this file except in compliance with the Elastic License.
  */
 
+import { compose } from './libs/compose/kibana';
+
 export const initServerWithKibana = (hapiServer: any) => {
-  hapiServer.route({
-    method: 'GET',
-    path: '/api/fleet/load',
-    handler: (request: any, h: any) => {
-      return 'Hello World!';
-    },
-  });
+  const libs = compose(hapiServer);
+  libs.framework.log('Ingest is composed -- debug message');
 };

x-pack/legacy/plugins/ingest/server/libs/adapters/configurations/adapter_types.ts

@@ -5,7 +5,6 @@
  */
 
 import * as t from 'io-ts';
-import { DateFromString } from '../../../../common/types/io_ts';
 
 export const RuntimeDatasourceInput = t.interface(
   {
@@ -48,13 +47,14 @@ export const NewRuntimeBackupConfigurationFile = t.interface(
   'BackupConfigurationFile'
 );
 
-const ExistingDocument = t.partial({
+const ExistingDocument = t.interface({
   id: t.string,
   shared_id: t.string,
   version: t.number,
-  updated_at: DateFromString,
+  active: t.boolean,
+  updated_at: t.string,
   created_by: t.union([t.undefined, t.string]),
-  updated_on: DateFromString,
+  updated_on: t.string,
   updated_by: t.union([t.undefined, t.string]),
 });
 

x-pack/legacy/plugins/ingest/server/libs/adapters/configurations/default.ts

@@ -4,30 +4,98 @@
  * you may not use this file except in compliance with the Elastic License.
  */
 
-import {
-  ConfigurationFile,
-  NewConfigurationFile,
-  DatasourceInput,
-  BackupConfigurationFile,
-} from './adapter_types';
-
-export class DefaultConfigAdapter {
+import { SODatabaseAdapter } from '../so_database/default';
+import { RuntimeConfigurationFile, NewConfigurationFile } from './adapter_types';
+
+import { ConfigurationFile, DatasourceInput, BackupConfigurationFile } from './adapter_types';
+
+export class ConfigAdapter {
+  constructor(private readonly so: SODatabaseAdapter) {}
+
   public async create(
     configuration: NewConfigurationFile
   ): Promise<{ id: string; shared_id: string; version: number }> {
+    const newSo = await this.so.create<ConfigurationFile>(
+      'configurations',
+      (configuration as any) as ConfigurationFile
+    );
+
     return {
-      id: 'fsdfsdf',
-      shared_id: 'wjkhefkjhfkjs',
-      version: 0,
+      id: newSo.id,
+      shared_id: newSo.attributes.shared_id,
+      version: newSo.attributes.version,
     };
   }
 
-  public async get(sharedID: string, version?: number): Promise<ConfigurationFile> {
-    return {} as ConfigurationFile;
+  public async get(id: string): Promise<ConfigurationFile> {
+    const config = await this.so.get<any>('configurations', id);
+
+    if (config.error) {
+      throw new Error(config.error.message);
+    }
+
+    if (!config.attributes) {
+      throw new Error(`No configuration found with ID of ${id}`);
+    }
+    if (RuntimeConfigurationFile.decode(config.attributes).isRight()) {
+      return config.attributes as ConfigurationFile;
+    } else {
+      throw new Error(`Invalid ConfigurationFile data. == ${config.attributes}`);
+    }
+  }
+
+  public async list(): Promise<ConfigurationFile[]> {
+    const configs = await this.so.find<any>({
+      type: 'configurations',
+      search: '*',
+      searchFields: ['shared_id'],
+    });
+    const uniqConfigurationFile = configs.saved_objects
+      .map<ConfigurationFile>(config => {
+        if (RuntimeConfigurationFile.decode(config.attributes).isRight()) {
+          return config.attributes;
+        } else {
+          throw new Error(`Invalid ConfigurationFile data. == ${config.attributes}`);
+        }
+      })
+      .reduce((acc, config: ConfigurationFile) => {
+        if (!acc.has(config.shared_id)) {
+          acc.set(config.shared_id, config);
+        }
+        const prevConfig = acc.get(config.shared_id);
+        if (prevConfig && prevConfig.version < config.version) {
+          acc.set(config.shared_id, config);
+        }
+
+        return acc;
+      }, new Map<string, ConfigurationFile>());
+
+    return [...uniqConfigurationFile.values()];
   }
 
-  public async list(sharedID: string, version?: number): Promise<ConfigurationFile[]> {
-    return [{} as ConfigurationFile];
+  public async listVersions(sharedID: string, activeOnly = true): Promise<ConfigurationFile[]> {
+    const configs = (await this.so.find<any>({
+      type: 'configurations',
+      search: sharedID,
+      searchFields: ['shared_id'],
+    })).saved_objects;
+
+    if (!activeOnly) {
+      const backupConfigs = await this.so.find<any>({
+        type: 'backup_configurations',
+        search: sharedID,
+        searchFields: ['shared_id'],
+      });
+      configs.concat(backupConfigs.saved_objects);
+    }
+
+    return configs.map<ConfigurationFile>(config => {
+      if (RuntimeConfigurationFile.decode(config.attributes).isRight()) {
+        return config.attributes;
+      } else {
+        throw new Error(`Invalid ConfigurationFile data. == ${config.attributes}`);
+      }
+    });
   }
 
   public async update(

x-pack/legacy/plugins/ingest/server/libs/adapters/database/adapter_types.ts → x-pack/legacy/plugins/ingest/server/libs/adapters/es_database/adapter_types.ts

@@ -3,41 +3,7 @@
  * or more contributor license agreements. Licensed under the Elastic License;
  * you may not use this file except in compliance with the Elastic License.
  */
-import { FrameworkRequest, FrameworkUser } from '../framework/adapter_types';
-
-export interface DatabaseAdapter {
-  get<Source>(
-    user: FrameworkUser,
-    params: DatabaseGetParams
-  ): Promise<DatabaseGetDocumentResponse<Source>>;
-  create(
-    user: FrameworkUser,
-    params: DatabaseCreateDocumentParams
-  ): Promise<DatabaseCreateDocumentResponse>;
-  index<T>(
-    user: FrameworkUser,
-    params: DatabaseIndexDocumentParams<T>
-  ): Promise<DatabaseIndexDocumentResponse>;
-  delete(
-    user: FrameworkUser,
-    params: DatabaseDeleteDocumentParams
-  ): Promise<DatabaseDeleteDocumentResponse>;
-  deleteByQuery(
-    user: FrameworkUser,
-    params: DatabaseSearchParams
-  ): Promise<DatabaseDeleteDocumentResponse>;
-  mget<T>(user: FrameworkUser, params: DatabaseMGetParams): Promise<DatabaseMGetResponse<T>>;
-  bulk(
-    user: FrameworkUser,
-    params: DatabaseBulkIndexDocumentsParams
-  ): Promise<DatabaseBulkResponse>;
-  search<T>(user: FrameworkUser, params: DatabaseSearchParams): Promise<DatabaseSearchResponse<T>>;
-  searchAll<T>(
-    user: FrameworkUser,
-    params: DatabaseSearchParams
-  ): Promise<DatabaseSearchResponse<T>>;
-  putTemplate(name: string, template: any): Promise<any>;
-}
+import { FrameworkRequest } from '../framework/adapter_types';
 
 export interface DatabaseKbnESCluster {
   callWithInternalUser(esMethod: string, options: {}): Promise<any>;

x-pack/legacy/plugins/ingest/server/libs/adapters/database/kibana_database_adapter.ts → x-pack/legacy/plugins/ingest/server/libs/adapters/es_database/default.ts

@@ -6,7 +6,6 @@
 import { FrameworkUser } from '../framework/adapter_types';
 import { internalAuthData } from '../framework/adapter_types';
 import {
-  DatabaseAdapter,
   DatabaseBulkIndexDocumentsParams,
   DatabaseCreateDocumentParams,
   DatabaseCreateDocumentResponse,
@@ -23,7 +22,7 @@ import {
   DatabaseSearchResponse,
 } from './adapter_types';
 
-export class KibanaDatabaseAdapter implements DatabaseAdapter {
+export class ESDatabaseAdapter {
   private es: DatabaseKbnESCluster;
 
   constructor(kbnElasticSearch: DatabaseKbnESPlugin) {