Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Data table] Reactify & EUIficate the visualization #70801

Merged
merged 103 commits into from
Dec 15, 2020
Merged
Show file tree
Hide file tree
Changes from 74 commits
Commits
Show all changes
103 commits
Select commit Hold shift + click to select a range
ed29fb2
Use data grid for table vis
sulemanof Jul 1, 2020
82712f0
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
sulemanof Jul 1, 2020
9e17f4d
Create basic table template
sulemanof Jul 2, 2020
3a693ef
Add table_vis_split component
sulemanof Jul 2, 2020
34a1375
Apply cell filtering
sulemanof Jul 2, 2020
de6dd8f
Add aria-label attributes
sulemanof Jul 2, 2020
3a6e113
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
sulemanof Jul 6, 2020
d2e8436
Use field formatters for values
sulemanof Jul 6, 2020
66654f8
Add no results component
sulemanof Jul 6, 2020
743fded
Remove legacy dependencies
sulemanof Jul 6, 2020
ca802cd
Add usePagination
sulemanof Jul 7, 2020
02013f1
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
sulemanof Jul 8, 2020
af74780
Create usePagination util
sulemanof Jul 8, 2020
f6cdb86
Use percentage column and total row
sulemanof Jul 8, 2020
c68b60e
Use csv export button
sulemanof Jul 9, 2020
dea9e48
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
sulemanof Jul 13, 2020
d485086
Update labels
sulemanof Jul 20, 2020
6c489dc
Merge branch 'master' into EUIfication/data_table
sulemanof Jul 24, 2020
409e34b
Fix merge conflicts
sulemanof Jul 24, 2020
14c27a4
Use split table
sulemanof Jul 24, 2020
5853a10
Fix functional tests
sulemanof Jul 27, 2020
7a561da
Fix dashboard tests
sulemanof Jul 28, 2020
f8accdf
Update data table functional tests
sulemanof Jul 28, 2020
9902674
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
sulemanof Jul 28, 2020
62ab3c4
Fix missed test
sulemanof Jul 29, 2020
0a90deb
Introduce showToolbar option
sulemanof Jul 29, 2020
a057da9
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
sulemanof Jul 29, 2020
59987e5
Merge branch 'master' into EUIfication/data_table
Sep 30, 2020
144a7cd
Remove useless package
Sep 30, 2020
b90d442
Fix merge conflicts
Sep 30, 2020
d4162fc
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
Oct 2, 2020
04eec45
Return back kibanaUtils required bundle
Oct 2, 2020
0bde52a
Merge branch 'master' into EUIfication/data_table
Oct 12, 2020
fe2501f
Revert "Remove useless package"
Oct 12, 2020
08944ca
Use feature flag for legacy vis
Oct 12, 2020
f4790c1
Add footer row
Oct 13, 2020
5c5acd7
Remove lock files
Oct 14, 2020
bc75fef
Revert "Remove lock files"
Oct 14, 2020
cd8d67f
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
Oct 14, 2020
9a54372
Minor fixes
Oct 14, 2020
7816196
Use common no result message
Oct 14, 2020
73fd7a9
Fix broken tests
Oct 14, 2020
2598656
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
Oct 14, 2020
ea0c16c
Use ui state sorting
Oct 16, 2020
395144f
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
Oct 16, 2020
448e23f
Fix error
Oct 16, 2020
89f2e0d
Merge branch 'master' into EUIfication/data_table
Oct 19, 2020
8ae598f
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
Oct 21, 2020
0bc512f
Fix merge conflicts
Oct 21, 2020
6f0aa33
Add legacy functional tests
Oct 21, 2020
7836ab1
Pull pagination footer up to keep with table
Oct 21, 2020
8d548da
Merge branch 'master' into EUIfication/data_table
Oct 28, 2020
4545764
Comments fixes
Oct 29, 2020
6372a59
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
Oct 29, 2020
e96b097
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
Nov 2, 2020
445b9ca
Use cell actions for filtering
Nov 2, 2020
1083217
Fix translations
Nov 2, 2020
c846aff
Merge branch 'master' into EUIfication/data_table
Nov 9, 2020
61730b3
Fix comments
Nov 9, 2020
47b0b0c
Reduce legacy tests amount
Nov 9, 2020
1029099
Update sorting
Nov 9, 2020
953fdaa
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
Nov 10, 2020
d8a50c9
Update split column layout
Nov 11, 2020
b51ea9c
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
Nov 11, 2020
df24a7b
Add telemetry for legacy vis
Nov 11, 2020
27c300f
Merge branch 'master' into EUIfication/data_table
Nov 24, 2020
d21236f
Apply latest changes for split table
Nov 24, 2020
74a71ea
Fix eslint errors
Nov 24, 2020
344fdea
Use aria labels with values
Nov 24, 2020
03888a0
Use aria label for export btn
Nov 24, 2020
8c95a55
Fix functional test
Nov 25, 2020
ae02c6d
Update translations
Nov 25, 2020
7ad54f1
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
Nov 25, 2020
1ace317
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
Nov 30, 2020
7ea584f
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
Dec 1, 2020
8bb70b5
Cleanup
Dec 1, 2020
e661af7
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
Dec 3, 2020
a13d027
Merge branch 'master' into EUIfication/data_table
kibanamachine Dec 3, 2020
528865f
Truncate cells content
Dec 4, 2020
ada2cae
Enhance types in table_vis_response_handler.ts
Dec 4, 2020
3c7950e
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
Dec 4, 2020
ad09412
Merge branch 'EUIfication/data_table' of github.com:sulemanof/kibana …
Dec 4, 2020
bb2a885
Persist columns width on change
Dec 7, 2020
94706bc
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
Dec 7, 2020
31ee02d
Fix sorting history
Dec 7, 2020
aac4c2d
Add a migration script for toolbar
Dec 7, 2020
ef00cc1
Export sorted table
Dec 7, 2020
17f381d
Use reportUiCounter instead of reportUiStats
Dec 7, 2020
b4effc4
Fix integration tests
Dec 7, 2020
e502e9d
Fix typos
Dec 8, 2020
558d2c7
Adjust FieldFormat type
Dec 8, 2020
e993a5e
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
Dec 8, 2020
6ace73a
Hide the density selector
Dec 8, 2020
0320d28
Update docs
Dec 8, 2020
f1cd098
Fix pagination
Dec 9, 2020
6d620d8
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
Dec 9, 2020
9048a45
Restrict hiding the toolbar
Dec 9, 2020
5285111
Fix column index on filter
Dec 10, 2020
edd4809
Merge remote-tracking branch 'upstream/master' into EUIfication/data_…
Dec 10, 2020
6cd3d3d
Merge branch 'master' into EUIfication/data_table
kibanamachine Dec 14, 2020
d9e7c96
Merge branch 'master' into EUIfication/data_table
kibanamachine Dec 15, 2020
d16b81a
Add closePopover action
Dec 15, 2020
26b9cae
Merge branch 'master' into EUIfication/data_table
kibanamachine Dec 15, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions scripts/functional_tests.js
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ const alwaysImportedTests = [
require.resolve('../test/ui_capabilities/newsfeed_err/config.ts'),
require.resolve('../test/new_visualize_flow/config.ts'),
require.resolve('../test/security_functional/config.ts'),
require.resolve('../test/functional/config.legacy.ts'),
];
// eslint-disable-next-line no-restricted-syntax
const onlyNotInCoverageTests = [
Expand Down
9 changes: 8 additions & 1 deletion src/plugins/vis_type_table/README.md
Original file line number Diff line number Diff line change
@@ -1 +1,8 @@
Contains the data table visualization, that allows presenting data in a simple table format.
Contains the data table visualization, that allows presenting data in a simple table format.

By default a new version of visualization will be used. To use the previous version of visualization the config must have the `vis_type_table.legacyVisEnabled: true` setting
configured in `kibana.dev.yml` or `kibana.yml`, as shown in the example below:

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not in favor of keeping the legacy data table available behind a flag. I would expect when we reactify and EUIify the data table it simply replaces the old angular one. If you make the old one available behind a flag, then we also have to have telemetry on it to know if users are still using it. If they're still using it, we'd need to know why.

Flags like this which require restarting Kibana also are painful for CI jobs. If we did keep the legacy table behind this flag, I would suggest we stop testing it.

If we're not confident to replace the old data table completely we should wait to merge it until we are confident.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should def add telemetry to this flag to make sure we understand if people disable this. The reasoning behind the flag was, that data table is the 2nd most used visualization across Kibana, and we're fundamentally exchanging the implementation here. I would like to merge it only once we're certain it's 100% working, but in the past we've seen with a lot of those fundamental refactorings, things still broke and even if it's just some edge-cases only "a handful" of users are having. By that measurement I am not sure I would ever feel "confident" enough to replace it without any fallback flag, or rather feel like we're again using a minor version and our user-case to "test" an implementation for us in all cases.

We've also seen that significant refactorings ever so often required some users to switch back to the old implementation (most recently the conversion from _msearch -> _search might have been a good example were we saw quit some users requiring this flag to fall back).

Ideally this flag only lives for a really short time - I would "timebox" it to max. 2 minor versions (ideally only 1). If you're fine with us not having functional tests for the old implementation in place in favor of not requiring another job runner, we could do that. It just felt safer from my perspective to have at least some very basic tests running.

Copy link
Member

@dmlemeshko dmlemeshko Oct 27, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Running all listed legacy tests will add extra ~20-25 min, that sounds too much for legacy functionality, we will have to re-group several CI groups to optimize overall execution time.
But I think having a few tests with basic checks (rendering data) is a good compromise (5-7 extra minutes to the same CI group 9), we don't want to worry if backward solution works. @sulemanof How about cut some tests from legacy/data_table/_data_table.ts ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dmlemeshko @LeeDr
I left only few legacy tests to check basic functionality (basic table, few aggs, filtering, table splitting)
CI time is increased with around 5 extra mins -> https://kibana-ci.elastic.co/job/elastic+kibana+pipeline-pull-request/86727/execution/node/531/log/?consoleFull
I believe this is quite enough to check the legacy functionality and is not so painful for CI. What do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I think it is a good way to go.

```yaml
vis_type_table.legacyVisEnabled: true
```
1 change: 1 addition & 0 deletions src/plugins/vis_type_table/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import { schema, TypeOf } from '@kbn/config-schema';

export const configSchema = schema.object({
enabled: schema.boolean({ defaultValue: true }),
legacyVisEnabled: schema.boolean({ defaultValue: false }),
});

export type ConfigSchema = TypeOf<typeof configSchema>;
4 changes: 3 additions & 1 deletion src/plugins/vis_type_table/kibana.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,10 @@
],
"requiredBundles": [
"kibanaUtils",
"kibanaReact",
"share",
"charts",
"visDefaultEditor"
]
],
"optionalPlugins": ["usageCollection"]
}

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 20 additions & 0 deletions src/plugins/vis_type_table/public/components/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
/*
* Licensed to Elasticsearch B.V. under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch B.V. licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

export { TableOptions } from './table_vis_options_lazy';
149 changes: 149 additions & 0 deletions src/plugins/vis_type_table/public/components/table_vis_basic.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
/*
* Licensed to Elasticsearch B.V. under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch B.V. licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

import React, { memo, useCallback, useMemo } from 'react';
import { EuiDataGrid, EuiDataGridSorting, EuiTitle } from '@elastic/eui';
import { i18n } from '@kbn/i18n';
import { orderBy } from 'lodash';

import { IInterpreterRenderHandlers } from 'src/plugins/expressions';
import { createTableVisCell } from './table_vis_cell';
import { Table } from '../table_vis_response_handler';
import { TableVisConfig, TableVisUiState } from '../types';
import { useFormattedColumnsAndRows, usePagination } from '../utils';
import { TableVisControls } from './table_vis_controls';
import { createGridColumns } from './table_vis_columns';

interface TableVisBasicProps {
fireEvent: IInterpreterRenderHandlers['event'];
setSort: (s?: TableVisUiState['sort']) => void;
sort: TableVisUiState['sort'];
table: Table;
visConfig: TableVisConfig;
title?: string;
}

export const TableVisBasic = memo(
({ fireEvent, setSort, sort, table, visConfig, title }: TableVisBasicProps) => {
const { columns, rows } = useFormattedColumnsAndRows(table, visConfig);

// custom sorting is in place until the EuiDataGrid sorting gets rid of flaws -> https://github.com/elastic/eui/issues/4108
const sortedRows = useMemo(
() =>
sort.columnIndex !== null && sort.direction
? orderBy(rows, columns[sort.columnIndex]?.id, sort.direction)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this working the same as the old implementation? I checked and it's using Array.sort under the hood, not sure whether there are differences to lodash orderBy - one difference that stood out to me is that boolean values are replaced by 0 and 1 in the legacy sorting, is this required here as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Firstly, I don't think there could be boolean values at all, since they come from es response already as numbers (0 and 1).
Secondly, lodash works fine with boolean values if they are.
Don't think we'll see any difference in orderBy from lodash and orderBy filter from angular

: rows,
[columns, rows, sort]
);

// renderCellValue is a component which renders a cell based on column and row indexes
const renderCellValue = useMemo(() => createTableVisCell(columns, sortedRows), [
columns,
sortedRows,
]);

// Columns config
const gridColumns = useMemo(() => createGridColumns(table, columns, sortedRows, fireEvent), [
table,
columns,
sortedRows,
fireEvent,
]);

// Pagination config
const pagination = usePagination(visConfig);
// Sorting config
const sortingColumns = useMemo(
() =>
sort.columnIndex !== null && sort.direction
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both sorting and column width is saved by index - this is confusing if the table is extended after configuring this:

  • Add column 1 to table
  • Change width of column 1
  • Add a new column 2 before the column 1
  • Now the new column 2 gets the width of column 1 and column 1 gets auto-sized

The same happens with sorting.

The sorting problem exists as well on the legacy table, but I wonder whether we can do better and use some unique id instead.

I wouldn't necessarily consider this a blocker, but it's really noticeable for the resizing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, we can't rely on a column's id, since it is dynamically created outside of visualization (from the expression input).

? [{ id: columns[sort.columnIndex]?.id, direction: sort.direction }]
: [],
[columns, sort]
);
const onSort = useCallback(
(sortingCols: EuiDataGridSorting['columns'] | []) => {
// data table vis sorting now only handles one column sorting
// if data grid provides more columns to sort, pick only the next column to sort
const newSortValue = sortingCols.length <= 1 ? sortingCols[0] : sortingCols[1];
setSort(
newSortValue && {
columnIndex: columns.findIndex((c) => c.id === newSortValue.id),
direction: newSortValue.direction,
}
);
},
[columns, setSort]
);

const dataGridAriaLabel =
title ||
visConfig.title ||
i18n.translate('visTypeTable.defaultAriaLabel', {
defaultMessage: 'Data table visualization',
});

return (
<>
{title && (
<EuiTitle size="xs">
<h3>{title}</h3>
</EuiTitle>
)}
<EuiDataGrid
aria-label={dataGridAriaLabel}
columns={gridColumns}
gridStyle={{
border: 'horizontal',
header: 'underline',
}}
rowCount={rows.length}
columnVisibility={{
visibleColumns: columns.map(({ id }) => id),
setVisibleColumns: () => {},
}}
toolbarVisibility={
visConfig.showToolbar && {
showColumnSelector: false,
showFullScreenSelector: false,
showSortSelector: false,
additionalControls: (
<TableVisControls
dataGridAriaLabel={dataGridAriaLabel}
cols={columns}
rows={rows}
table={table}
filename={visConfig.title}
/>
),
}
}
renderCellValue={renderCellValue}
renderFooterCellValue={
visConfig.showTotal
? // @ts-expect-error
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is concerning - why is this missing in the types? Is this is an oversight or is it intentional and shouldn't be used?

In the first case, please open an issue for EUI, otherwise we should work around this and look up the index by the columnId which seems to get passed in.

cc @chandlerprall

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll create a pr with fixing this on the eui side.

({ colIndex }) => columns[colIndex].formattedTotal || null
: undefined
}
pagination={pagination}
sorting={{ columns: sortingColumns, onSort }}
/>
</>
);
}
);
49 changes: 49 additions & 0 deletions src/plugins/vis_type_table/public/components/table_vis_cell.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
/*
* Licensed to Elasticsearch B.V. under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch B.V. licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/

import React from 'react';
import { EuiDataGridCellValueElementProps } from '@elastic/eui';

import { Table } from '../table_vis_response_handler';
import { FormattedColumn } from '../types';

export const createTableVisCell = (formattedColumns: FormattedColumn[], rows: Table['rows']) => ({
// @ts-expect-error
colIndex,
sulemanof marked this conversation as resolved.
Show resolved Hide resolved
rowIndex,
columnId,
}: EuiDataGridCellValueElementProps) => {
const rowValue = rows[rowIndex][columnId];
const column = formattedColumns[colIndex];
const content = column?.formatter?.convert(rowValue, 'html') || (rowValue as string) || '';

const cellContent = (
<div
/*
* Justification for dangerouslySetInnerHTML:
* The Data table visualization can "enrich" cell contents by applying a field formatter,
* which we want to do if possible.
*/
dangerouslySetInnerHTML={{ __html: content }} // eslint-disable-line react/no-danger
data-test-subj="tbvChartCellContent"
/>
);

return cellContent;
};
Loading