[DataGrid] Breaking change with the introduction of the inMemory prop not documented

[dej611]

The sorting behaviour in EUI Datatable changed and broke since the introduction of the inMemory prop to the component here.
The change wasn't easy to find as it kept on sorting correctly for numeric values, but for text it completely stopped unless the inMemory prop was configured as {level: "sorting"}.

It feels like a bug from the consumer prospective, as partial sorting was working without inMemory.
I went thru the changelog page but could not find any mention of the introduction of the inMemory prop with the associated behaviour change.

The problem arise when the sorting logic is controlled externally from the datagrid and this is informed of the new sorting direction via the sorting={ [{id: columnId, direction: "asc" | "desc" }] prop.
We use this logic in Kibana Lens in order to save the current sorting direction in the visualization state and re-render the table.

Related Kibana issue: elastic/kibana#160139

[cee-chen]

@dej611 do you have the link to the right EuiDataGrid PR in your description? The one you linked to has nothing to do with inMemory (affects the ref prop, and any inMemory usage is already extant and only applies to either tests or documentation).

Digging back through git blame, inMemory appears to have been implemented in EuiDataGrid since the beginning:

• https://github.com/elastic/eui/blame/523a1178f24dcd06318b1a24791d25c15f695105/src/components/datagrid/data_grid.tsx#L141-L142C10
• [EuiDataGrid] InMemory Performance #2374

It's not totally clear to me what the bug is or the behavior you're describing from your PR description. Are you saying that this combination of props works:

inMemory={{ level: 'sorting' }}
sorting={ [{id: columnId, direction: "asc" | "desc" }]

but this does not:

// no inMemory prop
sorting={ [{id: columnId, direction: "asc" | "desc" }]

If so, what exactly doesn't work regarding the sorting? If you're not providing the inMemory prop, then yes, you'll have to sort your data in-place, and the sorting prop primarily only informs the data grid's toolbar UI.

[dej611]

I'm trying to identify where changes happened but couldn't find the specific commit who broke it.
So I went investigating on the Kibana side, and this commit was the one who migrated from EuiBasicTable to EuiDataGrid.
The sorting logic was tested and it was working at the time, no change happened on the Kibana side since (we only moved out the code in separate module, it was a 1:1 exact copy), but apparently the sorting for text values broke in some EUI upgrade without the inMemory presence.

[dej611]

Ok, I think I've tracked down what's going on our side: the sorting feature in EuiDataGrid was never used and instead the data was sorted externally due to the requirement of using some custom sorters.
I think this issue can be closed and I've opened this instead: #6867