[Lens][ES|QL] Re-running the query should keep the selected configuration if possible

[stratoula]

Describe the feature:

Right now, every time a user submits a query in the editor, the Lens suggestions api runs and suggests a new chart. This results in changed made in the visualization level to be lost.

It would be cool if we could at least prioritize the activeVisualization chart Type and some of the configuration if is valid. The type will be very easy as the suggestions api can already do it (with the preferredChartType property). The visualization state is lost though (colors, legend settings etc).

We need to:

• ask from the suggestions api to prirotize the current visualizationType
• check if the query hasn't introduced major changes in the columns (check that the used columns in the datasourceState are being returned from the new query and in that case not change the chart (so keep the config)
• if the columns are different we can just keep the visualization type and render the attributes returned from the suggestions api.

[elasticmachine]

Pinging @elastic/kibana-visualizations (Team:Visualizations)

[elasticmachine]

Pinging @elastic/kibana-esql (Team:ESQL)

[stratoula]

As a feature related to ES|QL and of a mid - high impact I am adding this on the ES|QL project team roadmap. Related to #184631

[stratoula]

I think that this needs to wait for the Lens embeddable refactor

[markov00]

this is unblocked now