[Vis Builder] Vis Builder meta issue

[ashwin-pc]

Parent issue to track the different task for the Drag and Drop experience.

• Feedback: [Feedback] Visualization Builder - Experimental Release #2280

Tasklist:

Framework

• Drag and drop entrypoint [D&D] Add Icon to the "Create new visualization" popup #959
• State management [D&D] Add state management for the Drag & Drop plugin #947
• Visualization service [D&D] Add Visualization service #952
  • Contributions [D&D] Viz Service: Contributions #1274
  • Remove contributions [D&D] Remove contributions #1705
  • Expressions [D&D] Visualization type pipeline #1275
  • Agg service/viz class research [D&D] Aggregation service and Viz class research spike #1699
• Save and embed
  • Save visualization [Vis Builder] Save Visualization #1620
  • Add to Dashboard via saved object [D&D] Add to Dashboard via saved object #1703
  • Add to Dashboard directly from dashboard [D&D] Add to Dashboard directly from editor #960
  • Add to Dashboard directly from visualization [VisBuilder] When creating wizard from visualization page, provide additional options to add to new/existing dashboard, or add to library #2534
• Localization [Vis Builder] Add Localization  #958
• Testing
  • Initial unit tests [D&D] Add unit test coverage #1704
  • Integration and E2E tests [D&D] Functional Tests for the Drag and Drop experience #961
  • Increase test coverage [VizBuilder] Increase test coverage #2380
• Drag and drop [Vis Builder] Add drag and drop to main canvas #956
• Initial visualizations [Vis Builder] Add visualization types #957
  • Metric [D&D] Add Metric Visualization #1614
  • Datatable [D&D] Datatable visualization #1615
  • Bar [D&D] Bar Chart #1616
  • Line [D&D] Line Chart #1617
  • Area [D&D] Area chart #1618
  • Pie [VisBuilder] Add Pie Chart visualization type #1619

UI

• Add Topnav [D&D] TopNav Index pattern #949
  • Persistance [Vis Builder] Persist the global query parameters including global filters, time range and time filters #1913
• Add Index pattern selector [D&D] [UI] Update index pattern selector #948
• Add breadcrumbs [D&D] Add breadcrumbs #1621
• Empty workspace polish [D&D] empty workspace polish #1684
• Field summary view [D&D] Add Field summary view #950
• Field search [D&D] Field Search in Data panel #951
• Viz style [D&D] Add Visualization styling tab #953
• Viz toolbar [Vis Builder] Add Visualization controls #955
• Viz type selector [D&D] Visualization type selector #954

Meta

• Documentation [Vis Builder] Drag and drop experience documentation #962
• Replace "wizard" with real name [D&D] Replace wizard naming convention #1706
• Plan to release as beta/preview feature [D&D] Add support for release as beta/preview feature #1708

---

• Original Design: [Design] Drag & Drop Visualization CX #896
• Proposal: [Proposal] VisBuilder - Drag & Drop Visualization CX #883

[seanneumann]

For reference the RFC: #896

[seanneumann]

I'd like to make sure we start talking about two requirements:

• Supporting PPL
• Supporting multiple data sources.

We don’t need to implement them in the initial release, but should include them in the design so they will be incrementally added.

[ashwin-pc]

I'm currently working on the design for #1275 which will cover these two topics. But just to highlight the general approach, we will be using the expressions language already built into OpenSearch Dashboards to create a pipeline that a visualization will use to render given the various user inputs. Current visualizations also use the same flow.

A sample pipeline:

 opensearchDashboards                                                   // Retrieves global information for the dashboards query and filters
 | opensearch_dashboards_context query='QUERY' filters='[]'             // Retrieves Visualization specific information such as datasource and filters
 | opensearchaggs index='INDEX_ID'  aggConfigs='AGGREGATION_CONFIG'     // Performs aggregation
 | opensearch_dashboards_pie visConfig='VISUALIZATION_CONFIG'           // Renders visualization

The implementation details for each function in the pipeline is separate from the function calling it. This way we can swap out any section of the pipeline with a different datasource and/or different query engine to render the same visualization as long as the inputs and outputs to the other functions in the chain remain unchanged.

[kavilla]

[Groom]:
We should update this issue with our 2.2 planned deliverables.

@ahopp @opensearch-project/opensearch-ux the current 2.2 planned deliverables aren't feature complete and should be indicated as such. So that it should be configurable to toggle this on and off.

[seanneumann]

Hey team, I'd like to propose a new name for this functionality. While drag and drop is a part of the story, we should give it a name that is more encompassing. Here are some thoughts:

1. Vis Builder
2. Vis Wizard
3. Vis Designer
4. Vis Maker
5. Data Explorer

Let's discuss.

cc: @elfisher @ashwin-pc @ahopp

[kgcreative]

I'm partial to:

1. Vis builder
2. Data explorer

[xeniatup]

1. Viz builder - because it explains what it does really clearly
2. Data Wizard - hints on ease of use and emphasizes the underlying data

[btzeng]

Vis Builder

[abbyhu2000]

Vis builder or vis maker make the most sense for me:)

[seanneumann]

The official proposed name is "Visualization Builder"

@elfisher do you want to sign off on this?

[elfisher]

The official proposed name is "Visualization Builder"

@elfisher do you want to sign off on this?

Sounds good. I see there is a lot of momentum in 2.4 for this project. Are we overall on track to finish the tasks in this meta in 2.4?

[ashwin-pc]

Unfortunately no, we have focused our 2.4 efforts to harden the feature and fix bugs. I'm also hoping to have it enabled by default in 2.4.

Here is what we will actually be shipping for 2.4

• [VizBuilder] Increase test coverage #2380
• [D&D] Add to Dashboard directly from editor #960
• [VisBuilder] When creating wizard from visualization page, provide additional options to add to new/existing dashboard, or add to library #2534
• [Vis Builder] Persist the global query parameters including global filters, time range and time filters #1913
• [D&D] Replace wizard naming convention #1706
• Open bugs

Edit:

1. [VizBuilder] Increase test coverage #2380: All major functional tests complete (Unit tests will be covered in 2.5)
2. [VisBuilder] When creating wizard from visualization page, provide additional options to add to new/existing dashboard, or add to library #2534: Has blockers that are called out in the issue
3. [Vis Builder] Persist the global query parameters including global filters, time range and time filters #1913: Might be pushed to 2.5. Ran into an issue that that will take a little more time to resolve
4. [D&D] Datatable visualization #1615: @ananzh has picked this up as a part of [Initiative] Remove AngularJS as dependency #1560
5. [D&D] Add Field summary view #950: Also for 2.4
6. [Vis Builder] Enabled by default #2672: Also for 2.4

cc: @seanneumann

[ashwin-pc]

Hello all, I wanted to provide an update on the project given the upcoming 2.7 release and the latest status.

Upcoming release (2.7)

In the next release, we have planned the following features and improvements:

Highlights

1. Data persistence: We will now be persisting the data across refreshes, switching chart types and even chart specific information like color.
2. 2 way communication: The visualization types can now interact with the applicaation to set properties such as timeranges and filters.
3. Bug fixes: There are a lot of minor bugs that individually dont break the expereince of the app, but collectively make it a lot easier to use the tool

The detailed list of changes are below:

• [Vis Builder] Agg data persistence when switching among area, line and and histogram vis type #3157
• [Vis builder] Persist data when switching visualization types #2763
• [Vis Builder] Data persistence Meta Issue #2762
• [Proposal] [Vis Builder] Aggregation Persistence in VisBuilder #3482
• [VisBuilder] Remove warning popup for vis type change #3647
• [BUG] Firefox - the "Legend position" selector in Vis Builder doesn't allow changes #3134
• Fix type errors in Visualization Builder #3025
• [BUG][VisBuilder] Error when validate schema in vis builder #3276
• [Discuss] [Vis Builder] Visualizations should be able to interact with VisBuilder  #2868
• [BUG] [Vis Builder] Incorrect index pattern causes multiple errors #2643
• [Vis Builder] Support UiState in Vis Builder #3650
• [VisBuilder] Persist visualization internal state #3124
• [Vis Builder] Add validation steps between serialization/deserialization and state sync #1881
• [Vis Builder] Complete unhappy save paths (error handling flows) #1880
• [BUG] Can't get index-pattern fields at dashboard filter for vis-builder #3429

Latest status

To highlight the progess of the project as it stands now, here are some of them:

• Completed: 30/35 Originally identified subtasks
• Closed issues: 157
• Bugs: Only 2 non blocking bugs will remain in the 2.7 release.

Historical timeline

• Original release with Metric Visualization in 2.2. Brand new with the foundations setup.
• Released Bar, Line and Area charts using the existing charting libraries in 2.3
• Significantly improved test coverage, Added Table Vis and Field summary views in 2.4
• Fixed a lot of bugs, enabled the feature by default and renamed it to VisBuilder across the app in 2.4
• 2.5 &2.6 releases were focussed on incremental improvements to the user experience, focussed mainly around app persistence