Support Spark SQL data source

[andreashimin]

Is your feature request related to a problem? Please describe.
Provides users with the ability to connect and set up Spark SQL as a data source, enabling integration with distributed data processing frameworks directly within Wren AI's connection setup.

Describe the solution you'd like

• Add a Spark SQL connector in ibis-server
• Implement Spark SQL connection handling between ibis-server and Wren UI backend.
• Create a UI form for users to set up the Spark SQL data source (e.g., server address, port, authentication).

For more details, see Creating a New Data Source Connector in our CONTRIBUTING.md.

Additional context
Add any other context or screenshots about the feature request here.

[goldmedal]

The first step of this issue is implementing the SparkSQL connector on Wren engine side. Wren engine is powered by ibis-project to connect to different data sources.

Refer to ibis pyspark backend, you can follow the guide of adding DataSource to:

• define the connection info
• add the connector
• add test cases

By the way, It would be appreciated if you could also survey how to connect to Databricks using ibis-project. 🙇

[ichuniq]

Hi I'm new to this project and would like to work on this.

[chilijung]

@ichuniq great! Here you go!

[Prototype4988]

@chilijung Since there hasnt been any updates for this issue, I hope you dont mind if i can have a look at it.

[chilijung]

@Prototype4988 Sure go ahead!

[chilijung]

@ichuniq Are you working on this? I noticed you submitted a PR for a typo. I think you may be looking into this issue. Please help confirm. If yes, could you also share the correct progress? Thanks.

@Prototype4988 before you start, let me double check and confirm with @ichuniq thanks.

[ichuniq]

Hi @chilijung, I’ve looked into the task but haven’t started implementing it yet due to limited capacity. However, I’ll have some free time between now and the end of October.

[chilijung]

Thank you for the update, @Prototype4988. I will find other issues for you to solve, and I've cc'd you on other ones.

@ichuniq , I look forward to your contribution, and please keep us updated!