[BUG] Lux read from Pandas dataframe that was retrieved from a SQL database

[compunuke]

I see that you are working on enhancing the capabilities of Lux to work with SQL sources. However, the current functionality appears to be limited to some abilities with PostgreSQL. I have observed that if lux is imported but pandas read_SQL is used, a pandas dataframe is instantiated instead of a LuxDataFrame.

Would it be possible for LuxDataFrame to read or copy this pandas dataframe? If desired, the pandas dataframe could be destroyed afterwards.

[thyneb19]

Hi @compunuke, Thank you for trying out Lux! Currently, we are still in the process of developing Lux to work with SQL sources. In the future, Lux's visualization generation and recommendation systems will be able to sit on top of databases so you will not need to pull in data locally. However these features are still in progress and are limited to PostgreSQL at the moment. If you are interested in using Lux with databases, we would love to learn more about your use case to see how we can make the system better for you and other users.

To answer your question, you can convert a regular dataframe into a LuxDataFrame by doing the following:

pdf = some pandas dataframe
luxdf = LuxDataFrame(pdf)

If you do import lux, the dataframe that results from pd.read_sql should automatically be a LuxDataFrame.
Please let us know if you run into any more issues or have other questions!

[bimaldave]

I'm also having trouble getting lux to work with read_sql. Seems to work ok when reading in a CSV of the same dataset interestingly but even with an explicit conversion its not returning anything:

df = pd.read_sql(query, sql_conn)
luxdf = lux.LuxDataFrame(df)
luxdf.recommendations

[dorisjlee]

Hi @bimaldave, Thank you for raising this issue. We were able to reproduce the error that pd.read_sql is being read in not as a LuxDataFrame, but as a regular Pandas dataframe. We are looking into debugging this issue now and will respond as soon as we fix this.

[dorisjlee]

Hi @bimaldave, @compunuke,
We were able to resolve this issue and make pd.read_sql output a LuxDataFrame.
You can access these updated changes by upgrading to the latest version of Lux via the following instructions:

pip install --upgrade lux-api
jupyter nbextension install --py luxwidget
jupyter nbextension enable --py luxwidget

Please let us know if this addresses the issue above. I'll close this issue for now, but feel free to ping us again if there are any other questions that come up! Thanks again for pointing this out!