Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error creating table from Pandas DataFrame #2487

Open
greyscaled opened this issue Jan 24, 2024 · 4 comments
Open

Error creating table from Pandas DataFrame #2487

greyscaled opened this issue Jan 24, 2024 · 4 comments
Assignees
Labels
bug Something isn't working
Milestone

Comments

@greyscaled
Copy link
Contributor

greyscaled commented Jan 24, 2024

If I switch to_pandas to to_polars, everything works as expected.

import glaredb
import pandas as pd
import polars as pl

con = glaredb.connect("glaredb://...")

df = con.sql("SELECT * FROM './b.parquet'").to_pandas()

con.execute("create table test as SELECT * FROM df")

External error: Execution error: failed to create table in storage: Schema error: Invalid data type for Delta Lake: Null


I did not test if this is specific to Parquet, and so it might be useful to just create a simple df in pandas and try the same thing.

I suggest we add a Python test covering this case.

@greyscaled greyscaled added the bug Something isn't working label Jan 24, 2024
@tychoish tychoish added this to the next milestone Jan 25, 2024
@universalmind303
Copy link
Contributor

@greyscaled do you have an example of the parquet file you used for this?

@greyscaled
Copy link
Contributor Author

greyscaled commented Jan 29, 2024

parquet_file.zip

Had to upload in a zip - GitHub doesn't seem to allow direct uploading of a .parquet file

@universalmind303
Copy link
Contributor

hmm, yeah i'm not seeing the same error, but a different one

exceptions.ExecutionException: Error during planning: Unable to fetch table provider for 'test': Not a Delta table: no log files

@greyscaled
Copy link
Contributor Author

@universalmind303 ah, this is my bad. The error referenced in the original post is when I attempt to query the table.

The Python script exits without error, but SELECT * FROM ... does. My apologies.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants