Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unnest query does not allow one expression being referenced multiple times #11198

Closed
duongcongtoai opened this issue Jul 1, 2024 · 6 comments · Fixed by #11577
Closed

Unnest query does not allow one expression being referenced multiple times #11198

duongcongtoai opened this issue Jul 1, 2024 · 6 comments · Fixed by #11577
Labels
bug Something isn't working

Comments

@duongcongtoai
Copy link
Contributor

duongcongtoai commented Jul 1, 2024

Describe the bug

Given this slt

statement ok
CREATE TABLE unnest_table
AS VALUES
    ([1,2,3])
;

select unnest(column1), unnest(column1) + 1 from unnest_table;

Datafusion throws error

External error: query failed: DataFusion error: Error during planning: Projections require unique expression names but the expression "unnest_table.column1 AS unnest(unnest_table.column1)" at position 0 and "unnest_table.column1 AS unnest(unnest_table.column1)" at position 1 have the same name. Consider aliasing ("AS") one of them.

To Reproduce

No response

Expected behavior

unnest(expr) can be referenced multiple times within select statement

Additional context

No response

@duongcongtoai duongcongtoai added the bug Something isn't working label Jul 1, 2024
@duongcongtoai duongcongtoai changed the title unnest query does not allow one expression being referenced multiple times Unnest query does not allow one expression being referenced multiple times Jul 1, 2024
@jonahgao
Copy link
Member

jonahgao commented Jul 2, 2024

This should be a duplicate of #6543

@duongcongtoai
Copy link
Contributor Author

Although theoretically #6543 can solve this ticket, but i think the bug presented here comes from the internal implementation inside try_process_unnest function itself, and can be fixed within this function

CREATE TABLE unnest_table
AS VALUES
    ([1,2,3]);

query TT
explain select unnest(column1), unnest(column1) + 1 from unnest_table;
----
Current plan generated (which causes error)
logical_plan
01)Projection: unnest(unnest_table.column1), unnest(unnest_table.column1) + Int64(1)
02)--Unnest: lists[unnest(unnest_table.column1)] structs[]
03)----Projection: unnest_table.column1 AS unnest(unnest_table.column1), unnest_table.column1 AS unnest(unnest_table.column1)
04)------TableScan: unnest_table projection=[column1]

Better plan generated
logical_plan
01)Projection: unnest(unnest_table.column1), unnest(unnest_table.column1) + Int64(1)
02)--Unnest: lists[unnest(unnest_table.column1)] structs[]
03)----Projection: unnest_table.column1 AS unnest(unnest_table.column1)
04)------TableScan: unnest_table projection=[column1]

We also avoid unnecessary columns generated in the intermediate projection step

@duongcongtoai
Copy link
Contributor Author

let plan = LogicalPlanBuilder::from(intermediate_plan)

At this line, before create new projection nodes, we can deduplicate intermediate unnest expr within inner_projection_exprs

@duongcongtoai
Copy link
Contributor Author

take

@duongcongtoai
Copy link
Contributor Author

duongcongtoai commented Jul 16, 2024

I also found a different error for the following queries:

select unnest(column1) + unnest(column1) from unnest_table;
External error: query failed: DataFusion error: Schema error: No field named unnest_table.column1. Valid fields are "unnest(unnest_table.column1)".

@jayzhan211
Copy link
Contributor

jayzhan211 commented Aug 9, 2024

I think we can change expr from Debug to Display with this issue together. This todo is added in #11797

Expr::Unnest(Unnest { expr }) => {
    // TODO: use Display instead of Debug, there is non-unique expression name in projection issue.
    write!(f, "UNNEST({expr})")
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants