Push scalar functions into cross join

[lewiszlw]

Which issue does this PR close?

Closes #.

Rationale for this change

DataFusion CLI v37.0.0, pow function can be pushdown into cross join.

DataFusion CLI v40.0.0, pow function can't be pushdown into cross join.

I'm not sure if all UDFs should be pushdown into cross join, open for discussion.

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

[simonvandel]

👍
I think it would be nice to add the reproducer SQL query as a slt, so this won't regress in the future.

[lewiszlw]

👍 I think it would be nice to add the reproducer SQL query as a slt, so this won't regress in the future.

The previous behavior was that builtin scalar functions could be pushdown, but UDF couldn't. Now all scalar functions are converted to UDF.

[alamb]

Marking as draft as the CI is failing and looks like there is feedback to address (thanks @lewiszlw and @simonvandel !)

[lewiszlw]

The previous behavior was that builtin scalar functions could be pushdown, but UDF couldn't. Now all scalar functions are converted to UDF.

@alamb As the behavior changed, I'd like to receive feedback first, then I'll add test and fix ci.

[alamb]

The previous behavior was that builtin scalar functions could be pushdown, but UDF couldn't. Now all scalar functions are converted to UDF.

I think we should pushdown UDF functions into cross join unless there is some reason not to.

I suspect the reason they were not being pushed down before was simply an oversight rather than a conscious decision

I agree with @simonvandel that ensuring this is covered by tests (if existing tests need to be updated that shows the change is covered).

[alamb]

Makes sense to me -- thank you @lewiszlw and @simonvandel

cc @liukun4515 / @jackwener in case there is something we are missing about why we can't push filters into CROSS JOINs