Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Spark: Cached ClickHouse table is not used by SQL #274

Merged
merged 2 commits into from
Oct 23, 2023
Merged

Spark: Cached ClickHouse table is not used by SQL #274

merged 2 commits into from
Oct 23, 2023

Conversation

zhouyifan279
Copy link
Contributor

Run Spark SQL:

cache table clickhosue.db.tbl;
explain select * from clickhosue.db.tbl;
== Physical Plan ==
*(1) Project [i#776]
+- BatchScan[i#776] class xenon.clickhouse.read.ClickHouseBatchScan RuntimeFilters: []

From the Physical Plan, we found the cached table was not used by this query.

pan3793
pan3793 reviewed Oct 22, 2023
View reviewed changes
clickhouse-core/src/main/scala/xenon/clickhouse/spec/TableSpec.scala
metadata_path: String,
metadata_modification_time: LocalDateTime,
dependencies_database: Array[String],
dependencies_table: Array[String],
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the key point is equals? if so, does Seq work?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seq is not Serializable

@pan3793 pan3793 changed the title [Bug] Cached clickhouse table is not used by SQL Spark: Cached clickhouse table is not used by SQL Oct 23, 2023
@pan3793 pan3793 changed the title Spark: Cached clickhouse table is not used by SQL Spark: Cached ClickHouse table is not used by SQL Oct 23, 2023
@pan3793 pan3793 merged commit b58bb6c into ClickHouse:master Oct 23, 2023
20 checks passed
zhouyifan279 added a commit that referenced this pull request Oct 27, 2023
@zhouyifan279
Spark: Cached ClickHouse table is not used by SQL (#274)
1d377ba
zhouyifan279 added a commit that referenced this pull request Oct 27, 2023
@zhouyifan279
Spark: Cached ClickHouse table is not used by SQL (#274) (#275)
5e83d0c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pan3793 pan3793 pan3793 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@zhouyifan279 @pan3793