Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Clickhouse followup items #2315

Closed
6 tasks done
scsmithr opened this issue Dec 27, 2023 · 0 comments · Fixed by #2340
Closed
6 tasks done

Clickhouse followup items #2315

scsmithr opened this issue Dec 27, 2023 · 0 comments · Fixed by #2340
Assignees
@vrongmeal

Comments

@scsmithr
Copy link
Member

scsmithr commented Dec 27, 2023 •
edited by vrongmeal
Loading

Follow up items for Clickhouse support added in #2300.

  • Virtual listing (and related SLTs)

  • Handle empty tables

    Currently we require that tables have at least one item since we're detecting the table schema from the blocks that clickhouse returns. Clickhouse does not seem to return blocks on queries that return nothing.

    glaredb/crates/datasources/src/clickhouse/mod.rs

    Lines 81 to 84 in c50dda2

    // TODO: Does clickhouse actually return blocks for empty data sets?
    let mut blocks = client
    .query(format!("SELECT * FROM {name} LIMIT 1"))
    .stream_blocks();

  • Add WHERE clause in the scan

  • Add LIMIT clause in the scan

    glaredb/crates/datasources/src/clickhouse/mod.rs

    Lines 230 to 232 in c50dda2

    // TODO: Where, Limit
    let query = format!("SELECT {} FROM {};", projection_string, self.table);

  • Data types SLTs

  • Ensure this works with clickhouse cloud
@scsmithr scsmithr mentioned this issue Dec 27, 2023
Merged
scsmithr added a commit that referenced this issue Dec 28, 2023
@scsmithr
feat: Clickhouse data source (#2300)
eaf795a 
Adds support for clickhouse as a data source.

`read_clickhouse`:
```
> select * from read_clickhouse('clickhouse://localhost:9000/default', 'bikeshare_stations') limit 1;
┌────────────┬──────────┬────────┬─────────┬───┬─────────────────┬───────┬──────────────────┬───────────────────┐
│ station_id │ name     │ status │ address │ … │ footprint_width │ notes │ council_district │ modified_date     │
│         ── │ ──       │ ──     │ ──      │   │              ── │ ──    │               ── │ ──                │
│      Int32 │ Utf8     │ Utf8   │ Utf8    │   │         Float32 │ Utf8  │            Int32 │ Timestamp<s, UTC> │
╞════════════╪══════════╪════════╪═════════╪═══╪═════════════════╪═══════╪══════════════════╪═══════════════════╡
│          0 │ South C… │ active │ 1901 S… │ … │            10.0 │ In t… │                9 │ 2022-03-04T09:01… │
└────────────┴──────────┴────────┴─────────┴───┴─────────────────┴───────┴──────────────────┴───────────────────┘
```

External database:
```
> create external database ch
::: from clickhouse
::: options ( connection_string = 'clickhouse://localhost:9000/default' );
Database created
> select status, address from ch.default.bikeshare_stations limit 1;
┌────────┬──────────────────────────┐
│ status │ address                  │
│ ──     │ ──                       │
│ Utf8   │ Utf8                     │
╞════════╪══════════════════════════╡
│ active │ 1901 South Congress Ave. │
└────────┴──────────────────────────┘
```

External table:
```
> create external table stations
::: from clickhouse
::: options ( connection_string = 'clickhouse://localhost:9000/default',
:::           table = 'bikeshare_stations' );
Table created
> select council_district, modified_date from stations limit 1;
┌──────────────────┬─────────────────────┐
│ council_district │ modified_date       │
│               ── │ ──                  │
│            Int32 │ Timestamp<s, UTC>   │
╞══════════════════╪═════════════════════╡
│                9 │ 2022-03-04T09:01:00 │
└──────────────────┴─────────────────────┘
```

---

Follow up items: #2315
@vrongmeal vrongmeal mentioned this issue Jan 4, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vrongmeal vrongmeal

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: Clickhouse followup items GlareDB/glaredb
2 participants
@scsmithr @vrongmeal