bug: read_json doesn't work with data over http

[talagluck]

Description

I would expect this to work:

SELECT * FROM read_json('https://opdb.org/api/search/typeahead?q=*');

Instead, it raises External error: Accessing delta tables over http not supported

[tychoish]

This is a known issue: the object_store crate that we use for accessing data doesn't handle query strings. @vrongmeal has this in his queue.

I'm not sure that it's captured in a different issue though, so it's probably good to track it here.

[vrongmeal]

The best I could get is:

> select * from read_json('https://opdb.org/api/search/typeahead?q=*');
Error: External error: Missing content-length header

I think @universalmind303 has more context about this ^

[vrongmeal]

I've made this work:

> select * from read_json('http://localhost:3004/?path=test.json');
┌───────┬─────────┐
│   _id │ name    │
│    ── │ ──      │
│ Int64 │ Utf8    │
╞═══════╪═════════╡
│     1 │ Vaibhav │
│     2 │ Tal     │
└───────┴─────────┘

But the mentioned URL fails due to lack of "content-length" header. We need some other strategy to handle the case. Maybe download the contents locally into a file (similar to SQLite?)

PS: Query params working?

[vrongmeal]

So... have something working!

> select * from read_json('https://opdb.org/api/search/typeahead?q=*');
┌─────────────┬──────────────────────────────────┬──────────┬────────────────────────┬──────────────┐
│ id          │ text                             │ name     │ supplementary          │ display      │
│ ──          │ ──                               │ ──       │ ──                     │ ──           │
│ Utf8        │ Utf8                             │ Utf8     │ Utf8                   │ Utf8         │
╞═════════════╪══════════════════════════════════╪══════════╪════════════════════════╪══════════════╡
│ G5W1w-MkPy7 │ "300" (Gottlieb, 1975)           │ "300"    │ Gottlieb, 1975         │ reels        │
│ GrqL6-MLy7l │ 15 Pool (Playmec Flippers, 1976) │ 15 Pool  │ Playmec Flippers, 1976 │ reels        │
│ GRBWj-MQKlO │ 2 in 1 (Bally, 1964)             │ 2 in 1   │ Bally, 1964            │ reels        │
│ G48nw-M5RPZ │ 2001 (Gottlieb, 1971)            │ 2001     │ Gottlieb, 1971         │ reels        │
│ G48W0-MDy3v │ 21 (Williams, 1960)              │ 21       │ Williams, 1960         │ lights       │
│ GrEkZ-ML13O │ 24 (Stern, 2009)                 │ 24       │ Stern, 2009            │ dmd          │
│ Gr22Q-MQkbn │ 250 cc (Inder, 1992)             │ 250 cc   │ Inder, 1992            │ alphanumeric │
│ GR0Q1-MkPPd │ 3 Coins (Williams, 1962)         │ 3 Coins  │ Williams, 1962         │ reels        │
│ GrkK0-MDXxZ │ 3 Jokers (Williams, 1970)        │ 3 Jokers │ Williams, 1970         │ reels        │
│ GRz0q-MLe9N │ 3-D (Williams, 1958)             │ 3-D      │ Williams, 1958         │ lights       │
│ …           │ …                                │ …        │ …                      │ …            │
│ GRBDj-MJ9x6 │ Apollo (Williams, 1967)          │ Apollo   │ Williams, 1967         │ reels        │
│ G411e-MJrEL │ Apollo 13 (Sega, 1995)           │ Apollo … │ Sega, 1995             │ dmd          │
│ GRQVw-Mx06k │ Aqualand (Juegos Populares, 198… │ Aqualand │ Juegos Populares, 1986 │ alphanumeric │
│ GreKP-MLXzY │ Aquarius (Gottlieb, 1970)        │ Aquarius │ Gottlieb, 1970         │ reels        │
│ G59Xv-MJdYp │ Arabian Knights (Gottlieb, 1953) │ Arabian… │ Gottlieb, 1953         │ lights       │
│ G59dD-M7ZRb │ Arabian Night (Sega, 1976)       │ Arabian… │ Sega, 1976             │ reels        │
│ GrNyB-MDBkE │ Arcade (Williams, 1951)          │ Arcade   │ Williams, 1951         │ lights       │
│ G4EKb-MDx4p │ Arena (Gottlieb, 1987)           │ Arena    │ Gottlieb, 1987         │ alphanumeric │
│ Gr31Y-MQ9bZ │ Argosy (Williams, 1977)          │ Argosy   │ Williams, 1977         │ reels        │
│ G5VNb-MQoZ7 │ Army Navy (Williams, 1953)       │ Army Na… │ Williams, 1953         │ reels        │
└─────────────┴──────────────────────────────────┴──────────┴────────────────────────┴──────────────┘
 100 rows (20 shown)

[talagluck]

Incredible! Is this merged, or still in progress?

[vrongmeal]

Incredible! Is this merged, or still in progress?

Waiting for a review on #2784. Should be in soon!