feat: data source from JSON array data

[Lilit0x]

Closes: #2218

How to Test

SELECT * FROM read_arjson('path/to/file/or/urls', max_size, <creds>)

[tychoish]

I'm super excited about this!

I wonder if implementing the object store's file type ends up having you write more boilerplate than is needed.

I had great luck recently using the StreamingTable infrastructure in the bson functionality, though I don't know if it makes a lot of sense (also your code supports gzipped inputs and the BSON code does not.)

[Lilit0x]

I wonder if implementing the object store's file type ends up having you write more boilerplate than is needed.

You mean the FileFormat? Yes, it did. Especially in the create_physical_plan function because of the execution. I tried to look for a way around that, but I couldn't since the default JsonFormat is new line delimited which gives more credibility to @universalmind303's suggestion to making the change upstream. For now though, I think it is okay like this, albeit, I'll still have to write more for the create_writer_physical_plan.
The other option I had was to create a different type similar to FileType and then implement all the traits that datafusion's FileType did, but that would have been extremely longer and probably unnecessary.

I had great luck recently using the StreamingTable infrastructure in the bson functionality, though I don't know if it makes a lot of sense (also your code supports gzipped inputs and the BSON code does not.)

Hmm, I looked through it, I can't seem to see where and how it will fit this use case. Maybe I am missing something.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ Lilit0x
❌ Ubuntu

---

Ubuntu seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[tychoish]

I think given that df419aa has landed, we should let this one drop.

I think there's a lot of merit to this approach, and I'm super thankful that you did this work for us to learn from, and sorry that we didn't use the code directly.