Add OpenDAL Support

[Xuanwo]

Hello, I'm initiating this thread to discuss the potential benefits of integrating opendal support. OpenDAL offers a unified data access layer, enabling seamless and efficient data retrieval from a variety of storage services. It currently supports over 30 different storage services, making it a valuable complement to object_store.

OpenDAL has its own rust crate, opendal, and can be natively integrated with object_store through object_store_opendal. I believe the maintenance overhead will be minimal.

I am eager to assist in implementing this feature and help maintain it.

What are your thoughts?

[ion-elgreco]

For which object stores do you want to add support? We need some minimal supported requests to support the optimistic concurrency requirement.

I can see put_opts and copy_if_not_exists aren't implemented

[Xuanwo]

We can integrate native support for Aliyun OSS, Huawei Cloud OBS, Tencent COS, and more. Additionally, we plan to add Oracle Cloud OCIO support in the near future.

[ion-elgreco]

We can integrate native support for Aliyun OSS, Huawei Cloud OBS, Tencent COS, and more. Additionally, we plan to add Oracle Cloud OCIO support in the near future.

Alright, we should then have put_opts (putIfAbsent) available for these stores.

Are there any standard URI prefixes these stores use?

[Xuanwo]

Alright, we should then have put_opts (putIfAbsent) available for these stores.

Thank you for the information. I will double-check whether they correctly support If-None-Match.

Are there any standard URI prefixes these stores use?

Yes. I believe oss://, cos:// is widely used.

[Xuanwo]

Hi, your current preference is to have deltalake-aliyun / deltalake-tencentcloud instead of deltalake-opendal, correct?

[ion-elgreco]

Deltalake-opendal can also work but then you would have to configure, each of the stores in there according to the same manner of deltalake-aws/azure/gcs etc