Add add_files procedure in Iceberg connector

[erikerlandson]

like so:
https://github.com/RussellSpitzer/iceberg/blob/a4279fc5842046043f2afdc90f2428243958574d/spark3-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestAddFilesProcedure.java#L80

[osscm]

+1

[erikerlandson]

One of the use cases I had in mind for this was that the files I want to add are sitting on some s3 bucket. So in this use case,there needs to be a way to supply add_files with s3 credentials, as parameters.

[alexjo2144]

Similarly to the Hive connector's allow-register-partition-procedure this should be disabled by default and opted in using a catalog property. The idea being that it should only be turned on if file system location based access control is in place.

[blopezpi]

Any updates on this? It will be great to have this procedure for importing a bunch of data directly avoiding any insert command.

[anandsakhare]

+1

[ebyhr]

@martint Could you review the syntax of this procedure? The procedure name in Spark is add_files. The detailed information is documented at https://iceberg.apache.org/docs/latest/spark-procedures/#add_files

The arguments should follow Trino conventions (e.g. source_table should not be abused for locations), but the name looks good to me.

[MichaelTiemannOSC]

Awesome works. Can't wait to try it out in 461!