Add support for add_files and add_files_from_table procedures in Iceberg

[ebyhr]

Description

Spark supports adding files from tables or locations with add_files procedure. This procedure is helpful for migrating specific Hive partitions, or importing files on the storage. It would be nice to support the same procedure in Trino Iceberg connector.

This PR adds add_files and add_files_from_table procedures.

In add_files procedure, `recursive_directory argument is optional:

ALTER TABLE testdb.testdb EXECUTE add_files(
    location => 's3://my-bucket/a/path',
    format => 'ORC',
    [recursive_directory => 'true'])

In add_files_from_table procedure, partition_filter and recursive_directory arguments are optional:

ALTER TABLE testdb.testdb EXECUTE add_files_from_table(
    schema_name => 'testdb',
    table_name => 'hive_customer_orders',
    [partition_filter => map(ARRAY['region'], ARRAY['AMERICA'])], 
    [recursive_directory => 'true'])

The add_files procedure is disabled by default with iceberg.add-files-procedure.enabled config property because OSS Trino doesn't support location based access control.

Fixes #11744

Release notes

# Iceberg
* Add support for `add_files` and `add_files_from_table` procedures. ({issue}`11744`)

[alexjo2144]

These checks may be a little strict, if Iceberg supports coercion from the source column to the target type it should be okay as is.

Having additional columns in the source than the target also doesn't hurt, we'll just ignore them.

It is harder to mess up a call to the procedure this way, but I guess it's a question of how we expect people to use this. With perfectly matching schemas, or ones that have been tweaked a little.

[ebyhr]

I think we should start with strict checks and tweak them based on user feedback.

The common use case looks adding files from Hive partitions after the initial migration. For instance, migrating the entire table on July 24, adding files from July 25 partition next day, .... until the end of migration.

[cwsteinbach]

Does Spark's version of add_files enforce strict checks, or is it permissive?

I'm also in favor of defaulting to strict mode, but giving the user the ability disable it with a non-strict boolean flag.

[ebyhr]

#23677

[SemionPar]

LGTM!

[alonahmias]

What about sorted by? Is there a way to also allow sorted by columns in this way?

[raunaqmorarka]

@ebyhr please update description to match updated syntax

[martint]

Let's split the function for adding files from a path and from another table into separate ones. Having overloads where most of the arguments are different is error prone and confusing for users.

[ebyhr]

@martint Separated into add_files_from_location and add_files_from_table procedures. Let me know if you want to rename them.

[martint]

A couple of questions/comments:

• The _from_location suffix is necessary. Since we're talking about files, it's natural to think they are in object storage, at a given location. The _from_table suffix is necessary, though 1) to avoid the overload 2) since loading files from a table is a special case.
• Do we expect to be able to add files from other sources (e.g., Delta Lake tables) at some point?

[ebyhr]

@martint Removed _from_location suffix. There is no concrete plan to support other table formats, but it's a potential enhancement. I expect changing add_files_from_table behavior internally instead of adding a new procedure so that users can avoid rewriting procedure name based on source table types.