Support adding a field with ADD COLUMN in Iceberg

[ebyhr]

Description

Relates to #16897
Fixes #16248

Release notes

(x) Release notes are required, with the following suggested text:

# General & Iceberg
* Add support for adding nested fields with an `ADD COLUMN` statement. ({issue}`16248`)

[findepi]

"Parse column name as QualifiedName in column definition"

[ebyhr]

Addressed comments.

[kasiafi]

@ebyhr @findepi in my opinion this new functionality is not "ADD COLUMN". It effectively doesn't add a column, but it changes the type of an existing column. Hence, it falls under "ALTER COLUMN".

The user could probably use the existing syntax ALTER COLUMN ... SET DATA TYPE ... to add a nested field. I understand however that this syntax is very verbose, as it requires to explicitly describe the requested type. I think that we could extend the ALTER COLUMN syntax to support adding a nested field in a more concise form.
It could be something like:

ALTER COLUMN <column_name> ADD FIELD <field_name> <type>

The <field_name> could be an identifier for an immediately nested field, and a qualified name for deeper nesting (anyway, the passed name should be relative to the <column_name>).

Execution-wise, the two ways of adding a column, ALTER COLUMN ... SET DATA TYPE ... and ALTER COLUMN ... ADD FIELD ... should be unified. The new syntax should be considered syntactic sugar.

All the above applies as well to removing nested fields (#16002).

[martint]

Yes, adding nested fields should be handled with SET DATA TYPE. We talked about it nested fields when adding support for that functionality, but stopped before defining the syntax required.

[findepi]

adding nested fields should be handled with SET DATA TYPE.

I agree that adding nested fields should be possible via SET DATA TYPE.

it is, however, impractical to require users to provide current ROW definition plus the new field when they want to add a new field, that's why we need option to add/remove individual fields, or users will not use Trino for that. I don't think there is any disagreement, I just wanted to make sure others have the context.

Execution-wise, the two ways of adding a column, ALTER COLUMN ... SET DATA TYPE ... and ALTER COLUMN ... ADD FIELD ... should be unified. The new syntax should be considered syntactic sugar.

I would be hesitant about that. ALTER COLUMN ... ADD FIELD ... can be executed atomically by a connector, while the equivalent of reading the type, adding a field to the type and invoking SET DATA TYPE cannot.

[ebyhr]

@martint Could you take another look?

[ebyhr]

Rebased on master to resolve conflicts.

[ebyhr]

Rebased on master to resolve conflicts.

[ebyhr]

@martint Could you please take another look?

[findepi]

@ebyhr the build isn't green

[ebyhr]

The failure #18202 is unrelated to this change.

[findepi]

let's call it add field

Addressed comment.

[ebyhr]

Let me merge this PR today if there are no additional comments.

[martint]

I’m still reviewing