spec: clarify partition value when writing data files

[kevinjqliu]

Apache Iceberg version

None

Query engine

None

Please describe the bug 🐞

Relates to #14914

In the "Writing data files" section of the spec, it mentions that
"""
All columns must be written to data files even if they introduce redundancy with metadata stored in manifest files (e.g. columns with identity partition transforms).
"""

However, in the "Column Projection" section of the spec, the partition value can be missing from the data files for identity transforms. The identity partition values can be projected by the reader to support the hive migration use case.

We should clarify the "Writing data files" and mention this particular edge case

Willingness to contribute

• I can contribute a fix for this bug independently
• I would be willing to contribute a fix for this bug with guidance from the Iceberg community
• I cannot contribute a fix for this bug at this time

[singhpk234]

i believe the intention of this pr : https://github.com/apache/iceberg/pull/10835/changes was the same.
we want to store the column in the data files always irrespective of whether its redundant i.e in case of identity transform, the exception for the case of hive migrated table is because as part of the migration we just write metadata files and hive doesn't store the partition values in the data files itself it stores it in the directory path structure, so a wording is there, the identity tranform is percolated all the way down to the reader when reading such files.

can you please elaborate, what part of spec requires further clarification?