Amazon S3 Tables Now Support Apache Iceberg V3 Specification

Amazon S3 Tables now support all data types in the Apache Iceberg V3 specification, enabling developers to build petabyte-scale analytics tables with native handling for semi-structured and geospatial data. Announced by Daniel Abib, the update allows users to create new V3 tables or upgrade existing V2 tables directly within AWS environments. The integration brings deletion vectors, row lineage, and new data types including variant, nanosecond timestamps, geometry, geography, and unknown to cloud data lakes.

Apache Iceberg V3 Upgrades and New Capabilities

Teams running large analytics workloads on Apache Iceberg V2 tables frequently encounter performance bottlenecks as data scales. A compliance request to delete 50,000 user records from a 2-billion-row table typically leaves behind positional delete files that degrade query speeds until a compaction cycle runs. Furthermore, semi-structured events often arrive as JSON strings that require constant parsing at read time, while geospatial coordinates and nanosecond-precision timestamps are frequently forced into string or integer workarounds.

The Apache Iceberg V3 specification addresses these friction points directly. Deletion vectors replace legacy positional delete files with a compact binary format. When a compliance delete executes, the engine writes a single deletion vector file instead of thousands of small deletes, which cuts compaction overhead. Row lineage automatically introduces _row_id and _last_updated_sequence_number to every record, allowing downstream pipelines to identify modified rows without scanning an entire table. Additionally, native data types eliminate string encoding for specialized information. The variant type stores semi-structured data columnarly, shredding fields during writes and using statistics to prune files during queries. Other additions include nanosecond timestamp(tz) for precise timing, geometry and geography for geospatial datasets, and unknown for columns lacking a defined type.

Implementing V3 Tables in Retail Analytics

Retail analytics teams tracking user behavior across web and mobile platforms can use these features to manage varied data shapes within a single table. For example, a clickstream table can be created with a variant payload column using the Iceberg format version three property:

CREATE TABLE my_catalog.namespace.clickstream (
 event_id bigint,
 event_time timestamp,
 user_id string,
 payload variant
)
USING iceberg
TBLPROPERTIES ('format-version' = '3')

Events with different structures can then be inserted without complex schema evolution steps:

INSERT INTO my_catalog.namespace.clickstream VALUES
 (1, current_timestamp(), 'user-42',
 PARSE_JSON('{"action": "purchase", "amount": 99.99, "items": ["laptop_stand"]}')),
 (2, current_timestamp(), 'user-17',
 PARSE_JSON('{"action": "page_view", "url": "/products/webcam", "duration_ms": 4200}'));

Using Amazon EMR Spark, analysts query the variant column directly with variant_get without calling PARSE_JSON at read time:

SELECT
 event_id,
 user_id,
 variant_get(payload, '$.action', 'string') AS action,
 variant_get(payload, '$.amount', 'double') AS amount
FROM my_catalog.namespace.clickstream
WHERE variant_get(payload, '$.action', 'string') = 'purchase'
 AND variant_get(payload, '$.amount', 'double') > 50.00

To activate deletion vectors for write operations, tables are configured in merge-on-read mode:

ALTER TABLE my_catalog.namespace.clickstream
SET TBLPROPERTIES (
 'write.delete.mode' = 'merge-on-read',
 'write.update.mode' = 'merge-on-read',
 'write.merge.mode' = 'merge-on-read'
)

When a delete statement runs, S3 Tables write a small deletion vector rather than rewriting underlying data files, and automated compaction cleans up these files during subsequent maintenance windows.

Migration Pathways and Engine Compatibility

AWS provides backward compatibility to support migrations from V2 to V3. Existing V2 readers continue operating on upgraded tables until systems are ready for full V3 adoption. An existing table can be upgraded atomically without rewriting data:

ALTER TABLE my_catalog.namespace.existing_table
SET TBLPROPERTIES ('format-version' = '3')

During the next compaction cycle, old V2 delete files are removed, and row lineage fields initialize upon the first subsequent data modification. Because this is a one-way operation governed by the Apache Iceberg specification, administrators must verify that all accessing engines support V3 prior to execution.

Incremental pipelines benefit directly from row lineage metadata. By filtering on sequence numbers, downstream jobs process only changed rows:

SELECT *, _row_id, _last_updated_sequence_number
FROM my_catalog.namespace.clickstream
WHERE _last_updated_sequence_number > 42

AWS supports Apache Iceberg across ingestion, storage, catalog, and analytics layers. Storage and automatic optimization are handled by Amazon S3 Tables, data ingestion runs through Amazon EMR Spark, integration and management occur via AWS Glue, and analytics query workloads execute on Amazon Redshift. Both S3 Tables and the AWS Glue Data Catalog support the Iceberg REST Catalog API to maintain interoperability across different engines.

The new V3 data types require an engine built on Apache Spark 4.0 or later, such as AWS Glue 6.0 or later, or Amazon EMR release 8.1 or later. Furthermore, these data types are supported exclusively for tables using the Parquet file format rather than ORC or Avro. Support for Apache Iceberg V3 in Amazon S3 Tables is available across all AWS Regions where S3 Tables operate, incurring no additional charges beyond standard S3 Tables pricing.