Incremental Strategies:
- Added a new incremental strategy named "microbatch" in the
valid_incremental_strategies
method offabric_adapter.py
and corresponding SQL macros. [1]
Snapshot Handling:
- Updated the
fabric__build_snapshot_table
andfabric__snapshot_staging_table
macros to use configurable snapshot column names, enhancing flexibility and readability. [1] [2] - Improved the
fabric__snapshot_merge_sql
macro to handle dynamic column names and added checks fordbt_valid_to_current
. [1] - Modified
snapshot.sql
to integrate the new snapshot staging table logic and ensure proper column handling. [1] [2] - Added unique_key as list and hard deletes support to snapshots. [1]
Testing:
- Introduced new tests for the microbatch incremental strategy in
test_incremental_microbatch.py
, including setup for input models and insertion of test data.
Full Changelog: v1.8.9...v1.9.0