All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog and this project adheres to Semantic Versioning.
- Upgraded shunting-yard parent from
hotels-oss-parent
toeg-oss-parent
. - Upgraded version of
hive.version
to2.3.7
(was2.3.4
). Allows Shunting Yard to be used on JDK>=9. - Upgraded
hcommon.hive.metastore
from1.4.1
to1.4.2
. - Upgraded
circus.train
from15.0.0
to16.3.0
. - Upgraded AWS SDK (
aws.version
) from1.11.415
to1.11.852
. - Upgraded
micrometer.version
(in shunting-yard-replicator) from1.0.5
to1.5.4
.
- Circus Train
replication-Mode
will beFULL
forinsert overwrite
operation on source table (wasMETADATA_UPDATE
). - Circus Train
replication-Mode
will always beFULL
forALTER_PARTITION
event.
- Upgraded
hotels-oss-parent
to 4.1.0 (was 4.0.1). - Upgraded
hcommon-hive-metastore
to 1.4.1 (was 1.0.0). - Upgraded Circus Train version to 15.0.0 (was 12.0.0).
- Support for skipping Circus Train's housekeeping process.
- Maven group ID is now com.expediagroup.shuntingyard (was com.hotels.shunting.yard).
- All Java classes moved to com.expediagroup.shuntingyard (was com.hotels.shunting.yard).
- hotels-oss-parent version updated to 4.0.1 (was 2.3.5).
- Updated configuration properties to not have package names in them.
FilteringMessageReader
now deletes filtered messages to mimic previous behaviour of delete on read.
- Excluded
org.pentaho.pentaho-aggdesigner-algorithm
dependency as it's not available in Maven Central.
- Support for specifying target database and table name.
- Extracted Apiary SQS receiver logic to library in
apiary-receivers
. - Moved SQS message deletion out of receiver and into
MessageReaderAdapter
. - Upgraded
apiary-receiver-sqs
to 2.0.0 (was 1.4.0).
- Support for handling new elements (location and table parameters) coming in from Apiary events.
- Support for performing metadata only sync in alter table and alter partition event.
- Support for passing certain configurations like Graphite directly to internal CT instance using
--ct-config
argument.
- log4j.xml to be packaged with Shunting-Yard Binary.
- Upgraded
hive
to 2.3.4 (was 2.3.0). - ContainerCredentialsProvider to a more robust EC2ContainerCredentialsProvider.
- Support to read AWS Credentials from within the Elastic Container Service Task using ContainerCredentialsProvider.
- Refactored project to remove checkstyle and findbugs warnings, which does not impact functionality.
- Upgraded
hotels-oss-parent
to 2.3.5 (was 2.1.0).
- Support for selecting the tables to be replicated #6.
- Enforce exception handling #2.
- Event aggregation based on time windows #4. This is a breaking change since the event model has been changed to a more suitable structure for Circus Train.
- Support for handling Hive Metastore Events from Apiary.
- Receivers & emitters for Kinesis and Kafka.
- Generate a fat emitter JAR per emitter implementation #13.
- First release.