Skip to content

SETL-0.4.3

Latest
Compare
Choose a tag to compare
@qxzzxq qxzzxq released this 10 Jul 14:35
· 92 commits to master since this release

Changes:

  • Updated spark-cassandra-connector from 2.4.2 to 2.5.0
  • Updated spark-excel-connector from 0.12.4 to 0.13.1
  • Updated spark-dynamodb-connector from 1.0.1 to 1.0.4
  • Updated scalatest (scope test) from 3.1.0 to 3.1.2
  • Updated postgresql (scope test) from 42.2.9 to 42.2.12

New Features:

  • Added pipeline dependency check before starting the spark job
  • Added default Spark job group and description
  • Added StructuredStreamingConnector
  • Added DeltaConnector
  • Added ZipArchiver that can zip files/directories

Fixes

  • Fixed path separator in FileConnectorSuite that cause test failure
  • Fixed Setl.hasExternalInput that always returns false