v0.5.9

What's Changed

• Add Filtered Distinct Feature by @mbroecheler in #1018

Full Changelog: v0.5.8...v0.5.9

v0.5.8

Resolves an issue when using the avro schema with json formats.

v0.5.7

What's Changed

• Consistent jar handling for Local and Remote UDFs by @nandorsoma in #849
• Fix null values for case insensitive datafetcher (#871) by @henneberger in #872
• Run unit tests on main by @henneberger in #876
• Add checkpoint monitoring to test infrastructure by @henneberger in #929
• Update instructions by @Zesky665 in #936
• Fixed sensor-teaser tests by @Zesky665 in #935
• Include url on timeout exceptions by @velo in #946
• Flink functions test by @Zesky665 in #944
• Add comparison tests by @Zesky665 in #948
• Exclude like from imported flink functions by @henneberger in #950
• Enable Like test by @Zesky665 in #953

New Contributors

• @velo made their first contribution in #946

Full Changelog: v0.5.6...v0.5.7

v0.5.6

What's Changed

• Add more tests, install views by @henneberger in #843
• Fix avro config value by @henneberger in #844
• Adding some more logging to the tests by @Zesky665 in #852
• Enable Separate Validation for the Package Field During Publishing by @nandorsoma in #846
• Add udf support to run/test by @henneberger in #851
• Add compiled plan to flink plan by @henneberger in #853
• Move compiled plan to flink folder by @henneberger in #854

Full Changelog: v0.5.5...v0.5.6

v0.5.5

DataSQRL v0.5.5 🚀🌟

We're excited to bring you another packed release filled with improvements, new features, and fixes to further enhance your DataSQRL experience!

🌟 Major Enhancements:

• Flink 1.19 Support: We've upgrade to Flink 1.19 so you can take advantage of all of the features and bug fixes it offeres.
• Support for Multiple Database Engines: Now, you can connect and operate across multiple database engines, such as Postgres and DuckDB over iceberg effortlessly.
• Prometheus Metrics Endpoint: Added a new Prometheus endpoint for better monitoring and observability of your systems.
• Extended Avro Support: Added more support for avro data types. Be sure to see the timestamp_mapping.legacy flag for avro sources to take advantage of the new data types.
• Revamped Run and Test command: The docker command now supports a run and test target for faster iteration. We've removed the extra docker images so it can run in github actions and more.

📈 General Improvements:

• Better Error Handling in Schema Validation: We've improved error handling in schema validation, reducing the noise and providing clearer feedback when issues arise.
• Timestamps and Avro Schema Enhancements: Improved timestamp reassignment handling and expanded datatype support for Avro schema, making processing more reliable.
• IndexHints in DagPlanner: Added IndexHints for more optimized query execution plans.
• Improved Documentation: Updated and moved documentation to make it more accessible, including new details for Avro schema handling.

🛠️ Change Log:

• Fix generating subscriptions with nullable primary keys #782
• Fix tests in UseCasesFullIT #777
• Change postgres-cdc TableType to VERSIONED_STATE as a temporary fix #766
• Add support for log engine import/export #747
• Add Prometheus metrics endpoint #783
• Support for multiple database engines #784
• Clean up serializer code #791
• Add postgres-cdc to Vert.x #790
• Put subscriptions behind a feature flag #801
• Update README.md #799
• Fixes for timestamp reassignment #805
• Bump Flink to 1.19 #808
• Use greatest function for timestamp calculation #813
• Include more datatypes for Avro schema #811
• Move documentation to the sqrl repository #789
• Fix distinct casing in queries #817
• Add virtual columns and clean up code #815
• Remove count from Flink functions and clean up tests #816
• Set file monitoring to 10 seconds in discovery #820
• Add lowercase to string library #823
• IndexHints in DagPlanner #824
• Add views to Postgres plan #835
• Move to old run command and Flink 1.19 dependencies #836
• Re-enable the new run command #838
• Create Kafka topics from values array #841

New Contributors

• @Zesky665 made their first contribution in #799

Full Changelog: v0.5.4...v0.5.5

v0.5.4

DataSQRL v0.5.4 🚀🌟

In this release, we are thrilled to announce a host of enhancements and new features that significantly improve the capabilities and user experience of DataSQRL.

🌟 Major Enhancements:

• Create Table syntax: Log engine tables can now be created with CREATE TABLE statements to simply adding new tables.
• Subscription and Mutation Generation: New capability to generate GraphQL subscriptions and mutations for all stream tables automatically.
• Iceberg Integration: Added modularity and partition fixes for better Iceberg connector performance with Flink.
• Snowflake Integration: Added snowflake as a query engine
• DuckDB Integration: Extended support with DuckDB to bolster our analytical processing capabilities.

📈 General Improvements:

• String and Array Handling Functions: Improved handling of strings and arrays in data streams for more efficient processing.

🛠️ Change Log:

• Update README documentation by @mbroecheler in #726 and #730
• Add primary key in discovery by @mbroecheler in #739
• Generate and refine subscriptions/mutations by @henneberger in #735, #750, #753
• Add and adjust partitioning for Iceberg by @henneberger in #751
• Implement minor changes in Snowflake and string/array handling by @henneberger in #757, #774, #769
• Add Snowflake in Vert.x by @henneberger in #775
• Additional SQL features and fixes by @mbroecheler in #762, #771

Full Changelog: v0.5.3...v0.5.4

v0.5.3

[maven-release-plugin] copy for tag v0.5.3

v0.4.3

What's Changed

• Add native flink sql as deployment option by @henneberger in #505
• Add flexible schema as a tableapi format by @henneberger in #509
• Add optional engines by @henneberger in #510
• Do not add flink sql file to jar by @henneberger in #512
• Move flexible format to flink-core by @henneberger in #513
• Fix automatic downcast bug for connectors without native json support by @henneberger in #515
• Fix plan json by @henneberger in #516
• Add avro-confluent sink compatibility by @henneberger in #518
• Support key and value for kafka source and sink by @henneberger in #520
• skip adding sql by default for flink engine by @henneberger in #521
• Add native json handling to json format by @henneberger in #523
• Add other functions back on class path by @henneberger in #526

Full Changelog: v0.4.2...v0.4.3

v0.4.2

What's Changed

• Integration test cleanup by @henneberger in #468
• Remove custom function bridging in favor of flink function bridging by @henneberger in #462
• Validator consolidation by @henneberger in #467
• Migrate join types to rel hints by @henneberger in #466
• Migrate RelRule construction for future calcite versions by @henneberger in #464
• Register rules intentionally to resolve infinite loop in enumberable by @henneberger in #480
• Add test for infinite loop replication by @henneberger in #481
• Add date time as type alias by @henneberger in #475
• Add date time scalar from extended-scalars by @henneberger in #488
• Fix run command in README.md by @mbroecheler in #491
• Add video README.md by @mbroecheler in #492
• install pgvector extension for postgres by @wutianchen in #493
• Add profiles to CLI by @henneberger in #490
• Use "normal" aggregation on Stream and support Stream joins by @mbroecheler in #499
• Update json functions to handle nulls by @henneberger in #503

New Contributors

• @wutianchen made their first contribution in #493

Full Changelog: v0.4.1...v0.4.2

v0.4.1

Many minor bug fixes from v0.4.0

What's Changed

• Enable table shadowing for all tables by @henneberger in #435
• Add json tests, fix secure data type, expand calcite exec test by @henneberger in #440
• Support array types for subscriptions by @henneberger in #441
• Improve distinct error messages by @henneberger in #442
• Canonicalize path to lowercase for build dir by @henneberger in #444
• Fix duplicate error messages in tests, remove old snapshots by @henneberger in #445
• Remove volumes from docker compose by @henneberger in #446
• Add path to package error message by @henneberger in #443
• Allow empty query type by @henneberger in #447
• Support graphql schema root by @henneberger in #448
• Support enums in graphql for queries by @henneberger in #449
• Support ordered unions by @henneberger in #450
• Fix book examples by @henneberger in #452
• Add stub agg implementor (#453) by @henneberger in #455
• Invoke type inference on json functions by @henneberger in #458
• Validate all columns are named (#406) by @henneberger in #454
• Add Json concat (#457) by @henneberger in #459
• Add readme to udf example, fix minor udf related bugs by @henneberger in #461
• Allow != for not equals by @henneberger in #465
• Use alias name for udfs by @henneberger in #469
• Add Flink job listener pluggability by @henneberger in #470

Full Changelog: v0.4.0...v0.4.1