Release v0.9.0-nightly-20240701

What's Changed

• refactor: add Compactor trait to abstract the compaction by @zyy17 in #4097
• refactor: make region manifest checkpoint ran in background by @MichaelScofield in #4133
• refactor: Decouple dedup and merge by @evenyag in #4139
• feat(pipeline): transform support on_failure by @yuanbohan in #4123
• feat(pipeline): gsub prosessor by @yuanbohan in #4121
• feat: enable metasrv to accept flownode's heartbeats by @fengjiachun in #4160
• feat(pipeline): join processor by @shuiyisong in #4158
• feat: update default size of bgworkers, add hbworkers by @sunng87 in #4129
• feat: implement the OrderedBatchProducer by @WenyXu in #4134
• feat(flow): support datafusion scalar function by @discord9 in #4142
• feat: Implement memtable range by @evenyag in #4162
• refactor!: unify FrontendOptions and DatanodeOptions by using GrpcOptions by @realtaobo in #4088
• chore: bump datafusion version to fix last_value regression by @MichaelScofield in #4169
• feat: update dashboard to v0.5.1 by @ZonaHex in #4171
• chore: enhance add pipeline http api return data by @paomian in #4167
• feat: register flow node by @fengjiachun in #4166
• feat: show database options by @killme2008 in #4174
• feat: use the write runtime to handle the heartbeats by @fengjiachun in #4177
• perf: optimize RecordBatch to HttpOutput conversion by @waynexia in #4178
• feat: PREPARE and EXECUTE statement from mysql client by @CookiePieWw in #4125
• feat: introduce chaos crds by @WenyXu in #4173
• feat: add num_rows and num_row_groups to manifest by @waynexia in #4183
• feat: make RegionScanner aware of PartitionRange by @waynexia in #4170
• fix: region logical regions after catching up by @WenyXu in #4176
• feat: update dashboard to v0.5.2 by @ZonaHex in #4185
• chore: highlight our committers in CONTRIBUTING.md by @tisonkun in #4189
• refactor: make RegionOptions and MergeOutput serializable by @zyy17 in #4180
• fix(flow): fix call df func bug&sqlness test by @discord9 in #4165
• refactor: add region_dir in CompactionRegion by @zyy17 in #4187
• feat(puffin): support lz4 compression for footer by @zhongzc in #4194
• fix: region_peers returns same region_id for multi logical tables by @realtaobo in #4190
• feat: heartbeat task&peer lookup in proc by @discord9 in #4179
• feat(puffin): introduce puffin manager trait by @zhongzc in #4195
• feat(sql): improve interval expression, support shortened version by @etolbakov in #4182
• fix(sql): improve compound signed number processing by @etolbakov in #4200
• test: wait until checkpoint finish by @evenyag in #4202
• feat: Dedup strategy that keeps the last not null field by @evenyag in #4184
• fix: align workflows again for the troublesome GHA by @waynexia in #4196
• fix: format error correctly by @WenyXu in #4204
• feat(puffin): implement CachedPuffinWriter by @zhongzc in #4203
• feat: FlownodeClient by @discord9 in #4206
• feat: bulk memtable codec by @v0y4g3r in #4163
• docs: remove outdated docs by @evenyag in #4205
• refactor: migrate region failover implementation to region migration by @WenyXu in #4172
• feat(puffin): implement CachedPuffinReader by @zhongzc in #4209
• feat: validate partition rule on create table by @waynexia in #4213
• refactor: add SerializedPickerOutput and field modification of CompactorRequest by @zyy17 in #4198
• fix: wrong frontend registration address by @killme2008 in #4199
• feat(servers): make http timeout and body limit optional by @fengjiachun in #4217
• feat(sql): add casting support for shortened intervals by @etolbakov in #4220
• feat: Implements merge_mode region options by @evenyag in #4208
• refactor(flow): make from_substrait_* async& worker handle refactor by @discord9 in #4210
• test: add e2e test for region failover by @WenyXu in #4188
• feat!: reduce sorted runs during compaction by @v0y4g3r in #3702
• feat: output multiple partition in MergeScanExec by @waynexia in #4227
• fix: add serialize_ignore_column_ids() to fix deserialize region options failed from json string by @zyy17 in #4229

Full Changelog: v0.9.0-nightly-20240617...v0.9.0-nightly-20240701