RELEASE-NOTES.md

Ziggy Release Notes

These are the release notes for Ziggy. In the change log below, we'll refer to our internal Jira key for our benefit. If the item is associated with a resolved GitHub issue or pull request, we'll add a link to that. Changes that are incompatible with previous versions are marked below. While the major version is 0, we will be making breaking changes when bumping the minor number. However, once we hit version 1.0.0, incompatible changes will only be made when bumping the major number.

v0.7.0: Halloween release

This release is coming out just before Halloween, and it's full of tricks and treats. Behind the scenes, we continued to buy down decades of technical debt. Are we finally getting close to paying off that loan? By eliminating the StateFile API (ZIGGY-465) and fixing ZIGGY-432, ZIGGY-454, and ZIGGY-478, the pipeline no longer stalls or crashes for mysterious reasons. We fixed a few UI annoyances like collapsing tree controls and Control-Click not working as expected on the Mac.

New Features

1. Rename sub-task to subtask (ZIGGY-79)
2. Use log4j2 conventions and features (ZIGGY-82)
3. Provide means for clients/algorithms to add their software version to Ziggy's data accountability (ZIGGY-430)
4. Fix Group design (ZIGGY-431)
5. Refactor PipelineTask (ZIGGY-433)
6. Limit console to operations (ZIGGY-441)
7. Add PipelineInstanceIdOperations methods to PipelineInstanceOperations (ZIGGY-445)
8. Support copying files to task directory without datastore (ZIGGY-446)
9. Retrieve DatastoreRegexps from the database by name (ZIGGY-447)
10. Locate consumed files used to produce a file (ZIGGY-448)
11. Ensure importer can add and update module and pipeline definitions (ZIGGY-452)
12. Write HDF5 files usable by Zowie (ZIGGY-455)
13. Eliminate the need for programmatic appenders (ZIGGY-456)
14. TaskMonitor doesn't change processing step from QUEUED to EXECUTING (ZIGGY-457)
15. Add parameter retrieval to PipelineTaskOperations (ZIGGY-460)
16. Check for new vs existing files in datastore (ZIGGY-461)
17. Eliminate StateFile API (ZIGGY-465)

Bug Fixes

1. Double-click resize is lost when table auto-update occurs (ZIGGY-297)
2. Collapsing Parameter Library and Pipelines tree controls (ZIGGY-360)
3. Can't halt SUBMITTED tasks (ZIGGY-424)
4. Resume monitoring can't be stopped (ZIGGY-425)
5. Race condition in pipeline workers (ZIGGY-432)
6. Ziggy C++ Mex build tools set incorrect install name (ZIGGY-444)
7. Warning alert clears error alert status (ZIGGY-450)
8. Python distutils module removed from Python 3.12 (ZIGGY-451)
9. Local processing crashes sporadically (ZIGGY-454)
10. ZiggyQuery chunkedIn doesn't work (ZIGGY-462)
11. Remote execution dialog can't parse numbers with commas (ZIGGY-463)
12. Parameter API populates empty arrays (ZIGGY-468)
13. Module parameter sets in HDF5 have incorrect field order values (ZIGGY-469)
14. Worker never exits when subtask errors (ZIGGY-478)
15. Control-Click clears selection on the Mac (ZIGGY-479)
16. Exceptions when using pipeline instance filters (ZIGGY-489)

v0.6.0: You never have to wonder what Ae 4 / 3 / 0 means again

We fixed a confusing aspect of the user interface and a ton of bugs while we continued to buy down decades of technical debt. You can now halt tasks or instances from the command-line interface (CLI). We improved pipeline definitions by making datastore definitions more flexible and providing for user-specified data receipt unit of work (UOW) generators.

New Features

1. Combine task state and p-state on tasks display (ZIGGY-166)
2. Review Edit Trigger Validate button for deletion (ZIGGY-224)
3. Untangle table hierarchies in pipeline definitions (ZIGGY-268)
4. Put back the executableName attribute for pipeline modules (ZIGGY-281)
5. Remove DatabaseModelRegistry and ConsoleDatabaseModel (ZIGGY-311)
6. Merge CLI cancel and reset commands into a single halt command (ZIGGY-389)
7. Replace isEmpty() with isBlank() (ZIGGY-395)
8. Add regular expression support to data file type fileNameRegexp (ZIGGY-398)
9. Remove last circular dependency in the ui package (ZIGGY-407)
10. Support user-specified data receipt UOW generators (ZIGGY-408)
11. Clean up CRUD proxies in console (ZIGGY-410)
12. Move all database transactions to Operations classes (ZIGGY-419)
13. Clean up Parameters namespace (ZIGGY-421)

Bug Fixes

1. Pipeline doesn't error out when no UOWs are generated (ZIGGY-304)
2. Remote ComputeNodeMaster instances exit prematurely (ZIGGY-353)
3. Fix deprecated use of DefaultExecutor (ZIGGY-399)
4. Can't rename a group (ZIGGY-402)
5. Alerts panel shows hostname, not localhost (ZIGGY-403)
6. A slow data receipt module freezes the console (ZIGGY-411)
7. RemoteJob infrastructure doesn't detect failed jobs are finished (ZIGGY-414)
8. Restarting local tasks with resubmit hangs (ZIGGY-415)
9. Rename creates a copy of the pipeline (ZIGGY-416)
10. Averaging module writes results in dataset/L3 (ZIGGY-417)
11. Copy command behaves inconsistently (ZIGGY-418)
12. Setting debug logging level on messaging package hoses RMI (ZIGGY-422)

v0.5.0: A major overhaul of the datastore, some UI improvements, and documentation for the command-line interface

The title pretty much says it all. There was also a lot of internal refactoring to buy down more technical debt.

New Features

1. Review javadoc warnings (ZIGGY-126)
2. Remove GUI code from ProcessHeartbeatManager (ZIGGY-261)
3. Move remote execution configuration from parameters to pipeline node definition (ZIGGY-280)
4. Remove timestamp from pipeline instance name when launched by event handler (ZIGGY-299)
5. Add architecture and queue information to remote execution dialog (ZIGGY-306)
6. Increase number of digits displayed by cost estimate (ZIGGY-352)
7. Functionality inherited from Spiffy (ZIGGY-364)
8. Create ZiggyCliTest and add documentation to user manual (ZIGGY-370)
9. Remove security packages (ZIGGY-371)
10. Eliminate default from unit of work machinery (ZIGGY-374)
11. Refactor Parameters a bit more (ZIGGY-378)
12. Redesign datastore and data file type APIs (ZIGGY-380)
13. Implement subqueries in ZiggyQuery (ZIGGY-381)
14. Simplify ZiggyConfiguration and reduce test fixtures (ZIGGY-382)

Bug Fixes

1. Max worker count not set correctly (ZIGGY-302)
2. Timestamp fields not initialized (ZIGGY-375)
3. Reserved queue dialog issues (ZIGGY-377)
4. Exceptions running ziggy.pl with missing properties (ZIGGY-391)
5. Ziggy supervisor logging no longer goes to supervisor.log (ZIGGY-392)

v0.4.1: Fixed halt task and restart task commands

As promised, the Halt tasks (formerly Kill tasks) and Restart tasks commands have been fixed, along with a handful other buglets.

New Features

Bug Fixes

1. Remove option of UOW reuse (Incompatible change, ZIGGY-278)
2. Replace == with equals for non-primitives (ZIGGY-287)
3. Can't kill (local) tasks (ZIGGY-290)
4. Can't restart tasks (ZIGGY-291)
5. Incorrect number of downstream tasks (ZIGGY-303)
6. Increase pause after starting database (ZIGGY-354)
7. Undesired reprocessing (ZIGGY-361)
8. Tests sometimes fail in Eclipse (ZIGGY-367)
9. Unable to kill or restart tasks for first pipeline instance (ZIGGY-368)
10. Close and Cancel buttons in wrong order on resources dialogs (ZIGGY-372)
11. No transition after error resume (ZIGGY-373)

v0.4.0: Hibernate 6, reorganized properties, an improved UI, runjava renamed to ziggy

Last time we said that our next release will contain the result of replacing our TESS data analysis pipeline infrastructure with Ziggy. That work continues, so we'll try to get back into a regular cadence of releases to avoid astronomically large releases, like this one.

As promised, we reorganized our properties and eliminated the effects of 15 years of entropy. We also updated the version of Hibernate we use and updated the database schema.

The UI witnessed a major facelift, and runjava was renamed to ziggy and the --help option works consistently with ziggy and its commands. The sample pipeline now uses an internal HSQLDB database, so it's even easier to try.

The version generator was redone to avoid rebuilding the world every time. Third-party sources have been moved from buildSrc to outside, where they are still safe from gradle clean.

The Kill tasks and Restart tasks commands are broken and will be fixed shortly in 0.4.1.

New Features

1. Switch to Hibernate 6 (Incompatible change, ZIGGY-5)
2. Add additional queue support (ZIGGY-92)
3. Review handling of checked exceptions (ZIGGY-149)
4. Clean up of StateFile name / path management (ZIGGY-152)
5. Respond to requested changes on Ziggy remote execution dialog (ZIGGY-167)
6. Reorganization of console tabs and content (ZIGGY-169)
7. Clean up properties (Incompatible change, ZIGGY-172)
8. Replace ZiggyVersionGenerator / ZiggyVersion classes (ZIGGY-181)
9. Implement parameter set groups (ZIGGY-221)
10. Revise pipeline priorities (ZIGGY-225)
11. Changes needed for TESS-Ziggy (ZIGGY-240)
12. Implement subworkers (ZIGGY-242)
13. Add per-module thread settings (ZIGGY-243)
14. Switch sample pipeline to HSQLDB (ZIGGY-250)
15. Second Generation Messaging System (ZIGGY-259)
16. Refactor OpsInstancesPanel (ZIGGY-263)
17. Remove BeanWrappers class (ZIGGY-301)

Bug Fixes

1. runjava --help should provide help for (ZIGGY-155)
2. Incorrect version of wrapper jarfile (ZIGGY-196)
3. Configuration is mutable and use of system properties is misleading (ZIGGY-201)
4. cluster init --force can't delete write-protected files (ZIGGY-231)
5. Fix Remote Execution typical wall time label (ZIGGY-232)
6. Remote parameter set not updated by remote execution dialog (ZIGGY-233)
7. Ziggy FileUtil.cleanDirectoryTree() fails with symlinks (ZIGGY-236)
8. ZiggyTable wrapping text doesn't support text color (ZIGGY-246)
9. A little time in the great outdoors, er, outside (ZIGGY-247)
10. Improve runjava console user experience (ZIGGY-251)
11. Database classes have ambiguous versioning (ZIGGY-256)
12. Remove task counts from PipelineInstanceNode (ZIGGY-262)
13. Delay task request messages until tasks are committed to database (ZIGGY-269)
14. Unable to start pipeline from selected node (ZIGGY-270)
15. Allow ZiggyCppPojo to use correct compiler for a given source file (ZIGGY-279)
16. Editing a parameter clobbers other parameters in the set (ZIGGY-286)

v0.3.1: Fixed CITATION.cff and some news

This release includes a fix to our CITATION.cff courtesy of @dieghernan.

We are now working towards replacing our TESS data analysis pipeline infrastructure with Ziggy. Our next release will contain the result of that work. It's a large effort and we expect it to take at least two months if not longer.

We are still planning to reorganize our properties and eliminate the effects of 15 years of entropy. We'll also be updating the version of Hibernate we use and anticipate updating the database schema as a result. You have been warned! This will occur before the 1.0 release to maximize our chances of stability after that.

New Features

Bug Fixes

1. Correct conference section in CITATION.cff (ZIGGY-241, pull #1)

v0.3.0: Java 17, Gradle 7, and a new event manager

This release includes an update to the Gradle build system and an upgrade of Java to Java 17. It introduces an event manager system so that Ziggy can respond automatically to external events. The user manual was expanded to cover this feature and a few others.

We are planning to reorganize our properties and eliminate the effects of 15 years of entropy. We'll also be updating the version of Hibernate we use and anticipate updating the database schema as a result. You have been warned! This will occur before the 1.0 release to maximize our chances of stability after that.

New Features

1. Upgrade to Java 17 (Incompatible change, ZIGGY-22)
2. Add static analysis to build (ZIGGY-30)
3. Create prototype event manager system (ZIGGY-119)
4. Clean up unit test execution on Gradle (ZIGGY-180)
5. Create user manual, part II (ZIGGY-193)
6. Remove Sockets from Subtask Server and Client (ZIGGY-199)
7. Update copyright (ZIGGY-220)

Bug Fixes

1. Fix build warnings in both src and buildSrc (ZIGGY-142)
2. Delete setPosixPermissionsRecursively with a single permission (ZIGGY-197)
3. Ziggy "keep-up" processing fails (ZIGGY-204)
4. Correct P1 bugs identified by SpotBugs (ZIGGY-207)
5. NAS does not support Java 17 (Incompatible change, ZIGGY-234)

v0.2.2: More documentation goodness

This release adds Previous, Next, and Up buttons to the user manual to make it easier to read cover to cover. We also added a CITATION.cff file to make it easier for you to cite Ziggy in your own work. Finally, we changed some 3-byte quotes to ASCII as these quotes could not be compiled if LANG is C.

We have updated the Gradle build system and Java to Java 17. This change will appear in version 0.3.0.

New Features

1. Add "Prev", "Next", and "Up" buttons to user manual articles (ZIGGY-160)
2. Update GitHub documentation (ZIGGY-200)

Bug Fixes

1. Fix funny quotes in NASA notices (ZIGGY-188)

v0.2.1: Add GitHub info to docs

Once we uploaded our first version to GitHub, we could fill in some documentation TODOs like how to download the code.

We are still in the process of updating the build system Gradle and Java to at least Java 11 and possibly Java 17. We'll take advantage of post-Java 8 versions at that time. This change will appear in version 0.3.0.

New Features

1. Ziggy reuse (ZIGGY-134)
2. Update GitHub info in manual (ZIGGY-192)

Bug Fixes

v0.2.0: Initial release

This is the first Ziggy release to appear on GitHub. The major number of 0 indicates that we're still refactoring the Kepler and TESS codebases and reserve the right to make breaking changes from time to time as we make the pipeline more widely useful. However, the general pipeline infrastructure has been in production use since Kepler's launch in 2009.

This is the first Ziggy release.

We are in the process of updating the build system Gradle and Java to at least Java 11. It's possible that we will take advantage of post-Java 8 versions at that time. This change will appear in version 0.3.0.

0.1.0

This release was overtaken by events. This release came and went before we got authorization to push our code to GitHub.