Allow optional strands

Contents (#688)

New features

• Allow optional strands

Dependencies

• Use twined=0.6.0

Fixes

• Skip dataset validation for missing optional manifests

Add documentation on updating Octue services

Contents (#683)

Operations

• Use latest ruff pre-commit check

Dependencies

• Add ruff to dev dependencies

Other

• Add doc on updating an Octue service

Switch to ruff developer tooling

Contents (#682)

Operations

• Switch from flake8, black, and isort to ruff

Dependencies

• Remove old formatters/linters and add ruff config

Refactoring

• Apply ruff to all files

Check for service revision existence

Contents (#680)

IMPORTANT: There is 1 breaking change.

Enhancements

• 💥 BREAKING CHANGE: Use cloud URIs by default for datasets in output manifests
• Add comments around checking for service revision existence
• Improve error when octue.services topic doesn't exist

Fixes

• Raise error if service revision subscription doesn't exist when no service registry is in use
• Remove octue.services prefix from subscription names

Refactoring

• Avoid repeated conversion to Pub/Sub ID for a service

---

Upgrade instructions

💥 Use cloud URIs by default for datasets in output manifests

Set use_signed_urls_for_output_datasets to True in the app configuration to keep using signed URLs for datasets in output manifests.

Revert analysis output location removal

Contents (#677)

Fixes

• Pass output arguments into Analysis and use them

Reversions

• Revert "REF: Stop storing output_location in Analysis"

Make signed URLs for output datasets optional

Contents (#676)

IMPORTANT: There is 1 breaking change.

Enhancements

• Allow using non-signed URLs for datasets in output manifest (controllable via the app configuration file)
• Handle all requests errors while:
  • Getting cloud metadata for datafiles and datasets
  • Downloading datafiles

Fixes

• Avoid trying to access buckets for URL datasets

Refactoring

• 💥 BREAKING CHANGE: Stop storing output_location in Analysis
• Remove unnecessary finalisation from template apps

---

Upgrade instructions

💥 Stop storing `output_location` in `Analysis`

If calling Analysis.finalise manually, either stop doing this and rely on the output_location field of the app configuration or explicitly pass in the upload_output_datasets_to argument.

Improve event filtering

Contents (#673)

IMPORTANT: There are 2 breaking changes.

New features

• Add dictionary_product utility function

Enhancements

• 💥 BREAKING CHANGE: Disable event validation in EventReplayer by default
• 💥 BREAKING CHANGE: Enable filtering by multiple event kinds in get_events
• Enable excluding multiple event kinds in get_events
• Use all non-question events for question redelivery check in flask app
• Add ability to skip handling logs containing certain text in AbstractEventHandler and subclasses
• Return outside of ThreadPoolExecutor context managers

---

Upgrade instructions

💥 Disable event validation in `EventReplayer` by default

Set validate_events=True in the EventReplayer constructor to retain the previous behaviour.

💥 Enable filtering by multiple event kinds in `get_events`

To filter by one event kind as before, use kinds=[event_kind] instead of kind=event_kind.

Enable question retries on single questions

Summary

This release adds to the question retry capability already available on concurrent questions by allowing retries of single questions.

Contents (#671)

Enhancements

• Enable question retries on single questions with Child.ask
• Log when retries are prevented for an exception type in Child.ask
• Remove PYTHONUNBUFFERED warning

Refactoring

• Move retry logic from Child.ask_multiple into Child.ask

Speed up event replaying

Contents (#669)

Enhancements

• Skip non-result event validation if only result is required
• Add ability to skip event validation in event handlers
• Make diagnostics log messages more consistent
• Allow instantiation of Diagnostics, Topic, Subscription, and GoogleCloudPubSubEventHandler without cloud credentials

Refactoring

• Update from deprecated datetime.datetime.utcnow method
• Use cached_property in Service
• Remove unused attributes on MockService and Runner

Testing

• Implement MockSubscription.delete

Update child emulator and improve manifest dataset download

Contents (#668)

IMPORTANT: There are 2 breaking changes.

Enhancements

• 💥 BREAKING CHANGE: Update ChildEmulator to use EventReplayer, support schema-compliant events and attributes, and support heartbeats and delivery acknowledgement events. This significantly simplifies the emulator
• 💥 BREAKING CHANGE: Remove ChildEmulator.from_file
• Download manifest datasets to same directory by default

Refactoring

• Move ServicePatcher into its own module

---

Upgrade instructions

💥 Update `ChildEmulator` to use `EventReplayer` and full events

Give events (including attributes) that satisfy the service communication schema to child emulators.

💥 Remove `ChildEmulator.from_file`

Load the JSON file separately and pass the events into the ChildEmulator constructor.