2.9.0: Pebble log forwarding, unit testing actions, and more

This release makes it easy to write tests for your Charm actions, aligns testing secrets with upcoming Juju changes, and exposes the ability to use the new Pebble log forwarding functionality. There's also the usual collection of documentation improvements, bug fixes, and other minor improvements.

Unit Testing Actions

Writing unit tests for actions with Harness is now as simple as:

def test_action(harness):
    out = harness.run_action("my-action-name", {"param1": value, "param2": value2})
    assert out.results == {"result1": "expected"}

Added in #1053

Secrets

In the original implementation of Juju secrets, the owner of a secret would automatically peek (get the latest revision) at secret contents - this also meant that using 'refresh' was not allowed (since there was no revision tracking). This behaviour causes difficulties in some situations, so is being changed in the next versions of Juju (back to 3.1.7), removing the automatic peek and allowing refresh.

In ops, we are switching to the new behaviour, and this applies regardless of the detected/set JujuVersion. This primarily applies to Harness, which was updated in #1067, but also to the ops documentation. The constraints outside of Harness are applied by Juju, so your charms will reflect the new behaviour when Juju updates.

Note that you can get the old behaviour by always using refresh=True if that's what you prefer (to support both old and new Juju, you'll need to wrap this so that in older Juju you retry when it complains that you can't refresh).

We've also fixed the simulated access to app secrets for non-leaders in #1076.

Pebble Log Targets

Pebble 1.4 added the ability to configure log forwarding for services to a Loki server, and Pebble 1.6 extends this with the ability to add custom labels to the forwarded logs. This functionality is now exposed in ops via a new (optional) log-targets section in Pebble layers and plans. Added in #1074

Documentation Improvements

• A fresh README that's simpler and better aligns with the docs and other projects in the Juju universe in #1052 - many thanks to @tmihoc for this contribution!
• Clarified how custom events are emitted in emit() in #1072
• Fixed the Harness.get_filesystem_root example in the docs in #1065

Minor Improvements

• Removed the executable bit from all the .py files by in #1059 - thanks @jameinel!
• The test suite now passes under Python 3.12 in #1081

Fixes

• The SQLite storage file is no longer group/other readable. We strongly recommend against storing any secret data in stored state (use secrets), but we are aware of cases of this in the wild. This change prevents other users in the containers from being able to access the saved state file. Existing storage files will be chmod'd to 0o600 when ops first opens them after upgrading to 2.9.0+. #1057
• When parsing a timestamp returned from Pebble, there was a 1 in 2 million chance that ops would fail with a ValueError. Fixed in #1084
• Container.push_path and Harness.list_files would fail if the files/folders involved belonged to a user or group ID that did not have a name (for example, had been removed). Fixed in #1082. Thanks to @nedbat for the report!

Full Changelog: 2.8.0...2.9.0

2.8.0: Unit.reboot, RelationMeta.optional, and type-checked tests

This release includes a few small improvements: a new function Unit.reboot() (for machine charms only), the "optional" attribute of relations in metadata.yaml is now available in the RelationMeta object, and if the Pebble socket is missing (for example, if the container has just rebooted) we provide a clearer exception:

• feat: add Unit.reboot for machine charms in #1041
• feat: add RelationMeta.optional in #1038
• feat: provide a clearer exception when the Pebble socket is missing in #1049

We fixed a couple of small issues:

• fix: push_path and pull_path include empty directories in #1024
• fix: reset collected statuses in Harness.evaluate_status in #1048

We've also made a few small improvements to the documentation:

• docs: note that status changes are immediate in #1029
• docs: add note to ActionEvent.set_results about maximum size in #1047
• docs: document more of the exceptions that may be raised in #1044
• docs: make pebble.Client.remove_path and Container.remove_path docs consistent in #1031
• docs: update the PyPI development status to production/stable in #1020

Finally, we wrapped up our work on #1007, so the test suite is now fully type checked, did a couple of other minor type hinting improvements, and broke out the 'real' Pebble tests into a separate file for more clarity (you can still just run tox -e pebble):

• test: add type hints to test_testing in #1017
• test: add type hints to test_model in #1015
• test: add type hints to test_charm in #1022
• test: add type hints to test_storage in #1023
• test: add type hints to test_framework in #1025
• test: add type hints to test_main in #1028
• test: add type hints to pebble-related tests in #1030
• test: separate the real pebble tests into a separate module in #1018
• chore: remove as much 'if typing.TYPE_CHECKING:' as possible in #1034
• feat: narrow the type for 'app' and 'unit' in relation events by in #1032

Full Changelog: 2.7.0...2.8.0

2.7.0: Unit.set_ports(), and minor docs, CI, and test improvements

This release includes one new feature, a new function Unit.set_ports() to declare which ports should be open. We recommend using this method rather than managing multiple Unit.open_port() and Unit.close_port() calls:

• feat: add Unit.set_ports() for declarative port opening #1005

We've also made a few small improvements to the documentation:

• docs: reduce the amount of detail in the open/close port methods in #1006
• docs: remove 'you' and 'your' from docstrings in #1003
• docs: minor cleanup of the HACKING doc in #1016

Finally, we've also made a start expanding our type hinting to cover the test suite, and made a couple of minor improvements/fixes to our CI:

• test: add type hints to the test_infra tests in #1008
• test: add type hints to test_log tests by in #1009
• test: fix type hinting on test_jujuversion tests in #1011
• test: fix type hint warnings in test_private tests in #1012
• test: add type hinting to test_lib by in #1012
• test: add type hinting to test_helpers in #1014
• ci: use Go 1.20 for real Pebble tests in #1004
• ci: re-enable integration tests with mysql-*operator by in #1013

(Thanks @tonyandrewmeyer for your first contributions, and welcome to the team!)

Full Changelog: 2.6.0...2.7.0

2.6.0: Harness.handle_exec, Harness.add_relation improvements, and more

This release has two significant improvements to testing.Harness:

• Add Harness.handle_exec to allow charm tests to simulate Container.exec calls (Pebble one-shot commands) and provide expected output. Thanks @weiiwang01 for the design and implementation (PR 993).
• Simplify adding relation units and relation data by adding app_data and unit_data parameters to Harness.add_relation. Previously, charm tests would have to call add_relation followed by add_relation_unit and update_relation_data -- now only a single add_relation call is required. Implemented in PR 994.

In addition, several small improvements and fixes are included:

• feat(model): add Secret.unique_identifier (XID part); improve id/label docs in #986
• fix(testing): ignore push() encoding if source is a binary file or stream in #991
• fix(pebble): remove use of deprecated cgi module in Pebble code in #996
• fix(model): ensure Secret.get_content returns a copy of the dict in #1000
• fix(model): make Secret.set_content invalidate local cache by in #1001

See all changes from 2.5.0 to 2.6.0.

2.5.1: Fix str vs bytes issue with testing push()

This is a patch fix on top of 2.5.0 to fix #990: "TypeError: write() argument must be str, not bytes" in Container.push under Harness.

2.5.0: Collect-status, Harness.get_filesystem_root, Pebble service context

This release includes three significant additions:

1. Multi-status: An initial version of the much-discussed "multi-status" handling in the form of the new collect_app_status and collect_unit_status events. This is useful for letting the framework automatically evaluate application (or unit) status from various components of the charm. We'll be documenting this properly soon, but in the meantime, read the API reference docs on CollectStatusEvent. PR #954.
2. Harness.get_filesystem_root: Overhauling the Harness's test filesystem from an in-memory filesystem to use a temp directory on the real filesystem; use Harness.get_filesystem_root to get the temp filesystem directory for a specific container. This makes testing container filesystem operations significantly easier and allows tests to use the regular Python file APIs to interact with it. Thanks @weiiwang01 for his efforts on this in #960.
3. Service context: Added support for Pebble's new "service context" feature, to allow you to exec commands in the context of a specified service (where context means environment variables, working directory, and user/group). PR #957.

Other notable changes:

• Add JujuVersion.supports_open_port_on_k8s by @carlcsaposs-canonical in PR #965
• Support reading charmcraft.yaml in the testing harness by @syu-w in PR #977

In addition, the following developers made their first contribution:

• @paulomach in #975 (Add kill-delay to pebble ServiceDict type)
• @sobolevn in #978 (Replace typing.AnyStr with Union[str, bytes])
• @syu-w in #977 (Support reading charmcraft.yaml in testing harness)

We also made several other minor improvements to type annotations and a few other tweaks. See the full changelog: 2.4.1...2.5.0

2.4.1: Add type annotation for PebbleReadyEvent.workload

This is a patch release: a minor but important type annotation forgotten in 2.4.0. Without it, charms that use type annotation on the pebble-ready events will fail type checking, because attributes of the workload won't be available as Pyright/MyPy doesn't know the type.

Fix in #962.

2.4.0: Improvements to API docs and type annotations

This release has no new functionality, but includes various improvements to API docs (docstrings) and type annotations, as well as a couple of minor bug fixes. Our API reference docs on Read The Docs look nicer, are more consistent, and have better types.

One change that may affect charms that use type checking: the new ops.X syntax will now work with type checkers (Pyright and MyPy) due to adding an explicit __all__ to ops/__init__.py (#937). Oh, and we fixed a bug with relative paths in Container.push_path -- thanks @yanksyoon!

See the full list of PRs merged below.

What's Changed

• Lock docs dependencies; build docs with recent Sphinx version by @benhoyt in #938
• Explicitly add all public ops.X names to __all__ in __init__.py by @benhoyt in #937
• Use Canonical-ish theme from canonical/sphinx-docs-starter-pack by @benhoyt in #941
• Use PurePath type annotation when path casted to string by @carlcsaposs-canonical in #946
• Update Pyright version to latest (1.1.313) by @benhoyt in #944
• Fix sphinx-build warnings and turn warnings into errors by @benhoyt in #942
• Reinstate alertmanager-k8s-operator CI tests now that issue is fixed by @benhoyt in #951
• Remove jargon from and simplify README by @benhoyt in #950
• Fix issue with relative paths in Container.push_path by @yanksyoon in #949
• Avoid error in FileInfo repr due to permissions being None by @benhoyt in #956
• Various docstring and type annotation updates by @benhoyt in #953
• Add "-> None" to methods without other types by @benhoyt in #961

New Contributors

• @carlcsaposs-canonical made their first contribution in #946

Full Changelog: 2.3.0...2.4.0

2.3.0: Fix planned_units to exclude dying

This release includes only one minor functional changes: a fix to Application.planned_units to exclude dying units from the total (#936).

Also included is a type annotation fix to an internal testing method (#926).

To use this version, update the ops line in your requirements.txt to ops==2.3.0.

1.5.5: Backport "error" status fix to 1.5

This releases the backport of the "error" status fix #875 to the 1.5-maintenance branch (PR #927).