2.17.0: new unit testing API, and Secret fixes

What's Changed

This release adds a state-transition testing API for unit tests, formerly known as Scenario (many thanks to @PietroPasotti for his work developing and maintaining the library). Harness is still available, but we encourage all charmers to use the new API for unit tests going forward. The API is an optional install, for example: pip install ops[testing], so that the test API code does not get bundled into the charms. All of the classes that you would previously find when using ops-scenario in the scenario namespace can then be found in ops.testing. For example:

from ops import testing

ctx = testing.Context(MyCharm)
state = ctx.run(ctx.on.start(), testing.State(leader=True))
assert state.unit_status == testing.ActiveStatus()

There are also two improvements to Juju Secrets: firstly, the .id of a Secret will always include the model UUID, so it can be reliably used for cross-model relations. A more significant change is that, previously, if a charm called set_contents() and set_info() in the same hook (even if not the same event handler, or in a deferred event handler) only the last call would have any effect. This is now changed so that the calls accumulate - you can set both content and metadata in the same hook, and also if you set_contents() twice the contents will be combined. The Juju team intend to make this the secret-set behaviour in the future as well.

Additionally, ops.main is now type hinted correctly and will no longer require a type: ignore directive from users!

Features

• Optionally install Scenario with ops[testing] and expose the names in ops.testing in #1381
• Change ops.main() so that you don't need to type: ignore it in #1345
• Expand the secret ID out to the full URI when only given the ID in #1358
• Add a JujuVersion property for Pebble log forwarding to Loki in #1370
• Pre-emptively raise InvalidStatusError instead of waiting for Juju:
  • Make it an error to call CollectStatusEvent.add_status with error or unknown in #1386
  • Document and validate settable status values in _ModelBackend.set_status in #1354

Fixes

• Fix type of StatusBase subclasses by calling StatusBase.register in __init_subclass__ in #1383
• Secret.set_info and Secret.set_content can be called in the same hook in #1373

Documentation

• Add top-level intro and module-level intros in #1320
• Update the links to the Pebble docs in #1362
• Note about repeatedly setting secret value in Juju 3.6 in #1366
• config-changed is triggered by Juju trust in #1357
• Typo on CharmBase inheritance example by @theofpa in #1349
• Docs: move Pebble to a separate page in #1392

Continuous Integration

• Periodically run the unit tests of all GitHub-hosted published charms in #1365
• Update the TIOBE reporting for the changes in coverage calculation in #1367
• Spell-check the code as part of linting in #1388
• Run the smoke tests on a schedule in #1387

Testing

• Fix tests that leaked environment variables in #1385

Refactoring

• Move the content of ops.testing to ops._private.harness in #1369
• Keep the unittest.mock names in the 'mock' namespace in #1379
• Deprecate StatusBase.register decorator in #1384

Chores

• Note Juju version on legacy workaround in #1355
• Re-enable test now that Pebble directory permissions are fixed in #1363
• Generate warnings for events that will be removed in Juju 4.0 in #1374

New Contributors

• @theofpa made their first contribution in #1349

Full Changelog: 2.16.1...2.17.0

2.16.1 Don't have Harness alter os.environ

This is a small bug-fix release to address a regression in ops 2.16.0 where creating a Harness object would add a JUJU_VERSION to os.environ.

Fixes

• Don't alter os.environ in Harness in #1359

Full Changelog: 2.16.0...2.16.1

2.16.0

This release adds a private _JujuContext dataclass used to parse all `JUJU_*`` environment variables in one place; besides, a significant amount of work has been done on tests, continuous integration and documentation.

Features

• Add the description field to SecretInfo in #1338

Refactor

• Parse JUJU_* env in one place in #1313

Fixes

• Juju passes the expiry in a field 'expiry', not 'expires' in #1317
• Correct the signature of .events() in #1342

Documentation

• Security policy change to only support each active major release in #1297
• Add example Juju version markers in #1311
• Use Sphinx 8 in #1303
• Live reload with sphinx-autobuild in #1323

Tests

• Update the smoke test series/bases in #1318
• Run pytest in parallel with pytest xdist in #1319
• Bump pyright to 1.1.377 in #1332
• Run tests on Py 3.12 and install test on Py 3.13 in #1315

CI

• Add a workflow that runs the TIOBE quality checks in #1301
• Allow executing the TIOBE workflow manually in #1321
• Make Pyright report unnecessary type ignore comments in #1333
• Enable linting of docs/custom_conf.py in #1330

New Contributors

Thanks @james-garner-canonical for your first contributions, and welcome to the team!

Full Changelog: 2.15.0...2.16.0

2.15.0 Support Pebble check-failed and check-recovered events

What's Changed

This release adds support for new events based on Pebble checks. As of Juju 3.6b2, when a Pebble check reaches the failure threshold, a PebbleCheckFailedEvent will be emitted - and when the check starts passing again, the charm will get a PebbleCheckRecoveredEvent. Kubernetes charms can observe these events to react to failing checks - for example, change the unit or application status, output additional logging, or dynamically adjust the workload to work around the failure.

• Add support for Pebble check-failed and check-recovered events in #1281

Fixes

• Pass secret data to Juju via files, rather than as command-line values in #1290 fixing CVE-2024-41129
• Include checks and log targets when merging layers in ops.testing by @amandahla in #1268

Documentation

• Clarify distinction between maintenance and waiting status in #1148

CI

• Bump the Go version to match Pebble in #1285
• Run ruff format over charm pin update code in #1278
• Bump certifi from 2024.2.2 to 2024.7.4 in /docs in #1282
• Update charm pins in #1269

New Contributors

• @amandahla made their first code contribution in #1268

Full Changelog: 2.14.1...2.15.0

2.14.1 Fix a possible Pebble exec hang and minor other fixes

What's Changed

No new features with this release, but it includes a significant fix for Pebble exec to avoid hanging when Pebble is unable to respond quickly.

We've also corrected the Harness behaviour when working with secrets when the secret does not exist or the charm does not have permission to view/modify it, and also expanded the API reference documentation for secrets.

Fixes

• Add connect timeout for exec websockets to avoid hanging in #1247
• Adjust Harness secret behaviour to align with Juju in #1248

Tests

• Fix TypeError when running test.pebble_cli in #1245
• Properly clean up after running setup_root_logging in test_log in #1259
• Verify that defer() is not usable on stop,remove,secret-expired,secret-rotate in #1233

Documentation

• Fix HACKING.md link on PyPI, and internal links in #1261 and #1236
• Add a section to HACKING.md on PR titles (commit messages to main) in #1252
• Add release step to update pinned charm tests in #1213
• Add a security policy in #1266
• Add ops.main to API reference in #1273

CI

• Only run tests once on push to PR in #1242
• Validate PR title against conventional commit rules in #1262
• Only update ops, not all dependencies, in charm tests in #1275
• Add artefact attestation in #1267

Thanks go to the members of the Canonical security teams who helped out putting the security policy together!

Full Changelog: 2.14.0...2.14.1

2.14.0 Fix RelationDataContent.update, add ActionFailed.__str__

This release fixes the RelationDataContent.update method to follow dict.update semantics, that is to allow another dict, an iterable, keyword arguments or a mixture thereof.

Features

• feat: add a __str__ to ActionFailed, for better unexpected failure output in #1209

Fixes

• The other argument to RelatationDataContent.update(...) should be optional by @addyess in #1226

Documentation

• Use the actual emoji character rather than GitHub markup, to show properly on PyPI in #1221
• Clarify that SecretNotFound may be raised for permission errors in #1231

Refactoring

• Refactor tests to pytest style in #1199 #1200 #1203 #1206
• Use ruff formatter and reformat all code in #1224
• Don't use f-strings in logging calls in #1227 #1234

New Contributors

• @dimaqq made their first contribution in #1217. Thanks @dimaqq for your first contributions, and welcome to the team!
• @addyess made their first contribution in #1226

Full Changelog: 2.13.0...2.14.0

2.13.0 Testing User Secrets with Harness and Other Improvements

The main feature this release is the ability to work with user secrets (secrets that a Juju admin adds to the model) in Harness. You'll want to start with the new add_user_secret() method and from there can mostly manage them in the same way as application secrets.

Note that one of the fixes corrects the type of config values, which were previously strongly typed as str, when they may be int, float, bool, or str. This may break your existing type checks, although we have endeavoured to reach out preemptively with PRs to address this where possible.

Features

• Add support for user secrets in Harness in #1176
• Add pebble.CheckInfo.change_id field in #1197

Fixes

• Correct the model config types in #1183
• In Harness, only inspect the source file if it will be used - this fixes using Harness in a Python REPL in #1181

Documentation

• Update publishing a release in HACKING.md in #1173
• Add tox -e docs-deps to compile requirements.txt in #1172
• Update doc to note deprecated functionality in #1178

Tests

• First stage of converting tests from unittest to pytest in #1191, #1192, #1196, #1193, and #1195

Full Changelog: 2.12.0...2.13.0

2.12.0 Support getting cloud specs via the credential-get hook tool, and other improvements.

This release adds a new model method, get_cloud_spec, that returns information about the cloud where the charm is deployed. This may also include credentials that can be used by the charm to interact directly with the cloud, where Juju does not yet provide sufficient modelling. If your charm is currently using the credential-get tool directly, you can now make use of this native support in ops. Note that this functionality is only available on machine charms.

Features

• Added Model.get_cloud_spec which uses the credential-get hook tool to get details of the cloud where the model is deployed #1152

Fixes

• Add a consistency check and default network to add_relation by @PietroPasotti in #1138
• Warn when an observer weakref is lost by @PietroPasotti in #1142
• Update Pebble Notices get_notices parameter name to users=all (previously select=all) #1146
• More robust validation of observer signatures #1147
• Fix attaching storage in Harness before begin #1150
• Change Model.relation.app type from Application|None to Application #1151
• Fixed an issue where pebble.Client.exec might leak a socket.timeout (builtins.TimeoutError) exception #1155
• Don't special-case get_relation behaviour in leader-elected #1156
• Accept type: secret for config options by @jameinel in #1167

Refactoring

• Refactor main.py, creating a new _Manager class by @PietroPasotti in #1085

Documentation

• Use "integrate with" rather than "relate to" #1145
• Updated code examples in the docstring of ops.testing from unittest to pytest style #1157
• Add peer relation details in Harness.add_relation docstring #1168
• Update Read the Docs Sphinx Furo theme to use Canonical's latest styling #1163, #1164, #1165

New Contributors

Thanks @IronCore864 for your first contributions, and welcome to the team!

Full Changelog: 2.11.0...2.12.0

2.11.0 Marking additional events as non-defer()able, and other minor improvements

This release marks additional events as not suitable for deferring: StopEvent, RemoveEvent, and the LifecycleEvents: CollectStatusEvent, CommitEvent, and PreCommitEvent. The defer() method on these event classes is marked as NoReturn and calling it will raise a RuntimeError.

Features

• feat: add the action's ID to the ActionEvent object in #1124
• feat: add the ability to compare two Plan objects with == and create a Plan from a dict in #1134
• feat: only have a defer() method on events that can be deferred in #1122

Fixes

• fix(testing): set JUJU_REMOTE_APP and allow fetching relation data in relation-broken in #1130

Documentation

• docs: tweak the wording around can_connect() usage in #1123

Tooling

• chore: move to ruff for linting in #1120
• chore: adjust isort to be more compatible with ruff in #1139
• chore: expand ruff rule sets and make corresponding minor adjustments in #1114

Full Changelog: 2.10.0...2.11.0

2.10.0 Pebble custom notices, adjustments to relation-broken, and more

This release adds support for Pebble custom notices, including in the Harness testing framework, and removes the broken relation from the model's relations list when in the relation-broken event. There's also the usual collection of documentation improvements, bug fixes, and other minor improvements.

Pebble Notices

Pebble now includes a subsystem called Notices, each of which has a type and a key (currently the only type is "custom"). Custom notices allow the workload to wake up the charm when something interesting happens with the workload, for example, when a PostgreSQL backup process finishes, or some kind of alert occurs.

Workloads can run pebble notify to record an occurrence of a custom notice, providing a unique key and optional data, and Juju will trigger a PebbleCustomNotice event on the charm for it to take appropriate action.

Support for Pebble Notices is available from Juju 3.4.0 onwards. Read the docs about how to use custom notices from the workload container.

Relation-Broken

When a charm is handling a relation-broken event, the relation is on the verge of being removed. A common pattern in charms is to iterate through Model.relations (to generate a configuration file, for example), and the broken relation should not be included with the active relations. To simplify charm code, Model.relations now excludes the broken relation. The relation is still accessible via the event's .relation attribute, and all Relation objects now have an .active attribute to distinguish between the broken relation (active == False) and other relations (active == True).

The plan is that this change will also appear when using the Juju hook tools directly in a future version of Juju.

This does not change accessing the relation data during relation-broken, and this does not change any behaviour in relation-departed.

Features

• feat: support for Pebble Notices in #1086 and in #1100
• feat: add Relation.active, exclude inactive relations from Model.relations in #1091
• feat: when handling actions, print uncaught exceptions to stderr in #1087
• feat: model error on testing module if invalid status set by charm by @yanksyoon in #1107
• feat: add support for v2 fields in CharmMeta in #1106

Fixes

• fix: add pebble log targets and checks to testing plan by @PietroPasotti in #1111
• fix(charm): make collect-status a LifeCycleEvent to avoid logging it by @PietroPasotti in #1080

Documentation

• docs: update README.md in light of changes to charmcraft init output by @tmihoc in #1089
• docs: document limitations with pushing locked or bind-mount files in #1094
• docs: add instructions on how to use a custom version of ops in a Charm in #1092

Tooling

• build: migrate to pyproject.toml in #1068
• ci: use a trusted publisher token for publishing to PyPI in #1061
• fix(typing): update to latest version of Pyright and fix errors in #1105

Many thanks to @PietroPasotti, @yanksyoon, and @tmihoc for their contributions to this release!

Full Changelog: 2.9.0...2.10.0