Add ability to break parking lots, stop locks from stalling #3081

jakkdl · 2024-09-06T10:38:26Z

fixes #3035

TODO:

newsfragment(s)
docstrings
docs
tests of directly using the parking lot functions
lock test that checks that handing over the owner works (this is indirectly tested by a lot of other tests, but good to have a standalone test)
reraise the exception in Lock/_LockImpl to get a better error message

codecov · 2024-09-06T10:43:29Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.58%. Comparing base (d0158fa) to head (92f9799).
Report is 24 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff            @@
##             main    #3081    +/-   ##
========================================
  Coverage   99.58%   99.58%            
========================================
  Files         121      121            
  Lines       18002    18166   +164     
  Branches     3248     3275    +27     
========================================
+ Hits        17927    18091   +164     
  Misses         52       52            
  Partials       23       23

Files with missing lines	Coverage Δ
src/trio/_core/__init__.py	`100.00% <100.00%> (ø)`
src/trio/_core/_parking_lot.py	`100.00% <100.00%> (ø)`
src/trio/_core/_run.py	`99.02% <100.00%> (+<0.01%)`	⬆️
src/trio/_core/_tests/test_parking_lot.py	`100.00% <100.00%> (ø)`
src/trio/_sync.py	`100.00% <100.00%> (ø)`
src/trio/_tests/test_sync.py	`100.00% <100.00%> (ø)`
src/trio/lowlevel.py	`100.00% <ø> (ø)`

src/trio/_core/_parking_lot.py

A5rocks · 2024-09-06T17:10:54Z

@njsmith cause this is interface design, do you think there's a better way to express all of this?

A5rocks · 2024-09-07T08:24:00Z

src/trio/_core/_parking_lot.py

+        contains a reference to the task sent as a parameter."""
+        self.broken_by = task
+
+        # TODO: is there any reason to use self._pop_several?


Probably some thin veneer of thread safety? Or guaranteeing that we'll remove every task we raise an error in, even if somehow _core.reschedule raises an error (... well, we'll remove the task where this happens too. Oh well, still better behavior).

Neither is really something we care about but probably worth using cause it's a simple drop in replacement anyways.

unpark() above would have the same theoretical issue, if reschedule() failed it could drop tasks too. Probably not something to worry about, if that fails we've got bigger problems.

src/trio/_core/_parking_lot.py

jakkdl · 2024-09-10T11:35:50Z

@njsmith cause this is interface design, do you think there's a better way to express all of this?

as we don't actually have anybody requesting the functionality on ParkingLot or add_parking_lot_breaker/remove_parking_lot_breaker, we could opt not to make these public for now and document them as an experimental API. Or I suppose make them public but with a warning text in the docstring that they're experimental.

…as it in acquire, update docstrings. docs are failing to build locally but idk wth is wrong

newsfragments/3081.feature.rst

Zac-HD

This looks good to me! I'd appreciate a review by another experienced maintainer but I'd be inclined to merge once minor comments are addressed 😁

src/trio/_sync.py

src/trio/_core/_parking_lot.py

newsfragments/3081.feature.rst

src/trio/_core/_tests/test_parking_lot.py

A5rocks · 2024-09-18T07:31:22Z

src/trio/_core/_parking_lot.py

+        """
+        if task is None:
+            task = _core.current_task()
+        self.broken_by = task


I'm just now realizing that this relationship is a bit flawed. Lots can only be marked as broken by one task, but multiple tasks could be marked as a parking lot breaker and exit (or a task multiple times, but that's handled already). I'm not sure what's a good idea. We can't raise any error because there's no good place to raise errors, but I suppose we could warn? Or maybe broken_by should be a list.

Having a list feels overkill, and only really being useful in the case of multiple independent coding errors all leading to them wanting to break a lot. It feels like what would more commonly happen is multiple instances of the same task breaking a lot for the same reason, in which case the list would just fill up with tons of duplicates. Or cases where once one task has broken a lot it causes multiple subsequent tasks to re-break it.
So I feel like the first breaker is ""special"" in most cases, and we'd rarely care about the others.
But raising a warning sounds good, maybe RuntimeWarning?

should warning be raised if the same task breaks a lot multiple times?

Yeah that sounds like a fine warning and a task breaking a lot multiple times is probably fine. After all we allow nesting add/remove_parking_lot_breaker

Zac-HD

Two minor things below, and then unless @njsmith or @oremanj raise any concerns let's merge after the weekend!

src/trio/_core/_run.py

src/trio/_sync.py

A5rocks

Mostly just nitpicks

src/trio/_core/_parking_lot.py

src/trio/_core/_run.py

src/trio/_core/_tests/test_parking_lot.py

src/trio/_sync.py

src/trio/_tests/test_sync.py

A5rocks

A couple nitpicks but this looks good otherwise!

(I'm not going to lie add_parking_lot_breaker was not the best name I've ever come up with. At least it's descriptive...)

src/trio/_core/_parking_lot.py

src/trio/_core/_run.py

…docstring. fix runtimewarning transforming into triointernalerror. add a bunch of tests

jakkdl

turns out there were some nasty things still buried >.>

jakkdl · 2024-10-02T10:43:18Z

src/trio/_core/_tests/test_parking_lot.py

+    child_task = None
+    with pytest.warns(RuntimeWarning):
+        async with trio.open_nursery() as nursery:
+            child_task = await nursery.start(dummy_task)
+            # registering a task as breaker on an already broken lot is fine... though it
+            # maybe shouldn't be as it will always cause a RuntimeWarning???
+            # Or is this a sign that we shouldn't raise a warning?
+            add_parking_lot_breaker(child_task, lot)
+            nursery.cancel_scope.cancel()


I think it's fine, the main issue I think is just warning in trio internals being misleading.

I concur that this seems fine; we do want to give the warning but it's not otherwise a problem - in typical usage you'd get an error about the lot being broken when you go to park in it, and that will likely still happen here (if it hasn't already!).

jakkdl · 2024-10-02T10:45:46Z

src/trio/_core/_run.py

+    except RuntimeWarning:
+        raise
    except TrioInternalError:
        raise


This could be too broad (maybe warnings should cause internal errors and prompt people to open issues) or too narrow (if passing through runtimewarnings, why not all warnings?). Could make the RuntimeWarning a custom warning class instead

I'd avoid special-casing this at all; if users set -Werror they're signing up to deal with this kind of thing or changing their code to avoid it.

Hmm, I'm torn on this - in large part because TrioInternalError prompts users to open an issue to report it, and running pytest with -Werror isn't that obscure

Oh, I see! Yeah, keeping it with a custom warning type seems good then.

To be honest custom warning type is unnecessary if above lot.break_lot(task) we put a lot.broken_by check. I'm not sure that's indicative of quality of idea though.

Requiring that check (and erroring on attempts to break an already-broken lot) seems pretty sensible to me, it's not like ParkingLot is an interface that gets called directly very often, and so I don't mind adding some ceremony if that helps catch tricky concurrency bugs.

okay so with the check we no longer raise any warnings on task exit if multiple tasks try to break a task, which means we only ever give a warning if manually breaking a broken lot. That doesn't seem particularly logical to me, and we should probably do one of:

make breaking a broken lot a no-op

raise an error

...go with broken_by being a list (or tuple).

The warning was added after this convo: https://github.com/python-trio/trio/pull/3081/files/7a1ce5b755b5775153a3525bea793ab7da05fb44#r1764544347 where I argued against the list, but if we no longer want to raise a warning from the internals then maybe the list is the way to go? (that also allows for duplicates in the list).
My gut feeling would be 3>1>2, but idk.

I agree with that ordering (though I might slightly prefer 2 over 1? doesn't matter.)

Zac-HD

Only action items left AFAICT are to remove the except RuntimeWarning block, and optionally to clean up any comments that you want to adjust. Then let's merge 🚀

Also, thanks again @jakkdl - this has been a rather long project to fix a relatively small issue, but I think we've ended up with a really nice solution and really appreciate all your work on this 😍

Zac-HD · 2024-10-02T19:04:54Z

src/trio/_core/_run.py

+    except RuntimeWarning:
+        raise
    except TrioInternalError:
        raise


I'd avoid special-casing this at all; if users set -Werror they're signing up to deal with this kind of thing or changing their code to avoid it.

src/trio/_core/_run.py

Zac-HD · 2024-10-03T02:02:02Z

src/trio/_core/_tests/test_parking_lot.py

+    child_task = None
+    with pytest.warns(RuntimeWarning):
+        async with trio.open_nursery() as nursery:
+            child_task = await nursery.start(dummy_task)
+            # registering a task as breaker on an already broken lot is fine... though it
+            # maybe shouldn't be as it will always cause a RuntimeWarning???
+            # Or is this a sign that we shouldn't raise a warning?
+            add_parking_lot_breaker(child_task, lot)
+            nursery.cancel_scope.cancel()


I concur that this seems fine; we do want to give the warning but it's not otherwise a problem - in typical usage you'd get an error about the lot being broken when you go to park in it, and that will likely still happen here (if it hasn't already!).

src/trio/_core/_tests/test_parking_lot.py

Zac-HD

I like the state we've gotten to, and want to use the feature sometime this year, so I propose merging as-is 🙂

A5rocks · 2024-10-07T18:33:13Z

I like the state we've gotten to, and want to use the feature sometime this year, so I propose merging as-is 🙂

We can fix broken_by's representation then merge, everything requires a release anyways :P

jakkdl · 2024-10-08T14:22:04Z

Made broken_by a list, and cleaned up test_parking_lot that had gotten incredibly messy across all the iterations. I think it's ready to merge at this point after a quick review I didn't do anything dumb at the last second

A5rocks

Looks good to me based on a quick review

initial implementation of lot breaking

3bd67a5

jakkdl mentioned this pull request Sep 6, 2024

Problems with trio.Lock() across multiple tasks #3035

Closed

fix import cycle

4dfa1ad

jakkdl added 2 commits September 6, 2024 12:47

fix re-export for verifytypes visibility

543a087

update docstrings

c36cdad

TeamSpen210 reviewed Sep 6, 2024

View reviewed changes

src/trio/_core/_parking_lot.py Outdated Show resolved Hide resolved

TeamSpen210 reviewed Sep 6, 2024

View reviewed changes

src/trio/_core/_parking_lot.py Outdated Show resolved Hide resolved

jakkdl added 5 commits September 6, 2024 13:15

fixes after review by TeamSpen210

127c5fc

add tests

1f75d44

add lock handover test

6835e87

clean up breaker dict

3b86e80

clean up GLOBAL_PARKING_LOT_BREAKER when task releases or exits

94ff9a2

A5rocks reviewed Sep 6, 2024

View reviewed changes

src/trio/_core/_parking_lot.py Outdated Show resolved Hide resolved

src/trio/_core/_parking_lot.py Outdated Show resolved Hide resolved

src/trio/_core/_parking_lot.py Outdated Show resolved Hide resolved

A5rocks reviewed Sep 7, 2024

View reviewed changes

jakkdl commented Sep 10, 2024

View reviewed changes

src/trio/_core/_parking_lot.py Outdated Show resolved Hide resolved

jakkdl added 2 commits September 10, 2024 15:10

add newsfragments, add StalledLockError, reraise BrokenResourceError …

eb7a451

…as it in acquire, update docstrings. docs are failing to build locally but idk wth is wrong

add test for default argument of break_lot

e7d7205

jakkdl marked this pull request as ready for review September 10, 2024 13:14

Merge branch 'main' into break_the_lot

c89fb2a

jakkdl commented Sep 10, 2024

View reviewed changes

newsfragments/3081.feature.rst Outdated Show resolved Hide resolved

jakkdl requested review from njsmith and Zac-HD September 17, 2024 09:35

jakkdl mentioned this pull request Sep 17, 2024

raise BrokenResourceError if lock owner has exited. #3080

Closed

Zac-HD reviewed Sep 17, 2024

View reviewed changes

A5rocks reviewed Sep 18, 2024

View reviewed changes

jakkdl mentioned this pull request Sep 18, 2024

During docs builds, leave newsfragments unchanged #3086

Merged

various fixes after review

277c7da

Merge remote-tracking branch 'origin/main' into break_the_lot

21cf0d6

jakkdl requested review from Zac-HD and A5rocks September 18, 2024 10:36

Zac-HD approved these changes Sep 19, 2024

View reviewed changes

src/trio/_core/_run.py Outdated Show resolved Hide resolved

src/trio/_sync.py Outdated Show resolved Hide resolved

break lots before other checks, minor phrasing improvement in docstring

45f78f4

A5rocks reviewed Sep 19, 2024

View reviewed changes

jakkdl and others added 2 commits September 27, 2024 12:57

docstring updates after A5rocks review

ec48863

Merge branch 'main' into break_the_lot

c742a52

jakkdl requested a review from A5rocks September 27, 2024 11:00

A5rocks approved these changes Oct 2, 2024

View reviewed changes

src/trio/_core/_parking_lot.py Outdated Show resolved Hide resolved

src/trio/_core/_run.py Show resolved Hide resolved

raise brokenresourceerror if registering an already exited task. fix …

7a1ce5b

…docstring. fix runtimewarning transforming into triointernalerror. add a bunch of tests

jakkdl commented Oct 2, 2024

View reviewed changes

Zac-HD approved these changes Oct 3, 2024

View reviewed changes

jakkdl added 2 commits October 7, 2024 14:52

remove warning on task exit

cc97cca

Merge remote-tracking branch 'origin/main' into break_the_lot

b81e297

Zac-HD approved these changes Oct 7, 2024

View reviewed changes

jakkdl and others added 3 commits October 8, 2024 12:37

make broken_by attribute a list, clean up tests

1d7ece3

Merge branch 'main' into break_the_lot

b826210

fix test. polish comments and tests

92f9799

jakkdl requested review from A5rocks October 8, 2024 14:22

A5rocks approved these changes Oct 8, 2024

View reviewed changes

Zac-HD merged commit 2a66a0d into python-trio:main Oct 9, 2024
38 checks passed

jakkdl deleted the break_the_lot branch October 9, 2024 09:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ability to break parking lots, stop locks from stalling #3081

Add ability to break parking lots, stop locks from stalling #3081

jakkdl commented Sep 6, 2024 •

edited

Loading

codecov bot commented Sep 6, 2024 •

edited

Loading

A5rocks commented Sep 6, 2024

A5rocks Sep 7, 2024 •

edited

Loading

TeamSpen210 Sep 8, 2024

jakkdl commented Sep 10, 2024

Zac-HD left a comment

A5rocks Sep 18, 2024

jakkdl Sep 18, 2024

jakkdl Sep 18, 2024

A5rocks Sep 18, 2024 •

edited

Loading

Zac-HD left a comment

A5rocks left a comment

A5rocks left a comment

jakkdl left a comment

jakkdl Oct 2, 2024

A5rocks Oct 2, 2024

Zac-HD Oct 3, 2024

jakkdl Oct 2, 2024

Zac-HD Oct 2, 2024

jakkdl Oct 3, 2024

Zac-HD Oct 3, 2024

A5rocks Oct 3, 2024 •

edited

Loading

Zac-HD Oct 3, 2024

jakkdl Oct 7, 2024 •

edited

Loading

A5rocks Oct 7, 2024

Zac-HD left a comment

Zac-HD Oct 2, 2024

Zac-HD Oct 3, 2024

Zac-HD left a comment

A5rocks commented Oct 7, 2024

jakkdl commented Oct 8, 2024

A5rocks left a comment

Add ability to break parking lots, stop locks from stalling #3081

Add ability to break parking lots, stop locks from stalling #3081

Conversation

jakkdl commented Sep 6, 2024 • edited Loading

codecov bot commented Sep 6, 2024 • edited Loading

Codecov Report

A5rocks commented Sep 6, 2024

A5rocks Sep 7, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jakkdl commented Sep 10, 2024

Zac-HD left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

A5rocks Sep 18, 2024 • edited Loading

Choose a reason for hiding this comment

Zac-HD left a comment

Choose a reason for hiding this comment

A5rocks left a comment

Choose a reason for hiding this comment

A5rocks left a comment

Choose a reason for hiding this comment

jakkdl left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

A5rocks Oct 3, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jakkdl Oct 7, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Zac-HD left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Zac-HD left a comment

Choose a reason for hiding this comment

A5rocks commented Oct 7, 2024

jakkdl commented Oct 8, 2024

A5rocks left a comment

Choose a reason for hiding this comment

jakkdl commented Sep 6, 2024 •

edited

Loading

codecov bot commented Sep 6, 2024 •

edited

Loading

A5rocks Sep 7, 2024 •

edited

Loading

A5rocks Sep 18, 2024 •

edited

Loading

A5rocks Oct 3, 2024 •

edited

Loading

jakkdl Oct 7, 2024 •

edited

Loading