Support for scheduled transactions aka "tumbler" functionality

[dergigi]

Not sure yet what the best way to implement this is, but we probably want to offer an interface that:

• Creates a schedule in a (semi-)automated and user-friendly way
• Passes said schedule to the API (this endpoint does not exist yet afaik)
• Notifies the user about the process, clearly showing what is going on under the hood (refer to the output shown in Chris Belcher's comment)

We obviously want to show the progress in a visually appealing way, not log files.

Documentation & Further Resources

• scripts/tumbler.py
• JM Tumbler Guide by waxwing
• Running the Tumbler by openoms
• Community Call 12

[chris-belcher]

I suggest that there be a API call where the web-ui can request default configuration values from the JoinMarket backend.

These default values for tumbler here have been chosen with great care. So the user could be shown a configuration display with the default values already filled, and 95% of users won't change those defaults, although some users might change them (for example users who really don't mind waiting might choose to increase timelambda from its default value. Currently tumbler waits an average of 60 minutes between coinjoins, this user who doesn't mind waiting could increase it to 120 minutes or 180 minutes).

[dergigi]

These default values for tumbler here have been chosen with great care.

That's a good point, and thank you for the input in general @chris-belcher 🙏

---

The way I see it - and please correct me if I'm wrong - is that we will need to extend the API so it allows us to interact with the tumbler.

For a minimal implementation, we will probably need at least two new endpoints:

1. POST /wallet/{walletname}/tumbler/start to start the tumbler (parameters: destination addresses, [optional: tumbler options])
2. GET /wallet/{walletname}/tumbler/log to retrieve the TUMBLE.log so we can give the user feedback and show progress

This way we could at least start the tumbler with the default options and let the user know if any error occurred or if it succeeded as expected.

If we we want to provide the extended settings in the Web UI, similar to the options in the QT interface, we would also need the following:

3. GET /wallet/{walletname}/tumbler/config Retrieve tumbler config (to pre-fill UI, see above; values will be passed to (1), so the tumbler will start with the updated config, i.e. the "tumbler options" parameter that is optional)

Re (3): Would it make sense to move the tumbler default values to the config file? This way we could retrieve (and update) them via /configget and /configset, which already exists. (If we want to reduce the number of API calls, we should obviously have an endpoint that returns the config in one go. Could also be the same endpoint; do a GET with no parameter to get it all.)

Re (1): We could also think about using a separate tumble schedule file like JAM.schedule or something, since the --schedulefile option exists.

---

Please let me know what you think, and don't be afraid to hold back if I misunderstood things or if my proposals are stupid. Happy to implement the required changes myself @AdamISZ - my Python is a bit rusty, but at least some of the parts should be straightforward to implement.

P.S: I used "tumbler" as the API endpoint because it is used everywhere in JM, but I would be in favor of calling this thing "scheduler", since this is closer to what it does. But that's probably a debate for another day 😅

[AdamISZ]

Yes this looks generally like the right direction to me, too.

We could refactor the existing endpoint for coinjoin to accept a schedule instead of parameters for 1 schedule entry, but I agree it's more sensible to make a new endpoint with a schedule passed as json with lists.
And yes a tumbler/config endpoint makes sense too - given options and destinations, it'll generate the tumbler schedule just as is done in CLI and Qt, and that can get passed to the aforementioned endpoint.

See https://github.com/JoinMarket-Org/joinmarket-clientserver/blob/e915daef8819b812e0946757b2e366407aa3885a/jmclient/jmclient/schedule.py#L86-L96

(edit: oh, the start of that docstring is out of date ... anyway, that's the function we'd call)

[theborakompanioni]

This way we could at least start the tumbler with the default options and let the user know if any error occurred or if it succeeded as expected.

Hey @dergigi , this looks like a very balanced solution. Consistent with the previous design decisions and on target for what has to be achieved by the UI.

Re (1): We could also think about using a separate tumble schedule file like JAM.schedule or something, since the --schedulefile option exists.

👍

Please let me know what you think, and don't be afraid to hold back if I misunderstood things or if my proposals are stupid.

The only thing that came to mind is some sort of /stop endpoint similar to the one for the maker. At least some scenarios locally showed it would be beneficial, e.g. when the orderbook is empty (a somewhat unlikely case on mainnet, but you get the point).

Yes this looks generally like the right direction to me, too.

💪

We could refactor the existing endpoint for coinjoin to accept a schedule instead of parameters for 1 schedule entry, but I agree it's more sensible to make a new endpoint with a schedule passed as json with lists. And yes a tumbler/config endpoint makes sense too - given options and destinations, it'll generate the tumbler schedule just as is done in CLI and Qt, and that can get passed to the aforementioned endpoint.

@dergigi's proposal is similar, but a bit different as to pass the tumbler params like you would in the CLI instead of passing the schedule. I don't have that much insight and so my opinion shouldn't be given that much importance, but would this provide more flexibility, e.g when it comes to restarting the service (`--restart')?

Please disregard this if it can be solved in a better way, for example if a schedule with already completed steps can also be given.

It would be very nice to provide the schedule of course, but is all necessary info included in the schedule file? If not, is a mixture of both reasonable? Options + optional schedule?

All in all, that would be a great thing to have and a big move forward.

Edit: If a schedule can be passed, would it be reasonable that there is also an endpoint to retrieve the "current" schedule?

[dergigi]

Quick update: @dnlggr managed to implement a prototypical endpoint that is already functional. We'll do our best to write up our thoughts properly today and we'll create an issue in the clientserver repo soon.

[dergigi]

Issue created: JoinMarket-Org/joinmarket-clientserver#1239 - appreciate any and all feedback 🙏

[theborakompanioni]

Issue created: JoinMarket-Org/joinmarket-clientserver#1239 - appreciate any and all feedback pray

As the first PR is merged, and the second PR is ready, we know which data is available to Jam in the next server release.

Currently in Jam, a running taker is indicated only in the navbar. This is "fine" for single collaborative transactions, but when scheduled transactions are running (which can take multiple hours), a user wants to roughly know when the operation is expected to finish.

Following data is available for each scheduled transactions:

• mixdepth
• amount-fraction (integer value in satoshis to be spent, If 0, a sweep is performed.)
• N-counterparties
• destination address (special value: "INTERNAL")
• wait time in minutes (after confirmation of this transaction, before continuing to the next.)
• rounding
• flag indicating incomplete/broadcast/completed

For more information see schedules-transaction-lists.

e.g. this is an example schedule before it is started

1, 110000000, 3, INTERNAL, 0.8, 16, 0
0, 20000011, 2, mnsquzxrHXpFsZeL42qwbKdCP2y1esN3qw, 0, 16, 0

Obviously, summing up all wait-times will give the minimum duration the operation will take.

@editwentyone already showed some designs for "Cooking Jars" in the Call. Do you think something like this combined with a timer is suitable?

Example:

Screen.Recording.2019-01-21.mov

[editwentyone]

this is what I got so far, do I miss something? feedback always welcome

https://www.figma.com/file/kfejZJFlwBywvLEnPEmJo1/?node-id=4228%3A79798

[AdamISZ]

One thing I would say is, de-emphasize "time left" aspect and emphasize progress in terms of how many of the scheduled transactions have gone through. There are a lot of big variables affecting how long it can take, even things that are not obvious like: if you have to use a commitment from a utxo that was freshly created, in which case your bot will wait 20 minutes before trying again and then will only continue after 5 confs.

I know that in practice we're a good way away from being able to create a smooth UX (well, sometimes it will be smooth but that's not the point), but if we're thinking about concepts here: bear in mind that tumbler was designed to create a way to flow funds out of the Joinmarket wallet. It addresses timing and amount correlation issues and ends with flows out to multiple (3 is just default, more is better) destinations. So that sense of "flow out" might be a bit different from other encapsulations of coinjoin. I think that will be relevant to design/UI.

[theborakompanioni]

One thing I would say is, de-emphasize "time left" aspect and emphasize progress in terms of how many of the scheduled transactions have gone through.

Related image posted in the group:

[ghost]

Most of this is resolved through #242. Any additional improvements, redesigns, etc. are best tracked as new issues.

[editwentyone]

Figma