Skip to content

Advanced usage

Jump to bottom Edit New page
Arne Christian Beer edited this page Nov 28, 2024 · 6 revisions

Aliases

To get basic aliasing, simply put a pueue_aliases.yml besides your pueue.yml. Its contents should look something like this:

ls: 'ls -ahl'
rsync: 'rsync --recursive --partial --perms --progress'

When adding a command to pueue, the first word will then be checked for the alias. This means, that for instance ls ~/ && ls / will result in ls -ahl ~/ && ls /.
If you want multiple aliases in a single task, it's probably best to either create a task for each command or to write a custom script.

Callbacks

You can specify a callback that will be called every time a task finishes. The callback can be parameterized in the pueue.yml file in the daemon section with some variables.

These are the available variables that can be used to create a command:

  • {{ id }}
  • {{ command }}
  • {{ path }}
  • {{ result }} (Success, Killed, etc.)
  • {{ exit_code }} The shell exit code (0-255, can be None for killed/unspawned processes)
  • {{ group }}
  • {{ output }} The last few lines of process output. 10 by default, configurable in pueue.yml
  • {{ start }} (times are UNIX timestamps or empty)
  • {{ end }}

Example callback:

callback: "notify-send \"Task {{ id }}\nCommand: {{ command }}\nPath: {{ path }}\nFinished with status '{{ result }}'\nTook: $(bc <<< \"{{end}} - {{start}}\") seconds\""

Here's another example for macOS notifications:

callback: "osascript -e 'display notification \"Finished with status {{ result }}\" with title \"Pueue task {{ id }}\" subtitle \"{{ command }}\"'"

Depending on how complex your callback becomes it might make sense to put into an external file:

callback: |
    notify-send \
    "Task {{ id }}
    Command: {{ command }}
    Path: {{ path }}
    Finished with status '{{ result }}'"

Make sure to restart the daemon after you make changes to pueue.yml.

Shell completion files

Shell completion files can be created on the fly with pueue completions $shell $directory. There's also a build_completions.sh script, which creates all completion files in the utils/completions directory.

Here's an example of where those files should be placed in Arch-Linux:

  • Zsh: /usr/share/zsh/site-functions/_pueue
  • Bash: /usr/share/bash-completion/completions/pueue.bash
  • Fish: /usr/share/fish/completions/pueue.fish

The paths might differ for your Linux distribution, so please double-check those paths.

Scripting

(This section is no longer necessary with version 4.0.0). When calling pueue commands in a script, you might need to sleep for a short amount of time for now. The pueue server processes requests asynchronously, whilst the TaskManager runs it's own update loop with a small sleep. (The TaskManager handles everything related to starting, stopping and communicating with processes.)

JSON Support

The Pueue client status and log commands support JSON output with the -j flag. This can also be used to easily incorporate it into window manager bars, such as i3bar.

Filtering pueue status

It's possible to filter the pueue status output by filtering the json output with external tools and piping it back into pueue format-status

The most simple example would be something like this:

pueue status --json | jq -c '.tasks' | pueue format-status

Summarising pueue status job states

image

This is a script for jq, the command-line JSON processor, to output a short summary of your pueue queues. Save it somewhere (I put mine in ~/.config/pueue/pueue_status.jq), then use:

pueue status -j | jq -r -f <path_to_the_script>

I've made that an alias in my shell (alias pstatus='....'). This script requires jq version 1.6 or up, as it uses the \\?X destructuring alternative operator syntax:

# use jq to turn pueue status -j output into grouped counts of states
# requires jq 1.6 or newer
#
# usage: pueue status -j | jq -r -f <path_for_this_script>

# ANSI color functions
def _ansi_bright($c): tostring | "\u001b[\($c);1m\(.)\u001b[0m";
def red:    _ansi_bright(31);
def green:  _ansi_bright(32);
def yellow: _ansi_bright(33);
def cyan:   _ansi_bright(36);
def white:  _ansi_bright(37);

def styled_status:
    # turn the status structure into a string with ANSI color applied
    # this uses destructuring with the ?// alternatives operator
    # (requires jq 1.6 or newer)
    . as
        {Done: $done}
    ?// {Stashed: {enqueue_at: $enq}}
    ?// $status
    | if $done then
        $done as
            {FailedToSpawn: $serr, Failed: $ferr}
        ?// $result
        | if $ferr then
            "Failed" | red
        elif $serr then
            "Failed to spawn" | red
        elif $result == "DependencyFailed" then
            "Dependency failed" | red
        elif $result == "Success" then
            "Success" | green
        else
            $result | red
        end
    elif $status == "Running" then
        "Running" | green
    elif ($status == "Paused" or $status == "Locked") then
        $status | white
    elif $status then
        $status | yellow
    else  # {Stashed: {enque_at: ...}}, but $enq can be null
        "Stashed" | yellow
    end;

# Main filter
[.tasks[] | {"status": .status | styled_status, "group": .group | cyan}]
| [group_by(.)[] | {"group": .[0].group, "status": .[0].status, "count": length}]
| [
    group_by(.group)[]
    | (map("  \(.status): \(.count)") | join("\n")) as $result
    | "\(.[0].group): \(map(.count) | add) tasks\n\($result)"
]
| join("\n\n")

Interactive fuzzy-finder

pueue

These scripts provide interactivity, fuzzy-finding and live-logging facilities to pueue using jq, fzf and bat. It's assumed that these dependencies are installed, and that a POSIX shell is being used (dash, bash, zsh, etc.).

To install, save the following function in a script sourced by your shell, such as ~/.profile, ~/.bash_profile, ~/.login, etc.:

pf() {
  set -f
  local PUEUE_TASKS="pueue status --json | jq -c '.tasks' | jq -r '.[] | \"\(.id | tostring | (\" \" * (2 - length)) + .) | \(.group) | \(.path[-15:]) | \(.status) | \(.command[-15:]) | \(.start[:19])\"'"
  local header="p:pause | s:start | r:restart | k:kill | l:log | f:reload"

local bind="\
ctrl-p:execute-silent(echo {} | cut -d'|' -f1 | xargs pueue pause > /dev/null)+reload^$PUEUE_TASKS^,\
ctrl-s:execute-silent(echo {} | cut -d'|' -f1 | xargs pueue start > /dev/null)+reload^$PUEUE_TASKS^,\
ctrl-r:execute-silent(echo {} | cut -d'|' -f1 | xargs pueue restart -ik > /dev/null)+reload^$PUEUE_TASKS^,\
ctrl-k:execute-silent(echo {} | cut -d'|' -f1 | xargs pueue kill > /dev/null)+reload^$PUEUE_TASKS^,\
ctrl-l:execute-silent(echo {} | cut -d'|' -f1 | xargs pueue log | less > /dev/tty),\
ctrl-f:reload^$PUEUE_TASKS^\
"

  echo $PUEUE_TASKS | sh | fzf --header "${header}" -m \
    --preview="echo {} | cut -d'|' -f1 | xargs pueue log | bat -l log --style=rule,numbers --color=always -r ':200'" \
    --bind="$bind"
  set +f
}

After restarting your shell, you can then invoke pf (stands for pueue fuzzy) to activate the interactive fuzzy-finder. You'll see a hint about hotkeys. For example, p:pause | s:start | f:reload. Means, press Ctrl + p to pause a task, Ctrl + s to start the task, Ctrl + f to force reload status of all tasks in a view from pueue. When you starting/pausing/restarting/killing a task, the view will be automatically reloaded to show an actual status and logs. You can change fzf appearance, examples can be found in fzf docs.

Sometimes after viewing logs with Ctrl + l, and exiting from less afterwards, terminal can have unexpected characters in it instead of fzf view. Try to increase/decrease terminal window to force it to redraw. After that everything will look correctly again.

Single-line summaries for wget jobs

image

When not in interactive mode, wget uses a verbose dot output format; you can explicitly choose this format to alter how much data a dot represents:

$ wget https://www.example.com/sample_file.zip -q --show-progress --progress=dot:mega
Resolving www.example.com (www.example.com)... 233.252.0.127
Connecting to www.example.com (www.example.com)|233.252.0.127|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 222882919 (213M) [application/zip]

     0K ........ ........ ........ ........ ........ ........  1% 40.4K 88m33s
  3072K ........ ........ ........ ........ ........ ........  2% 39.9K 87m50s
  6144K ........ ........ ........ ........ ........ ........  4% 39.9K 86m45s
  9216K ........ ........ ........ ........ ........ ...

When resuming, the same format uses commas to denote the skipped part:

        [ skipping 215040K ]
215040K ,,,,,,,, ,,,,,,,, ,,,,,,,, ,,,..... ........

This format can be summarised quite nicely, using pueue log -j and the following jq script, taking the last two lines of the output and replacing consecutive dots with elipsis (…) characters, and commas with a sinus character (∿), for a very readable per-job status board:

# for use with pueue log --json
# Usage: pueue log -j | jq -r --arg group [groupname] -f [path to this file]
# where 'groupname' is the pueue task group that outputs wget download logs
# if omitted, defaults to "download"

# ANSI color functions
def _ansi_bright($c): tostring | "\u001b[\($c);1m\(.)\u001b[0m";
def red:    _ansi_bright(31);
def green:  _ansi_bright(32);

[
    .[]
    | select(
        .task.group==($ARGS.group // "download")
        and .task.status!="Queued"
        and .task.status!={"Done": "Success"}
    )
    | (
        if .task.status=="Running"
            then ("\(.task.label)" | green)
            else ("\(.task.label)" | red)
        end
    ) as $task_label
    | (
        .output
        | split("\n")[-2:]
        | join(" ")
        | gsub("\\.{8} "; "\u2026")               # ........ -> U+2026 HORIZONTAL ELLIPSIS
        | gsub(",[.,]{7} "; "\u223F")             # ,,,,.... -> U+223F SINE WAVE
        | sub("(?<e>[\u2026\u223F]+)"; "\(.e) ")  # keep the last space
    ) as $shortened
    | "\(.task.id): \($task_label): \($shortened)"
)
| join("\n")

Logs

All logs can be found in ${pueue_directory}/logs. Logs of previous Pueue sessions will be created whenever you issue a reset or clean. In case the daemon fails or something goes wrong, the daemon will print to stdout/stderr. If the daemon crashes or something goes wrong, please set the debug level to -vvvv and create an issue with the log!

If you want to dig right into it, you can compile and run it yourself with a debug build. This would help me a lot!

Add a custom footer
Add a custom sidebar
Clone this wiki locally