Skip to content

v0.3.6

Compare
Choose a tag to compare
@brentp brentp released this 02 Feb 13:13
· 8 commits to master since this release

0.3.6

  • output gargs version in help.
  • restore --ordered (-o) to keep order of output same as input.
    this will cache 3*proccesses output waiting for the slowest job to finish.
    This means that if the user requested 10 processes (-p 10) then there could
    be up to 30 finished jobs waiting for a slow job to finish. If these are input
    memory, they are guaranteed to take <= 1MB (+ go's overhead). If they are larger
    than 1MB, then their data will be on disk.
    This is implemented carefully such that the performance penalty will be small
    unless there are few extremely long-running process outliers.
  • set $PROCESS_I environment variable for each line (or batch of lines).
  • read GARGS_PROCESS_BUFFER to let user set size of data before a tempfile is used.
  • read GARGS_WAIT_MULTIPLIER to determine how many finished processes will wait for single slow processes
    higher values improve concurrency at the expense of memory.
  • better cleanup of tmp files in case of process halt.