[feature] Keep Alive for goQuery: log progress during query execution

[els0r]

This commit provides a simple (!) keep-alive mechanism, which can be used to inform the caller of goQuery that a query is still running.

The --query.keepalive or -k shorthand specifies an interval after which the caller gets a query processing update. By default, it is 0, which equates no keep alive.

Example Output

ts=2024-05-23T14:39:01.715Z level=info msg="processed work dir" version=devel worker.id=7 iface=eth0 dir=.db/eth0/2024/03/1710460800 stats.bytes_loaded=0 stats.bytes_decompressed=3848214 stats.blocks_processed=17860 stats.blocks_corrupted=0 stats.directories_processed=31 stats.workloads=1
ts=2024-05-23T14:39:01.931Z level=info msg="processed workload" version=devel iface=eth0 dir=.db/eth0 stats.bytes_loaded=0 stats.bytes_decompressed=7357134 stats.blocks_processed=68980 stats.blocks_corrupted=0 stats.directories_processed=60 stats.workloads=2
ts=2024-05-23T14:39:01.933Z level=info msg="processed workload" version=devel iface=eth0 dir=.db/eth0 stats.bytes_loaded=0 stats.bytes_decompressed=10069197 stats.blocks_processed=105844 stats.blocks_corrupted=0 stats.directories_processed=92 stats.workloads=3
ts=2024-05-23T14:39:01.933Z level=info msg="processed workload" version=devel iface=eth0 dir=.db/eth0 stats.bytes_loaded=0 stats.bytes_decompressed=12768653 stats.blocks_processed=142716 stats.blocks_corrupted=0 stats.directories_processed=124 stats.workloads=4
ts=2024-05-23T14:39:01.968Z level=info msg="processed workload" version=devel iface=eth0 dir=.db/eth0 stats.bytes_loaded=0 stats.bytes_decompressed=16810721 stats.blocks_processed=179588 stats.blocks_corrupted=0 stats.directories_processed=156 stats.workloads=5
ts=2024-05-23T14:39:01.969Z level=info msg="processed workload" version=devel iface=eth0 dir=.db/eth0 stats.bytes_loaded=0 stats.bytes_decompressed=19428092 stats.blocks_processed=216456 stats.blocks_corrupted=0 stats.directories_processed=188 stats.workloads=6
ts=2024-05-23T14:39:01.970Z level=info msg="processed workload" version=devel iface=eth0 dir=.db/eth0 stats.bytes_loaded=0 stats.bytes_decompressed=22129050 stats.blocks_processed=253324 stats.blocks_corrupted=0 stats.directories_processed=220 stats.workloads=7

                           packets   packets            bytes     bytes       
         sip         dip        in       out      %        in       out      %
  200.13.1.2  200.24.1.2   41.38 M   49.49 M  25.26   5.09 GB   6.09 GB  22.47
                               ...       ...              ...       ...       
                          191.16 M  168.53 M         28.11 GB  21.62 GB       
                                                                       
     Totals:                        359.69 M                   49.73 GB  

Timespan    : [2023-09-06 23:58:54, 2024-04-13 20:35:00] (219d20h36m0s)
Interface   : eth0
Sorted by   : accumulated data volume (sent and received)
Query stats : displayed top 1 hits out of 506 in 7.448s

Other Changes

The stats label in the log output stems from a WorkloadStats struct which tracks interactions with the DB engine. Mainly how much data is loaded and processed.

It is meant to be extended in future iterations.

[fako1024]

Ideally I'd probably spend more time on this not-so-trivial change, but since time is pressing: Some nitpicks, aside from that g2g...