Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature] Goquery: less verbose and frequent keep-alive messages #327

Conversation

els0r
Copy link
Owner

@els0r els0r commented Jun 12, 2024

Closes #326

@els0r els0r linked an issue Jun 12, 2024 that may be closed by this pull request
2 tasks
t.footerWriter.WriteEntry(queryStatsKey, "displayed top %s hits out of %s in %s",
hitsDisplayed,
hitsTotal,
textFormatter.Duration(result.Summary.Timings.QueryDuration),
)
result.Query.PrintFooter(t.footerWriter)

if t.printQueryStats {
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's if you want them. Looks as follows:

                                                       packets   packets           bytes     bytes       
    iface            sip            dip  dport  proto       in       out      %       in       out      %
     eth7  10.236.33.234  10.236.130.23    443    TCP  24.00 M   34.21 M  51.27  2.48 GB  31.53 GB  50.57
                                                           ...       ...             ...       ...       
                                                       47.24 M   66.30 M         5.55 GB  61.68 GB       
                                                                                                  
  Totals:                                                       113.55 M                  67.23 GB  

Timespan              : [2023-09-06 23:58:54, 2024-04-13 20:35:00] (219d20h36m0s)
Interface             : 11 queried
Sorted by             : accumulated data volume (sent and received)
Conditions            : dport = 443
Query stats           : displayed top 1 hits out of 23 in 1m0.517s
Bytes loaded          : 0.00  B
Bytes decompressed    : 100.19 MB
Blocks processed      : 5.11 M
Blocks corrupted      : 0.00  
Directories processed : 2.22 k
Workloads             : 72.00  

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something that keeps bothering me (nitpick, but just because I see it again): We format integers / counters as floats, even if they are zero or on the "base" unit like B or just a number (not only here, but also in the general table). IIRC there is some humanize method somewhere and could be slightly adjusted in a fly-by? 😉

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That bugs me too. I was about to adjust it already. Will see if I can do it non-invasively.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed: even allowed me to remove some code:

Timespan              : [2023-09-06 23:58:54, 2024-04-13 20:35:00] (219d20h36m0s)
Interface             : 11 queried
Sorted by             : accumulated data volume (sent and received)
Conditions            : dport = 443
Query stats           : displayed top 1 hits out of 23 in 9.036s
Bytes loaded          : 0 B
Bytes decompressed    : 100.19 MB
Blocks processed      : 5.11 M
Blocks corrupted      : 0
Directories processed : 2.22 k
Workloads             : 72

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do in separate issue to keep the changelog clean

@els0r els0r marked this pull request as ready for review June 12, 2024 11:51
@els0r els0r requested a review from fako1024 June 12, 2024 11:52
pkg/e2etest/e2e_test.go Show resolved Hide resolved
t.footerWriter.WriteEntry(queryStatsKey, "displayed top %s hits out of %s in %s",
hitsDisplayed,
hitsTotal,
textFormatter.Duration(result.Summary.Timings.QueryDuration),
)
result.Query.PrintFooter(t.footerWriter)

if t.printQueryStats {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something that keeps bothering me (nitpick, but just because I see it again): We format integers / counters as floats, even if they are zero or on the "base" unit like B or just a number (not only here, but also in the general table). IIRC there is some humanize method somewhere and could be slightly adjusted in a fly-by? 😉

pkg/results/TablePrinter.go Show resolved Hide resolved
pkg/types/hashmap/aggregate.go Show resolved Hide resolved
pkg/types/workload/stats.go Show resolved Hide resolved
@els0r els0r changed the title [feature] Goquery: provide stats updates less frequently using flow map aggregates [feature] Goquery: provide stats updates less frequently + more concise number formatting Jun 12, 2024
@els0r els0r changed the title [feature] Goquery: provide stats updates less frequently + more concise number formatting [feature] Goquery: less verbose and frequent keep-alive Jun 12, 2024
@els0r els0r changed the title [feature] Goquery: less verbose and frequent keep-alive [feature] Goquery: less verbose and frequent keep-alive messages Jun 12, 2024
@els0r els0r merged commit 8d48cb1 into main Jun 12, 2024
8 checks passed
@els0r els0r deleted the 326-goquery-provide-stats-updates-less-frequently-and-with-flow-map-aggregates branch June 12, 2024 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

GoQuery: provide stats updates less frequently and with flow map aggregates
2 participants