Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

log: Add module and subsystem identifiers to log #6023

Closed
wants to merge 9 commits into from

Conversation

jlucovsky
Copy link
Contributor

Continuation of #6022

This changeset provides subsystem and module identifiers in the log when
the log format string contains "%S". By convention, the log format
surrounds "%S" with brackets.

The subsystem name is generally the same as the thread name. The module
name is derived from the source code module name and usually consists of
the first one or 2 segments of the name using the dash character as the
segment delimiter.

Issue 2497: redmine

Updates:

  1. Clang-format fixups

Describe changes:
This PR adds a subsystem and module identifier to SCLog messages when the log format includes %S. Subsystem and module identifiers are intrinsic properties of threads and source code modules (respectively).

New threads are assigned a subsystem identifier when the thread is created; the identifier is a Thread-Local-Storage variable declared in util-debug.c; values are assigned to it as threads are created using SCSetSubsystem (a macro defined in util-debug.h).

Module identifiers are derived from the source code module emitting the log message. A new CPP define __SCFILENAME__ is assigned to _sc_module (util-debug.h) at compile time. Rust source module names are determined dynamically during the calls to the log function.

Subsystem and module identifiers are added to log messages when the format contains %S. The generated log message will substitute a tag built from

  • The subsystem identifier. This corresponds to the calling thread which may be the Suricata main thread, or a subordinate thread for a task-specific function, e.g., RX#01.
  • (Optional) The module identifier will be included if set (not all source modules will set one but most source modules have been modified to set one.)
    The constructed tag is of the form subsystem-id[:module-identifier] (the brackets surrounding the module-identifier indicate the module-identifier is optional and are not included in the output; output formatting is strictly controlled by the log format in effect).

Also, two travis related changes are included in this PR

  1. Update Travis to use Xenial (16.04) rather than Trusty (14.04)
  2. Update to display config.log when Travis build errors occur.

#suricata-verify-pr:
#suricata-verify-repo:
#suricata-verify-branch:
#suricata-update-pr:
#suricata-update-repo:
#suricata-update-branch:
#libhtp-pr:
#libhtp-repo:
#libhtp-branch:

This changeset provides subsystem and module identifiers in the log when
the log format string contains "%S". By convention, the log format
surrounds "%S" with brackets.

The subsystem name is generally the same as the thread name. The module
name is derived from the source code module name and usually consists of
the first one or 2 segments of the name using the dash character as the
segment delimiter.
Temporary update to display config.log file on failure.
Address the issue with supplying the actual thread name for the thread
formatting character `%m`. The subsystem identifier is a reliable proxy
for the thread name and can be used here; it avoids the potential
deadlock condition.
This commit converts the file logging logic to use JsonBuilder instead
of libjansson.
@codecov
Copy link

codecov bot commented Apr 4, 2021

Codecov Report

Merging #6023 (e87b822) into master (b05bd05) will decrease coverage by 0.00%.
The diff coverage is 73.84%.

@@            Coverage Diff             @@
##           master    #6023      +/-   ##
==========================================
- Coverage   76.91%   76.91%   -0.01%     
==========================================
  Files         612      612              
  Lines      187721   187795      +74     
==========================================
+ Hits       144391   144442      +51     
- Misses      43330    43353      +23     
Flag Coverage Δ
fuzzcorpus 52.64% <46.09%> (+0.05%) ⬆️
suricata-verify 50.27% <75.21%> (-0.03%) ⬇️
unittests 63.20% <44.91%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@jlucovsky
Copy link
Contributor Author

Continued in #6024

@jlucovsky jlucovsky closed this Apr 5, 2021
@jlucovsky jlucovsky deleted the 2497/20 branch January 30, 2022 15:30
catenacyber added a commit to catenacyber/suricata that referenced this pull request Jun 22, 2023
As it is part of the boundary

Ticket: OISF#6023

On the way, look for urls even on incomplete lines
catenacyber added a commit to catenacyber/suricata that referenced this pull request Jun 22, 2023
As it is part of the boundary

Ticket: OISF#6023

On the way, look for urls even on incomplete lines
catenacyber added a commit to catenacyber/suricata that referenced this pull request Jun 22, 2023
As it is part of the boundary

Ticket: OISF#6023

On the way, look for urls even on incomplete lines
catenacyber added a commit to catenacyber/suricata that referenced this pull request Jun 29, 2023
As it is part of the boundary

Ticket: OISF#6023

On the way, look for urls even on incomplete lines
catenacyber added a commit to catenacyber/suricata that referenced this pull request Jun 29, 2023
As it is part of the boundary

Ticket: OISF#6023

On the way, look for urls even on incomplete lines
victorjulien pushed a commit to victorjulien/suricata that referenced this pull request Jun 29, 2023
As it is part of the boundary

Ticket: OISF#6023

On the way, look for urls even on incomplete lines
victorjulien pushed a commit to victorjulien/suricata that referenced this pull request Jun 29, 2023
As it is part of the boundary

Ticket: OISF#6023

On the way, look for urls even on incomplete lines
victorjulien pushed a commit to victorjulien/suricata that referenced this pull request Jun 29, 2023
As it is part of the boundary

Ticket: OISF#6023

On the way, look for urls even on incomplete lines
victorjulien pushed a commit to victorjulien/suricata that referenced this pull request Jun 30, 2023
As it is part of the boundary

Ticket: OISF#6023

On the way, look for urls even on incomplete lines
victorjulien pushed a commit to victorjulien/suricata that referenced this pull request Jun 30, 2023
As it is part of the boundary

Ticket: OISF#6023

On the way, look for urls even on incomplete lines
victorjulien pushed a commit to victorjulien/suricata that referenced this pull request Jun 30, 2023
As it is part of the boundary

Ticket: OISF#6023

On the way, look for urls even on incomplete lines

(cherry picked from commit 2a768df)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

1 participant