Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd: improve command usage message by grouping related flags #443

Merged
merged 2 commits into from
Dec 9, 2020

Conversation

rolinh
Copy link
Member

@rolinh rolinh commented Dec 3, 2020

I wanted to create multiple commits but it turns out to be complicated.
Let's summarize what this PR does:

  • Add cmd/common/template where a Usage() can be used by commands to
    generate a custom template based on the provided pflag.FlagSets.
    This allows grouping related flags when displaying help/usage of a
    command or subcommand.
  • With the help of Usage() described above, the custom usage function
    for the observer subcommand can be dropped. It was quite hacky and
    easy to get out of sync by adding a flag to the observe command and
    forgetting to update the usage formatting function to indicate to
    which flag group the newly added flag belongs. The observe command
    now has several defines several pflag.FlagSet for each flag group
    (selectors, filters, formatting, others).
  • Global flags (ie flags that are persisted from the root command) have
    been split into to pflag.FlagSet in cmd/common/config:
    GlobalFlags and ServerFlags. The former contains all global flags
    (ie: debug and config) and the latter all flags related to connecting
    to a Hubble server instance.
  • The global flags section is always printed for each command or
    subcommand as it apply to any command. However, the new ServerFlags
    flag set needs to be added to the help message by any command or
    subcommand that makes use of it. For the user, this is really
    convenient as these flags appear in the help message of a command only
    when relevant instead of always (eg: the completion command does not
    use the server flags and this does not display help for these when
    running hubble completion -h). The ServerFlags are nonetheless
    global in the sense that they are persistent flags added to the root
    command. This is done for 2 reasons: not breaking the existing
    behavior and allowing users to use these flags in any order (e.g.
    hubble --server foo observe also works).
  • Instead of using hardcoded strings for every configuration key in
    every command where they need to be used, constants have been defined
    in cmd/common/config and shall be used instead.
  • Defaults for the configuration directory, fallback configuration
    directory and configuration file have been moved to pkg/defaults.

EDIT: sometimes, it's clearer to understand with examples.
This is the help for observe, (the grouping of flags is now automatic and based on which flagset the flag is added):

$ hubble observe -h
Observe provides visibility into flow information on the network and
application level. Rich filtering enable observing specific flows related to
individual pods, services, TCP connections, DNS queries, HTTP requests and
more.

Usage:
  hubble observe [flags]

Selectors Flags:
      --all            Get all flows stored in Hubble's buffer
  -f, --follow         Follow flows output
      --last uint      Get last N flows stored in Hubble's buffer (default 20)
      --since string   Filter flows since a specific date (relative or RFC3339)
      --until string   Filter flows until a specific date (relative or RFC3339)

Filters Flags:
      --fqdn filter             Show all flows related to the given fully qualified domain name (e.g. "*.cilium.io").
      --from-fqdn filter        Show all flows originating at the given fully qualified domain name (e.g. "*.cilium.io").
      --from-identity filter    Show all flows originating at an endpoint with the given security identity
      --from-ip filter          Show all flows originating at the given IP address.
      --from-label filter       Show only flows originating in an endpoint with the given labels (e.g. "key1=value1", "reserved:world")
      --from-namespace filter   Show all flows originating in the given Kubernetes namespace.
      --from-pod filter         Show all flows originating in the given pod name ([namespace/]<pod-name>). If namespace is not provided, 'default' is used
      --from-port filter        Show only flows with the given source port (e.g. 8080)
      --from-service filter     Show all flows originating in the given service ([namespace/]<svc-name>). If namespace is not provided, 'default' is used
      --http-method filter      Show only flows which match this HTTP method (e.g. "get", "post")
      --http-path filter        Show only flows which match this HTTP path regular expressions (e.g. "/page/\\d+")
      --http-status filter      Show only flows which match this HTTP status code prefix (e.g. "404", "5+")
      --identity filter         Show all flows related to an endpoint with the given security identity
      --ip filter               Show all flows related to the given IP address.
  -l, --label filter            Show only flows related to an endpoint with the given labels (e.g. "key1=value1", "reserved:world")
  -n, --namespace filter        Show all flows related to the given Kubernetes namespace.
      --node-name filter        Show all flows which match the given node names (e.g. "k8s*", "test-cluster/*.company.com")
      --not filter[=true]       Reverses the next filter to be blacklist i.e. --not --from-ip 2.2.2.2
      --pod filter              Show all flows related to the given pod name ([namespace/]<pod-name>). If namespace is not provided, 'default' is used
      --port filter             Show only flows with given port in either source or destination (e.g. 8080)
      --protocol filter         Show only flows which match the given L4/L7 flow protocol (e.g. "udp", "http")
      --service filter          Show all flows related to the given service ([namespace/]<svc-name>). If namespace is not provided, 'default' is used
      --to-fqdn filter          Show all flows terminating at the given fully qualified domain name (e.g. "*.cilium.io").
      --to-identity filter      Show all flows terminating at an endpoint with the given security identity
      --to-ip filter            Show all flows terminating at the given IP address.
      --to-label filter         Show only flows terminating in an endpoint with given labels (e.g. "key1=value1", "reserved:world")
      --to-namespace filter     Show all flows terminating in the given Kubernetes namespace.
      --to-pod filter           Show all flows terminating in the given pod name ([namespace/]<pod-name>). If namespace is not provided, 'default' is used
      --to-port filter          Show only flows with the given destination port (e.g. 8080)
      --to-service filter       Show all flows terminating in the given service ([namespace/]<svc-name>). If namespace is not provided, 'default' is used
  -t, --type filter             Filter by event types TYPE[:SUBTYPE] ([policy-verdict drop debug capture trace l7 agent])
      --verdict filter          Show only flows with this verdict [FORWARDED, DROPPED, ERROR]

Formatting Flags:
      --compact           Deprecated. Use '--output compact' instead.
      --dict              Deprecated. Use '--output dict' instead.
      --ip-translation    Translate IP addresses to logical names such as pod name, FQDN, ... (default true)
  -j, --json              Deprecated. Use '--output json' instead.
      --numeric           Display all information in numeric form
  -o, --output string     Specify the output format, one of:
                           compact:  Compact output
                           dict:     Each flow is shown as KEY:VALUE pair
                           json:     JSON encoding
                           jsonpb:   Output each GetFlowResponse according to proto3's JSON mapping
                           table:    Tab-aligned columns
      --print-node-name   Print node name in output

Server Flags:
      --server string                 Address of a Hubble server (default "unix:///var/run/cilium/hubble.sock")
      --timeout duration              Hubble server dialing timeout (default 5s)
      --tls                           Specify that TLS must be used when establishing a connection to a Hubble server.
                                      By default, TLS is only enabled if the server address starts with 'tls://'.
      --tls-allow-insecure            Allows the client to skip verifying the server's certificate chain and host name.
                                      This option is NOT recommended as, in this mode, TLS is susceptible to machine-in-the-middle attacks.
                                      See also the 'tls-server-name' option which allows setting the server name.
      --tls-ca-cert-files strings     Paths to custom Certificate Authority (CA) certificate files.The files must contain PEM encoded data.
      --tls-client-cert-file string   Path to the public key file for the client certificate to connect to a Hubble server (implies TLS).
                                      The file must contain PEM encoded data.
      --tls-client-key-file string    Path to the private key file for the client certificate to connect a Hubble server (implies TLS).
                                      The file must contain PEM encoded data.
      --tls-server-name string        Specify a server name to verify the hostname on the returned certificate (eg: 'instance.hubble-relay.cilium.io').

Other Flags:
  -s, --silent-errors   Silently ignores errors and warnings

Global Flags:
      --config string   Optional config file (default "/home/isovalent/.config/hubble/config.yaml")
  -D, --debug           Enable debug messages

Get help:
  -h, --help    Help for any command or subcommand

This is the help for config. Notice that the Server Flags section is absent.

$ hubble config -h
Config allows to modify or view the hubble configuration. Global hubble options
can be set via flags, environment variables or a configuration file. The
following precedence order is used:

1. Flag
2. Environment variable
3. Configuration file
4. Default value

The "config view" subcommand provides a merged view of the configuration. The
"config set" and "config reset" subcommand modify values in the configuration
file.

Environment variable names start with HUBBLE_ followed by the flag name
capitalized where eventual dashes ('-') are replaced by underscores ('_').
For example, the environment variable that corresponds to the "--server" flag
is HUBBLE_SERVER. The environment variable for "--tls-allow-insecure" is
HUBBLE_TLS_ALLOW_INSECURE and so on.

Usage:
  hubble config [flags]
  hubble config [command]

Available Commands:
  get         Get an individual value in the hubble config file
  reset       Reset all or an individual value in the hubble config file
  set         Set an individual value in the hubble config file
  view        Display merged configuration settings

Global Flags:
      --config string   Optional config file (default "/home/robin/.config/hubble/config.yaml")
  -D, --debug           Enable debug messages

Get help:
  -h, --help    Help for any command or subcommand

Use "hubble config [command] --help" for more information about a command.

@rolinh rolinh added ⌨️ area/cli Impacts the command line interface of any command in the repository. release-note/minor This PR introduces functionality that users may find relevant to operating Hubble. labels Dec 3, 2020
cmd/common/template/usage_test.go Show resolved Hide resolved
cmd/node/list.go Show resolved Hide resolved
As part of refactoring work to improve commands help message output, it
is useful to be able to get the name of a given `pflag.FlagSet`. In the
latest released version (v1.0.5, released 2019.09.18), this information
is not available as the `FlagSet` name, although provided by the caller,
is a private field. However, the commit that was pushed immediately
after v1.0.5 adds a `Name()` method to FlagSet, which comes in handy
(see [0] for details). As the difference with the v1.0.5 is low risk and
no new releases have been published, update pflag to the commit right
after v1.0.5.

[0]: spf13/pflag@81378bb

Signed-off-by: Robin Hahling <robin.hahling@gw-computing.net>
I wanted to create multiple commits but it turns out to be complicated.
Let's summarize what this commit does:

- Add `cmd/common/template` where a `Usage()` can be used by commands to
  generate a custom template based on the provided `pflag.FlagSets`.
  This allows grouping related flags when displaying help/usage of a
  command or subcommand.
- With the help of `Usage()` described above, the custom usage function
  for the `observer` subcommand can be dropped. It was quite hacky and
  easy to get out of sync by adding a flag to the `observe` command and
  forgetting to update the usage formatting function to indicate to
  which flag group the newly added flag belongs. The `observe` command
  now has several defines several `pflag.FlagSet` for each flag group
  (selectors, filters, formatting, others).
- Global flags (ie flags that are persisted from the root command) have
  been split into to `pflag.FlagSet` in `cmd/common/config`:
  `GlobalFlags` and `ServerFlags`. The former contains all global flags
  (ie: debug and config) and the latter all flags related to connecting
  to a Hubble server instance.
- The global flags section is always printed for each command or
  subcommand as it apply to any command. However, the new `ServerFlags`
  flag set needs to be added to the help message by any command or
  subcommand that makes use of it. For the user, this is really
  convenient as these flags appear in the help message of a command only
  when relevant instead of always (eg: the `completion` command does not
  use the server flags and this does not display help for these when
  running `hubble completion -h`). The `ServerFlags` are nonetheless
  global in the sense that they are persistent flags added to the root
  command. This is done for 2 reasons: not breaking the existing
  behavior and allowing users to use these flags in any order (e.g.
  `hubble --server foo observe` also works).
- Instead of using hardcoded strings for every configuration key in
  every command where they need to be used, constants have been defined
  in `cmd/common/config` and shall be used instead.
- Defaults for the configuration directory, fallback configuration
  directory and configuration file have been moved to `pkg/defaults`.

Signed-off-by: Robin Hahling <robin.hahling@gw-computing.net>
Copy link
Member

@gandro gandro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome! Due the the large size of the PR, it's hard to track what are actual code changes and what just moves code around, but I did try out the branch locally with a bunch of commands and it worked as expected.

@maintainer-s-little-helper maintainer-s-little-helper bot added the ready-to-merge This PR has passed all tests and received consensus from code owners to merge. label Dec 8, 2020
@rolinh rolinh merged commit 8435223 into master Dec 9, 2020
@rolinh rolinh deleted the pr/rolinh/refactor-help branch December 9, 2020 08:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
⌨️ area/cli Impacts the command line interface of any command in the repository. ready-to-merge This PR has passed all tests and received consensus from code owners to merge. release-note/minor This PR introduces functionality that users may find relevant to operating Hubble.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants