sys/optparse: Add command line parser.

[jcarrano]

Contribution description

UPDATE April 2019: I completely reworked the API and the code and this description too.

This PR adds a generic command line parser that can be used by shell commands. See #11400 for an example.

Features

• Supports short (-c) and long (--color) options and switches in addition to positional arguments (and also optional positional arguments).
• Handling of arguments beginning with "-" by using "--" to mark the end of options.
• Configuration for parsers can be stored in static structures and arrays (in flash.)
• Generates friendly error messages and help strings.
• Common functionality included: parse ints, floats, set flags, count occurrences, etc.
• Extensible via user callbacks.

There are two components to this module: the data structures (i.e. the header) and the code itself (i.e. what is run). I'll explain why I consider both separately:

Code.

Currently shell commands implement their own parsing leading to:

• Duplicated code and duplicated effort.
• Duplicated bugs.
• Possible inconsistencies in command line argument syntax.
• The parsers end up being pretty basic.

The reason is that parsing anything is usually very easy to get wrong.

Data structures

Beyond de-duplicating code, a major motivation for this module is enabling a high-level interface to RIOT commands.

Interaction with RIOT commands is currently mediated by the shell. This is a text interface that is essentially free-form both in its input as well as the output. PR #10624 tries to solve this for the output side. This PR can provide a mechanism by which command input can be delivered. This would involve commands declaring their signature via "opt_conf_t" but, instead of using the code in "optparse.c" to convert lists of strings to values, a special module can be devised specifically for machine-to-machine interaction.

It is out of scope for this PR to explore such a module. The goal of this is to introduce a way to declare command line arguments that decouples the specification from the parsing.

Testing

I included unit tests. They are not yet complete.

Issues/PRs references

See: #11400.
For an example in which parsing took more time than it should, see #9523 . It was not a huge deal, but more than it could have been.

Fixes #3355

[kaspar030]

Thanks a lot for working on this!

Some first comments:

• we'll need to see some example code using this module
• there's a lot of macro usage. please try to remove those.
• riot convention is to typedef struct-types we use
• no need to use "extern" for function declarations (and against RIOT conventions)
• documentation comments should be in doxygen format

[jcarrano]

@kaspar030 The macros are to help define the setters. However, I'm thinking of removing the setters altogether. The reason I put the setters is to prevent the error where one could forget to set certain member of the structure.

Also, what are your thoughts on splitting the configuration structure so that the parser results are placed somewhere else? That would allow the configuration to be defined in a global const variable.
I'm thinking of making another struct like this:

typedef union opt_data { /**< this used to be in opt_rule */
        int *d_int;
        bool *d_bool;
        double *d_double;
        float *d_float;
        char **d_str;       /**< Pointer to a user-defined pointer */
        const char **d_cstr;  /**< Pointer to a user-defined pointer, constant variant */
        void *cb_data;
} opt_data_t;

typedef struct opt_result {
     void *arg_parser_data; /**< Callback data for arg_parser. moved from opt_conf */
     opt_data_t *rules_result; /*<  */
} opt_result_t;

[kaspar030]

BTW, did you check out existing code, like https://github.com/cofyc/argparse or https://github.com/docopt/docopt.c?

[jcarrano]

@kaspar030 docopt generates code (ugh!). argparse is quite similar to this, and seems good, but it does not support as many option types as this module.
In addition, as I mentioned before, I'm thinking it may be a good idea to modify this module so that the configuration is strictly read-only, but I wanted to know what other people thought.