Bad checking of the --pattern argument

[elibarzilay]

First of all, the --pattern argument description is giving very little
information: "used for files globbing"?? I had to try it out to see
how it's used.

Second, the way that it's implemented is probably going to be surprising
for most people. I'm guessing that almost everyone would expect a
pattern like x* to match all files that begin with x, but it looks
like the pattern is actually matched against the full path that is
copied, and therefore a pattern like x* will either copy a whole
toplevel directory that starts with x or nothing at all if there is
none.

Related to this, it should be clear exactly what gets matched. In an
attempt to avoid bogus matches in a list that iterates over a list of
globs, I prepended */ to each one (yes, that was a mistake) which
resulted in skipping a toplevel index.html file, and this is
regardless of specifying a full path as my --source or cding into
the directory and using a --source .. So I'm guessing that this
pattern is matched against the full path without the prefix --source
directory (which is the same as the full container + / + blob name
without the part that is specified in --destination). Summarizing
this in a short CLI description is going to be difficult, but it is
very needed.

Yet another bit that must be documented is the fact that * matches
any character, including /. It would have been much better to
have * not match /s, and instead add a ** pattern for that.

Given that there are probably scripts that rely on this behavior of
--pattern (at least the script that I wrote does so), a way to resolve
this would be to add some new --file-pattern that gets matched only
against the basename of files, and not against any directories. Maybe
also another --path-pattern flag with * not matching /s, and **
that does.

[tjprescott]

@troydai can speak to more detail, but you can use the --dryrun flag to see what files/blobs would be targeted with --pattern without actually performing the operation.

[yonzhan]

add to S165.

[Juliehzl]

Hi @elibarzilay, thanks a lot for your feedback. Have you tried --dryrun flag to see the summary of target files?
I will add more detailed description and additional examples for --pattern usage. If you have any suggestion, feel free to let me know.

[elibarzilay]

@juhee0202: first, a dryrun flag is useful, but shouldn't come at the expense of documenting things. Second, I think that it should be very clear how the match is done, see my comments in the original issue text. For example, saying that * matches anything including / is useful -- but would ? match / too? What about [!x]? Will there be a match if the pattern matches any part of the name, or must it match from the start, from the end, or maybe it must match the whole thing?

I see in your PR that you added some text and some examples, but IMO it's still not answering many of these important questions?

(Extended note: the reason dryrun is insufficient: when you write some script to do something, you want to know that it will continue to behave as it should in the future. So a dry run on some patterns with some current state of my storage will not be sufficient.)