Add support for collecting and recording per-file metadata

CHANGELOG.md

@@ -29,6 +29,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 
 - The Git repository cloning behavior in the `scan` command can now be controlled with the new `--git-clone-mode MODE` parameter.
 
+- In the `scan` command, basic blob metadata is recorded in the datastore for each discovered blob, including blob size in bytes and guessed mime type and charset when available.
+  A path-based mechanism is used to guess mime type; at present, this only works for plain file inputs (i.e., not for blobs found in Git history).
+  Optionally, if the `libmagic` Cargo feature is enabled, libmagic (the guts of the `file` command-line program) is used to guess mime type and charset based on content for blobs from all sources.
+  This metadata is recorded for each blob in which matches are found, but this behavior can be enabled for all blobs using the new `--record-all-blobs true` parameter.
+  This newly-recorded metadata is included in output of the `report` command.
+
 
 ### Changes
 - Existing rules were modified to reduce both false positives and false negatives:
@@ -42,7 +48,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 - When a Git repository is cloned, the default behavior is to match `git clone --bare` instead of `git clone --mirror`.
   This new default behavior results in cloning potentially less content, but avoids cloning content from forks from repositories hosted on GitHub.
 
-- The command-line help has been refined for clarity
+- The command-line help has been refined for clarity.
+
+- Scanning performance has been improved on particular workloads by as much as 2x by recording matches to the datastore in larger batches.
+  This is particularly relevant to heavy multithreaded scanning workloads where the inputs have many matches.
 
 
 ### Fixes