zuschauer

IoT simplified - watchdog + azure storage options

(Der Zuschauer dt. - spectator)

Details

Watch a (or more) specified folder(s) for newly created or modified files and copy them to configured storage option. Supported options are Azure Storage Blob, ADLS Gen 2 or on-premise network drives. Azure functionality is implemented by leveraging Azure Blob Storage Python SDK. The AzureBlobFileSystem accepts all of the Async BlobServiceClient arguments. Zuschauer uses official APIs and opens files in read-only byte mode to copy files, it waits a second to prevent data loss. By default, write operations create BlockBlobs in Azure, which, once written can not be appended.

Usage

Easiest configuration is via gooey, first configure with via the interface:

python zuschauer.py

Specify required (and optional) arguments in interface.

Test configuration with a flag --dryrun to save config file and test connection without actually uploading anything.

python zuschauer.py --dryrun

By default, zuschauer is saving a JSON-config file if arguments are correct and connection can be established. That enables a headless mode: Just run it a second time (after configuration) and it will automatically load all pre-configured details. Use --existing flag to also upload all existing files (only necessary at first time or after interruption). It will not overwrite already uploaded files.

python zuschauer.py --existing

Zuschauer looks for a .config file in its root. If necessary, refer to any other path with --load:

python zuschauer.py --load 'path/to/config.ajsonfile'

Example config.ajsonfile:

{"paths": ["/path/to/watched_folder", "/second/path/to/watched_folder"], "filetypes": "pdf;tex", "storage": "Blob", "proxy": "", "refresh": 1, "recursive": true, "verbose": true, "dryrun": false}

Every other option is described via help:

python zuschauer.py -h

Upcoming features:

• write unit tests with complete coverage; although all features are tested and working fine
• no new features are planned yet

Author

Bernhard Häußler, TU Berlin

License

zuschauer is licensed under the MIT license, as included in the LICENSE file.

• Copyright (C) 2021 zuschauer contributors

Please see the git history for authorship information.

If not stated elsewise: Copyright (C) 2019-2021 Bernhard J. Häussler "drahnreb"

gooey: Copyright (c) 2013-2017 Chris Kiehl

azure sdk: Copyright (c) 2016 Microsoft