Armory 0.14.2

There was a build error in v0.14.1 so it should not be used. Use v0.14.2 (this release) or later.

Changelog

Datasets

New CARLA multimodal (rgb, depth) object detection test set (#1211)
New CARLA video tracking test set (#1219)

Attacks

Fixed issue with CARLA video tracking attack where patch sometimes didn't fully cover green screen (#1213)
Updated CARLA object detection patch attack to increase its efficacy (#1212)
Updated attack parameters for CARLA object detection default configs (#1230)

Models

Added CARLA multimodal object detection robust fusion model as a baseline defense (#1217)

Metrics

Updated CARLA object detection scenario to additionally measure metrics using benign predictions as ground-truth (#1225)

Armory 0.14.0

Changelog

Datasets

New CARLA multimodal (rgb, depth) object detection train (#1173) and dev (#1182) datasets
New CARLA video tracking dev dataset (#1170)
Updated RESISC-10 dataset from 64x64 images to 256x256 (#1155)

Scenarios

Major refactor/modularization of scenario code (#1114)
New CARLA video tracking scenario (#1170)
New CARLA object detection scenario (#1182)

Attacks

Integrated AdversarialPhysicalTexture attack on CARLA video tracking into armory.art_experimental.attacks (#1170)
Integrated Robust DPatch attack on CARLA object detectors with color correction into armory.art_experimental.attacks (#1182)

Models

Added CARLA single-modality object detection baseline PyTorch Faster-RCNN (#1160)
Added CARLA multimodal object detection baseline PyTorch Faster-RCNN (#1161)
Added CARLA video tracking baseline PyTorch GoTurn model (#1170)

Metrics

New video tracking mean iou metric for CARLA video tracking scenario (#1170)
Various new metrics for increased granularity in measurement of object detection performance (#1182)
For metric functions with kwargs, kwarg values are now accessible from configuration file (#1187)

User experience

Overhaul of scenario code now allows for more interactive sample-by-sample execution of scenarios (#1114)
New CLI tool for filtering datasets by class (#1162)
New CLI tool for custom-indexing specific dataset elements (#1162)

Dependency/library updates

Upgraded twosixarmory Docker images to ART 1.8.1 (#1181)

Armory 0.13.5

• adds an abstains() metric
• additional artifact storing for poisoning scenarios

Armory 0.13.4

• updates scenario configs according to the eval 3 T&E plan
• enables random patch location for PGDPatch and a newly added art_experimental version of RobustDPatch. Note: this, along with applying the patch inside of generate(), is the only difference compared to ART's RobustDPatch

Armory 0.13.3

Updates

Upgrade to ART 1.6.2 (#1106)
Enables computing use of ground truth y with Faster-RCNN model (#1106)
Improved instrutions for --no-docker installations (#1112)

Armory 0.13.2

Datasets

Integration of COCO dataset (#1097)
Integration of DAPRICOT test set (#1096)

ARMORY v0.13.1

Datasets

Integration of CIFAR100 dataset (#1048)
New RESISC-10 datset for poisoning (#1038)

Scenarios

Poisoning scenario with blended trigger (#1049)
Refinement of D-APRICOT scenario including new attack success metric (#1040)
RESISC-10 poisoning scenario (#1065)

Performance improvement/bug fixes

Updates to ensure compatibility with newer versions of ART (#1037, #1060, #1062, #1063)
Refactor of how targeted attacks are configured, allowing users to customize target label generation function (#1052)
Refactor/simplification of average precision (AP) object detection metrics (#1046)
Pinned to numpy 1.19.2 for TF1 Docker image to avoid TF/Numpy bug (#1056)
No longer assume external repository default branch is 'master' (#1064)
Pytorch example for GTSRB dirty-label backdoor attack (#1067)

ARMORY v0.13.0

Changelog

Dependency/library updates

• Update docker dependencies (including Pytorch 1.7 update and ART 1.6) and simplify Docker build process (#1006, #1008, #1017)

Datasets

• DAPRICOT dev adversarial dataset (#1021)
• Add optional indexing to datasets (#1003)
• Add optional filtering by class of datasets (#1019)

Scenarios

• DAPRICOT scenario (#1021)
• Add (optional) single multipath channel to ASR scenario (#1030) with example config file scenario_configs/asr_deepspeech_baseline_fgsm_channel.json

User experience

• Add option to skip misclassified examples in adversarial attacks (#1005)
• Ensure that all control arguments, e.g. num-eval-batches or skip-benign, are both obeyed from configs and recorded in output (#1027)

ARMORY v0.12.3

Changelog

Breaking change

So2Sat has its own new scenario -- configs for this scenario should be updated accordingly.

Dependency/library updates

• New containers for clean-label poisoning scenario (#949)

Documentation updates

• Updated scenarios documentation (#931)

Datasets

• UCF101 dataset (#954)
• Full Librispeech dataset with all splits (#939)
• Updated object detection label format (#941)

Attacks

• Cascading attack (#953)
• Frame saliency attack (#942)
• Updated hyperparameters for Imperceptible ASR attack (#925)
• Enable targeted object detection attacks (#920)
• Kenansville DFT attack (#916)
• Adding patch_method for overriding model methods for adaptive attacks (#914)

Scenarios

• Separate So2Sat Scenario with direct attacks on SAR/optical, separate
  perturbation metrics for SAR/optical, and explicit masking of SAR/optical
  (#948, #915).
• Clean-label backdoor poisoning scenario (#949)

Baselines

• Added baseline defense of Apricot adversarial dataset (#930)

Metrics

• SNR: remove constraint on 1D inputs (#946)
• Enable OD metrics to handle arbitrary batch size (#924)

User experience

• Auto-expand tarballs submitted as weights files (#937)
• Save adversarial examples for listening/viewing (#928, #956)
• Update HOME directory in container for jupyter to work in non-root mode (#945)
• Log output dir at end of run (#912)

Bugfixes

• Fix TORCH_HOME in --nodocker mode (#944)
• Fix clip values in some baseline models (#940)
• Canonical preprocessing: correct type for image datasets with variable length
  when batch elements are same shape (#927)

ARMORY v0.12.2

Changelog

Dependency/library updates

• Pin to ART 1.4.2 rather than dev branch (#887)
• Update container for ASR scenario (#820, #891)

Documentation updates

• Update documentation on datasets, licensing, scenarios, baseline models, and
  metrics (#826, #844, #879)
• FAQ update (#831)
• Documentation of new --validate-config flag (#875)
• Documentation of --skip-attack and --skip-benign flags (#900)

Datasets

• Support variable length labels in xView dataset (#841)
• Dataset index and slicing (#878)
• Canonical preprocessing of adversarial, non-official scenario, poisoning
  datasets (#829, #888)

Integration

Sanity checks of model configurations (#875)

Attacks

• Example configs for FGSM with ASR model (#859)
• Targeted labels that (roughly) match length of true transcript in ASR attack
  (#863)
• Rescale epsilon in UCF101 attacks (#865)
• Added pgd_patch art_experimental attack (#883)

Metrics

• APRICOT patch targeted adversarial patch metric (#866)
• Split metrics when targeted attack is present (#884)

Performance / infrastructure

• Faster untarring of cached datasets when possible (#832)
• Save deep speech model weights in Armory directory (#870)
• Optionally truncate very long videos in MARS model to limit memory usage
  (#880)

User interface

• Allow piping of configs from STDIN (#842)
• Add --skip-attack flag (#877)

Bugfixes

• Update UCF101 canonical preprocessing to account for 4 videos with nonstandard
  shapes and update MARS model to properly handle the nonstandard shapes (#889)
• Eliminate spurious errors on container shutdown after armory exec or launch
  (#828)
• Jpegcompression defense consistent datatypes (#839)
• Updated global_max_length parameter for imperceptible ASR attack to be set to
  the max length of the test and dev dataset splits (#845)
• Enable xView model to run on GPU (#853)