0.13.1 release (#1068)

* update version (#1034)

* update version

* update json version

* set channels_first False for relevant pytorch models (#1037)

* Resisc10 poison dataset (#1038)

* update version

* revert version

* added resisc10 poison dataset

* Update refs to point to S3, add cached dataset

* Add test for resisc10 dataset

Co-authored-by: David Slater <david.slater@twosixlabs.com>
Co-authored-by: Neal Gupta <neal.gupta@twosixlabs.com>

* Build tag script (#1035)

* update build script

* added command echoes

* pinning to numpy 1.19.2 to avoid ART error (#1056)

* updating comment on relevant np issue (#1057)

* CIFAR-100 dataset (#1048)

* Add CIFAR100 dataset

* Typo

* label targeter refactor (#1052)

* renamed file

* fix typo while remaining backwards compatible

* refactored label targeter config loading logic

* updating configs accordingly

* adding one more config

* changing filename back to labels.py

* adding warning message for deprecated 'scheme' key

* removing code that shouldn't have been pushed/fixing typo

* update configs for label_targeters.py --> labels.py change

* removing configs i didn't meant to push

* keyword-only args; change config 'args' --> 'kwargs'

* refactor object detection metrics (#1046)

* refactored object_detection_AP_per_class

* refactor dapricot and apricot AP functions

* update tests for od metrics refactor

* removing od metrics that aren't useful

* modify od format check function; renamed a couple variables

* refactor to remove unnecessary elifs; rename append() to add_results()

* formatting

* renamed method

* document function input format

* bumping ART 1.6.0 --> 1.6.1 (#1062)

* updating baseline config to be compatible with newer versions of ART (#1063)

* don't assume default branch is named master (#1064)

* Poisoning scenario with blended trigger (#1049)

* * Update image-based trigger to allow blending
* Use blended trigger to enable bullethole clbd attack

* Update docker image reference in config

* Update pathing to load image path when armory is pip installed

* Use armory.__file__ to simplify relative pathing

* preprocessing defense fixes (#1060)

* call set_params() so classifier.all_framework_preprocessing attribute is updated

* no longer using kwarg which ART has removed

* use get_params() to append defenses; removed if ART < 1.5 logic

* flake8

* dapricot updates (#1040)

* adjust scale for insert_patch(); make patch shape square

* force dapricot attacks to be targeted

* formatting

* increment label index in loss_gradient for baseline 0-indexed model

* need to decrement not increment

* adding dapricot_patch_target_success metric

* resetting this variable to empty list since dparicot has no nontargeted tasks

* this workaround is no longer necessary per previous commit

* deleting commented out code that was accidentally pushed

* removing config since DPatch doesn't support targeted attack yet

* formatting

* reshape box to flat array

* add docs for fn input format

* formatting

* updated dapricot RobustDPatch attack and associated files

* ran black, flake8, and format_json

* adding targeted Dpatch to file itself so we dont need to use dev version of ART

* minor documentation/error msg update

* removing channels_first logic since x will always be channels_last with armory

* black formatting

* adding clarifying comment

* set num_images_per_patch in scenario code; force threat model to be specified in scenario code

* minor modifications to error messages

* dont overwrite model kwargs; add 'batch_size' kwarg to baseline models get_art_model()

* add warning if batch_size model_kwarg isnt set; also edited comment at top of script

* removing unused line of code

* removing code that has no effect on attack

* avoid warning message by renaming colour fn to its updated name

* set check on lower bound of brightness range

* fix typo

* point to armory 0.13.1 in config

* point to armory 0.13.1 in pgd config too

* only display warning for physical attacks

* flake8

* the code in this file was moved to inside the attack

* removing dapricot robust dpatch attack and associated utility functions

* flake8

Co-authored-by: Yusong Tan <ytan@mitre.org>

* Resisc10 poison (#1065)

* * Update image-based trigger to allow blending
* Use blended trigger to enable bullethole clbd attack

* Update docker image reference in config

* Update pathing to load image path when armory is pip installed

* resisc10 poison scenario related files

* Updated poisoning attack call based on ART updates, fix channel ordering for image data

* Update metrics method names

* Update config to work with pip-installed armory

Co-authored-by: Neal Gupta <neal.gupta@twosixlabs.com>

* Poisoning scenario Pytorch example (#1067)

* Pytorch compatibility for poisoning scenarios, example Pytorch config for dlbd

* Configs closer to eval approach

Co-authored-by: davidslater <david.slater@twosixlabs.com>
Co-authored-by: yusong-tan <59029053+yusong-tan@users.noreply.github.com>
Co-authored-by: Neal Gupta <neal.gupta@twosixlabs.com>
Co-authored-by: Yusong Tan <ytan@mitre.org>

MANIFEST.in

@@ -1,3 +1,4 @@
 recursive-include armory *.json
 recursive-include armory *.txt
 recursive-include armory *.ini
+recursive-include armory *.png

armory/__init__.py

@@ -8,7 +8,7 @@
 
 
 # Semantic Version
-__version__ = "0.13.0"
+__version__ = "0.13.1"
 
 
 # Submodule imports

armory/art_experimental/attacks/dapricot_patch.py

@@ -1,110 +1,117 @@
+"""
+Copyright 2021 The MITRE Corporation. All rights reserved
+"""
+import logging
 import numpy as np
 import cv2
-
-from art.attacks.evasion import RobustDPatch, ProjectedGradientDescent
-from armory.art_experimental.utils.dapricot_patch_utils import (
-    insert_patch,
-    shape_coords,
-    create_mask,
-)
-
-
-class DApricotPatch(RobustDPatch):
-    """
-    """
-
-    def __init__(self, estimator, **kwargs):
-        super().__init__(estimator=estimator, **kwargs)
-
-    def generate(self, x, y_object=None, y_patch_metadata=None, **generate_kwargs):
-
-        if "threat_model" not in generate_kwargs:
-            raise ValueError(
-                "'threat_model' kwarg must be defined in attack config's"
-                "'generate_kwargs' as one of ('physical', 'digital')"
+import math
+
+from art.attacks.evasion import ProjectedGradientDescent
+
+logger = logging.getLogger(__name__)
+
+
+def shape_coords(h, w, obj_shape):
+    if obj_shape == "octagon":
+        smallest_side = min(h, w)
+        pi = math.pi
+        h_init = 0
+        w_init = smallest_side / 2 * math.sin(pi / 2) / math.sin(3 * pi / 8)
+        rads = np.linspace(0, -2 * pi, 8, endpoint=False) + 5 * pi / 8
+        h_coords = [h_init * math.cos(r) - w_init * math.sin(r) + h / 2 for r in rads]
+        w_coords = [h_init * math.sin(r) + w_init * math.cos(r) + w / 2 for r in rads]
+        coords = [
+            (int(round(wc)), int(round(hc))) for hc, wc in zip(h_coords, w_coords)
+        ]
+    elif obj_shape == "diamond":
+        if h <= w:
+            coords = np.array(
+                [
+                    [w // 2, 0],
+                    [min(w - 1, int(round(w / 2 + h / 2))), h // 2],
+                    [w // 2, h - 1],
+                    [max(0, int(round(w / 2 - h / 2))), h // 2],
+                ]
+            )
+        else:
+            coords = np.array(
+                [
+                    [w // 2, max(0, int(round(h / 2 - w / 2)))],
+                    [w - 1, h // 2],
+                    [w // 2, min(h - 1, int(round(h / 2 + w / 2)))],
+                    [0, h // 2],
+                ]
             )
-        elif generate_kwargs["threat_model"].lower() not in ("physical", "digital"):
-            raise ValueError(
-                f"'threat_model must be set to one of ('physical', 'digital'), not {generate_kwargs['threat_model']}."
+    elif obj_shape == "rect":
+        # 0.8 w/h aspect ratio
+        if h > w:
+            coords = np.array(
+                [
+                    [max(0, int(round(w / 2 - 0.4 * h))), 0],
+                    [min(w - 1, int(round(w / 2 + 0.4 * h))), 0],
+                    [min(w - 1, int(round(w / 2 + 0.4 * h))), h - 1],
+                    [max(0, int(round(w / 2 - 0.4 * h))), h - 1],
+                ]
             )
-
         else:
-            threat_model = generate_kwargs["threat_model"].lower()
+            coords = np.array(
+                [
+                    [0, max(0, int(round(h / 2 - w / 1.6)))],
+                    [w - 1, max(0, int(round(h / 2 - w / 1.6)))],
+                    [w - 1, min(h - 1, int(round(h / 2 + w / 1.6)))],
+                    [0, min(h - 1, int(round(h / 2 + w / 1.6)))],
+                ]
+            )
+    else:
+        raise ValueError('obj_shape can only be {"rect", "diamond", "octagon"}')
 
-        num_imgs = x.shape[0]
-        attacked_images = []
+    return np.array(coords)
 
-        if threat_model == "digital":
-            for i in range(num_imgs):
-                gs_coords = y_patch_metadata[i]["gs_coords"]
-                patch_x, patch_y = gs_coords[0]  # upper left coordinates of patch
-                self.patch_location = (patch_x, patch_y)
-                patch_width = np.max(gs_coords[:, 0]) - np.min(gs_coords[:, 0])
-                patch_height = np.max(gs_coords[:, 1]) - np.min(gs_coords[:, 1])
-                self.patch_shape = (patch_height, patch_width, 3)
-
-                # self._patch needs to be re-initialized with the correct shape
-                if self.estimator.clip_values is None:
-                    self._patch = np.zeros(shape=self.patch_shape)
-                else:
-                    self._patch = (
-                        np.random.randint(0, 255, size=self.patch_shape)
-                        / 255
-                        * (
-                            self.estimator.clip_values[1]
-                            - self.estimator.clip_values[0]
-                        )
-                        + self.estimator.clip_values[0]
-                    )
-
-                patch = super().generate(np.expand_dims(x[i], axis=0))
-
-                patch_geometric_shape = (
-                    y_patch_metadata[i]["shape"].tobytes().decode("utf-8")
-                )
-                img_with_patch = insert_patch(
-                    gs_coords, x[i], patch, patch_geometric_shape,
-                )
-                attacked_images.append(img_with_patch)
-        else:
-            # generate patch using center image
-            gs_coords_center_img = y_patch_metadata[1]["gs_coords"]
-            patch_x, patch_y = gs_coords_center_img[
-                0
-            ]  # upper left coordinates of patch
-            self.patch_location = (patch_x, patch_y)
-            patch_width = np.max(gs_coords_center_img[:, 0]) - np.min(
-                gs_coords_center_img[:, 0]
-            )
-            patch_height = np.max(gs_coords_center_img[:, 1]) - np.min(
-                gs_coords_center_img[:, 1]
-            )
-            self.patch_shape = (patch_height, patch_width, 3)
-
-            # self._patch needs to be re-initialized with the correct shape
-            if self.estimator.clip_values is None:
-                self._patch = np.zeros(shape=self.patch_shape)
-            else:
-                self._patch = (
-                    np.random.randint(0, 255, size=self.patch_shape)
-                    / 255
-                    * (self.estimator.clip_values[1] - self.estimator.clip_values[0])
-                    + self.estimator.clip_values[0]
-                )
 
-            patch = super().generate(np.expand_dims(x[1], axis=0))
+def in_polygon(x, y, vertices):
+    """
+    Determine if a point (x,y) is inside a polygon with given vertices
 
-            for i in range(num_imgs):
-                gs_coords = y_patch_metadata[i]["gs_coords"]
-                patch_geometric_shape = (
-                    y_patch_metadata[i]["shape"].tobytes().decode("utf-8")
-                )
+    Ref: https://www.eecs.umich.edu/courses/eecs380/HANDOUTS/PROJ2/InsidePoly.html
+    """
+    n_pts = len(vertices)
+    i = 0
+    j = n_pts - 1
+    c = False
+
+    while i < n_pts:
+        if (
+            # y coordinate of the point has to be between the y coordinates of the i-th and j-th vertices
+            ((vertices[i][1] <= y) and (y < vertices[j][1]))
+            or ((vertices[j][1] <= y) and (y < vertices[i][1]))
+        ) and (
+            # x coordinate of the point is to the left of the line connecting i-th and j-th vertices
+            x
+            < (vertices[j][0] - vertices[i][0])
+            * (y - vertices[i][1])
+            / (vertices[j][1] - vertices[i][1])
+            + vertices[i][0]
+        ):
+            c = not c
+        j = i
+        i = i + 1
+    return c
+
+
+def create_mask(mask_type, h, w):
+    """
+    create mask according to shape
+    """
 
-                img_with_patch = insert_patch(
-                    gs_coords, x[i], patch, patch_geometric_shape,
-                )
-                attacked_images.append(img_with_patch)
-        return np.array(attacked_images)
+    mask = np.zeros((h, w, 3))
+    coords = shape_coords(h, w, mask_type)
+
+    for i in range(h):
+        for j in range(w):
+            if in_polygon(i, j, coords):
+                mask[i, j, :] = 1
+
+    return mask
 
 
 class DApricotMaskedPGD(ProjectedGradientDescent):
@@ -114,21 +121,7 @@ class DApricotMaskedPGD(ProjectedGradientDescent):
     def __init__(self, estimator, **kwargs):
         super().__init__(estimator=estimator, **kwargs)
 
-    def generate(self, x, y_object=None, y_patch_metadata=None, **generate_kwargs):
-
-        if "threat_model" not in generate_kwargs:
-            raise ValueError(
-                "'threat_model' kwarg must be defined in attack config's"
-                "'generate_kwargs' as one of ('physical', 'digital')"
-            )
-        elif generate_kwargs["threat_model"].lower() not in ("physical", "digital"):
-            raise ValueError(
-                f"'threat_model must be set to one of ('physical', 'digital'), not {generate_kwargs['threat_model']}."
-            )
-
-        else:
-            threat_model = generate_kwargs["threat_model"].lower()
-
+    def generate(self, x, y_object=None, y_patch_metadata=None, threat_model="digital"):
         num_imgs = x.shape[0]
         attacked_images = []
 
@@ -140,7 +133,7 @@ def generate(self, x, y_object=None, y_patch_metadata=None, **generate_kwargs):
                     x[i], y_object[i]["area"], gs_coords, shape
                 )
                 img_with_patch = super().generate(
-                    np.expand_dims(x[i], axis=0), mask=img_mask
+                    np.expand_dims(x[i], axis=0), y=[y_object[i]], mask=img_mask
                 )[0]
                 attacked_images.append(img_with_patch)
         else:
@@ -153,7 +146,7 @@ def generate(self, x, y_object=None, y_patch_metadata=None, **generate_kwargs):
     def _compute_image_mask(self, x, gs_area, gs_coords, shape):
         gs_size = int(np.sqrt(gs_area))
         patch_coords = shape_coords(gs_size, gs_size, shape)
-        h, status = cv2.findHomography(patch_coords, gs_coords)
+        h, _ = cv2.findHomography(patch_coords, gs_coords)
         inscribed_patch_mask = create_mask(shape, gs_size, gs_size)
         img_mask = cv2.warpPerspective(
             inscribed_patch_mask, h, (x.shape[1], x.shape[0]), cv2.INTER_CUBIC

armory/art_experimental/attacks/poison_loader.py

@@ -1,9 +1,11 @@
 """
 This module enables loading of different perturbation functions in poisoning
 """
+import os
 
 from art.attacks.poisoning import PoisoningAttackBackdoor
 from art.attacks.poisoning import perturbations
+import armory
 
 
 def poison_loader_GTSRB(**kwargs):
@@ -24,13 +26,39 @@ def mod(x):
             raise ValueError(
                 "poison_type 'image' requires 'backdoor_path' kwarg path to image"
             )
+        backdoor_packaged_with_armory = kwargs.get(
+            "backdoor_packaged_with_armory", False
+        )
+        if backdoor_packaged_with_armory:
+            backdoor_path = os.path.join(
+                # Get base directory where armory is pip installed
+                os.path.dirname(os.path.dirname(armory.__file__)),
+                backdoor_path,
+            )
         size = kwargs.get("size")
         if size is None:
             raise ValueError("poison_type 'image' requires 'size' kwarg tuple")
         size = tuple(size)
+        mode = kwargs.get("mode", "RGB")
+        blend = kwargs.get("blend", 0.6)
+        base_img_size_x = kwargs.get("base_img_size_x", 48)
+        base_img_size_y = kwargs.get("base_img_size_y", 48)
+        channels_first = kwargs.get("channels_first", False)
+        x_shift = kwargs.get("x_shift", (base_img_size_x - size[0]) // 2)
+        y_shift = kwargs.get("y_shift", (base_img_size_y - size[1]) // 2)
 
         def mod(x):
-            return perturbations.insert_image(x, backdoor_path=backdoor_path, size=size)
+            return perturbations.insert_image(
+                x,
+                backdoor_path=backdoor_path,
+                size=size,
+                mode=mode,
+                x_shift=x_shift,
+                y_shift=y_shift,
+                channels_first=channels_first,
+                blend=blend,
+                random=False,
+            )
 
     else:
         raise ValueError(f"Unknown poison_type {poison_type}")

armory/art_experimental/defences/jpeg_compression_normalized.py

@@ -12,7 +12,6 @@ def __init__(
         self,
         clip_values,
         quality=50,
-        channel_index=3,
         apply_fit=True,
         apply_predict=False,
         means=None,
@@ -22,7 +21,6 @@ def __init__(
         super().__init__(
             clip_values,
             quality=quality,
-            channel_index=channel_index,
             apply_fit=apply_fit,
             apply_predict=apply_predict,
         )