Generic Masked Patch Attack #1904
Conversation
|
|
||
|
|
||
| @dataclass | ||
| class PatchMask: |
There was a problem hiding this comment.
I'm not sure if this file is the correct place for this class since we do not have a dedicated 'attack utils' file. Should this be moved to its own file?
There was a problem hiding this comment.
Should be moved to be within armory.utils
|
@christopherwoodall Still WIP, not worth you trying to debug the current issue with patch_base_image but feel free to address other comments; will request review by EOW with fixes |
…utils. Split armory cli tools into separate files.
…t for patch shape kwargs; working attack
|
@mwartell I am running into python package issues when installed not in editable mode but it works as expected when it is. Specifically armory cannot find |
|
The culprit is an overzealous exclusion in pyproject.toml I'm looking for the syntax to fix that, likely Hatch claims to use the gitignore pattern rules which is a good choice.
Therefore will exclude |
armory/cli/tools/generate_shapes.py
Outdated
| def generate_shapes(command_args, prog, description): | ||
| from armory.utils.shape_gen import ( # move to lazy load to avoid PIL import error | ||
| Shape, | ||
| ) |
There was a problem hiding this comment.
Is there a better way to handle this? In particular, Pillow is listed under engine and develop but when you install armory-testbed standard it is not included and will error when trying to run armory --help. This fixes that problem but creates another that if generate_shapes is called at runtime from an environment without PIL the error message will not be useful. However since this is strictly code that is called from either the cli in which the user will know how to fix the issue or from the carla_adversarial_patch_pytorch code which must be running with engine installed and PIL must also be installed I left it this way.
There was a problem hiding this comment.
We had wanted to make the armory launcher clean so that the host could have a thin (no ML framework install) loader that relied on the contents of the container for the heavy libraries). This was a big motivation for keeping the dependencies lean. Since Armory v23.x has dropped containers and the two-stage armory launch, and with Armory v0.x.x having an informal end-of-life in under 18 months, I think we just don't care that much any more.
Do you need PIL in pyproject dependencies? Then put it there, it isn't a big deal anymore. @christopherwoodall care to render an opinion?
There was a problem hiding this comment.
Totally in agreement that all required libraries should be in the dependencies section.
armory/cli/tools/generate_shapes.py
Outdated
| def generate_shapes(command_args, prog, description): | ||
| from armory.utils.shape_gen import ( # move to lazy load to avoid PIL import error | ||
| Shape, | ||
| ) |
There was a problem hiding this comment.
Totally in agreement that all required libraries should be in the dependencies section.
jprokos26
changed the title
The goal of this PR is to implement a generic masked patch module to be integrated into existing attacks with minimal code change to the attack module
How to use this code
See
To illustrate what this does see the example below:
Example Run
Attack kwargs from config
"attack": { "knowledge": "white", "kwargs": { "batch_size": 1, "learning_rate": 0.1, "max_iter": 2, "optimizer": "pgd", "targeted": false, "verbose": true, "patch_base_image": "https://github.com/jprokos26/armory/blob/public-base-images/armory/art_experimental/attacks/base_images/%5BCARLA_OD%5D_stable_diffusion_gemstones.png?raw=true", "patch_mask": { "shape": "grid", "shape_kwargs": { "num_circles": 5, "circle_radius": 0.1, "circle_spacing": 2 }, "invert": false, "fill": "https://github.com/jprokos26/armory/blob/public-base-images/armory/art_experimental/attacks/base_images/white_w_black_text.png?raw=true" } }, "module": "armory.art_experimental.attacks.carla_obj_det_adversarial_patch", "name": "CARLAAdversarialPatchPyTorch", "use_label": true },Resultant image