Audio echo

[davidslater]

Fixes #1016

I'm running into issues in a couple of places outside my code:

1. When running on art 1.6.0, I can't instantiate the PyTorchDeepSpeech class:

>>> from art.estimators.speech_recognition import PyTorchDeepSpeech
>>> PyTorchDeepSpeech()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Can't instantiate abstract class PyTorchDeepSpeech with abstract methods compute_loss
>>> import art
>>> art.__version__
'1.6.0'

2. When running on art 1.5.3, I run into this error when the attack is run (benign works fine):

Attack:   0%|                                                                                                                                                  | 0/1 [00:00<?, ?it/s]
2021-03-25 00:34:53 a4b546f3d365 armory.scenarios.base[2547] ERROR Encountered error during scenario evaluation.
Traceback (most recent call last):
  File "/workspace/armory/scenarios/base.py", line 84, in evaluate
    config, num_eval_batches, skip_benign, skip_attack, skip_misclassified
  File "/workspace/armory/scenarios/audio_asr.py", line 236, in _evaluate
    x_adv = attack.generate(x=x, y=y_target)
  File "/workspace/art/attacks/attack.py", line 74, in replacement_function
    return fdict[func_name](self, *args, **kwargs)
  File "/workspace/art/attacks/evasion/fast_gradient.py", line 279, in generate
    adv_x_best = self._compute(x, x, y, None, self.eps, self.eps, self._project, self.num_random_init > 0,)
  File "/workspace/art/attacks/evasion/fast_gradient.py", line 431, in _compute
    perturbation = self._compute_perturbation(batch, batch_labels, mask_batch)
  File "/workspace/art/attacks/evasion/fast_gradient.py", line 342, in _compute_perturbation
    grad = self.estimator.loss_gradient(batch, batch_labels) * (1 - 2 * int(self.targeted))
  File "/workspace/art/estimators/speech_recognition/pytorch_deep_speech.py", line 370, in loss_gradient
    from warpctc_pytorch import CTCLoss
  File "/opt/conda/lib/python3.7/site-packages/warpctc_pytorch/__init__.py", line 6, in <module>
    from ._warp_ctc import *  # noqa
ImportError: libcudart.so.10.0: cannot open shared object file: No such file or directory

3. The ASR model seems to ignore the preprocessing and preprocessing_defenses, so the channel never gets called. (This is running on 1.5.3).

[davidslater]

I think I solved the preprocessing_defences thing:

classifier._update_preprocessing_operations() was needed - I think this is a broader bug.

[lcadalzo]

@davidslater I assume the ImportError: libcudart.so.10.0: cannot open shared object file: No such file or directory error still persists for you? It looks like warp_ctc requires cuda 10.* (or less), so I think we might need to revert the pytorch-deepspeech image to cuda 10.

The available versions of warp_ctc for PyTorch 1.6: 0.2.1+torch16.cpu, 0.2.1+torch16.cuda100, 0.2.1+torch16.cuda101, 0.2.1+torch16.cuda102, 0.2.1+torch16.cuda92

[lcadalzo]

If we want to get past TypeError: Can't instantiate abstract class PyTorchDeepSpeech with abstract methods compute_loss without waiting for an ART patch, we could modify the baseline model to something like:

class PyTorchDeepSpeechModel(PyTorchDeepSpeech):
    def __init__(self, **kwargs):
        super().__init__(**kwargs)

    def compute_loss(self):
        raise NotImplementedError


def get_art_model(
    model_kwargs: dict, wrapper_kwargs: dict, weights_path: Optional[str] = None
) -> PyTorchDeepSpeech:
    return PyTorchDeepSpeechModel(**wrapper_kwargs)

[lcadalzo]

Are some of the logger.error() calls there for debugging purposes? Should they be removed?

[lcadalzo]

LGTM