Bug: Athena Partition Refresh KeyError on SQS Delete

[jacknagz]

Background

The Athena Partition Refresh Lambda function is responsible for creating Hive Partitions after Firehose writes data to S3. This works by emitting S3 Event Notifications to a SQS Queue, and consuming the messages via Lambda.

Description

When deleting messages off the Athena SQS Queue, the StreamAlertSQSClient.delete_messages method first checks for successfully deleted messages prior to looking for failures. This results in a KeyError in the latter scenario:

'Successful': KeyError
Traceback (most recent call last):
File "/var/task/stream_alert/athena_partition_refresh/main.py", line 607, in handler
stream_alert_sqs.delete_messages()
File "/var/task/stream_alert/athena_partition_refresh/main.py", line 513, in delete_messages
_delete_messages_from_queue()
File "/var/task/backoff/_sync.py", line 42, in retry
ret = target(*args, **kwargs)
File "/var/task/stream_alert/athena_partition_refresh/main.py", line 500, in _delete_messages_from_queue
self.deleted_messages += len(resp['Successful'])
KeyError: 'Successful'

Desired Change

Conditional checks for Successful/Failed response keys.