[Filebeat] fix organization and customer prefix for aws/cloudtrail (e…

…lastic#23126)

* fix organization and customer prefix for aws/cloudtrail

- use ^.*AWSLogs as beginning of prefix, matches:
  + custom-prefix/AWSLogs/1234567890/CloudTrail/
  + o-xxxxxxx/AWSLogs/1234567890/CloudTrail/
  + AWSLogs/1234567890/CloudTrail/

Closes elastic#23109

Co-authored-by: Andrew Kroh <andrew.kroh@elastic.co>

CHANGELOG.next.asciidoc

@@ -351,6 +351,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
 - Fix Cisco ASA/FTD module's parsing of WebVPN log message 716002. {pull}22966[22966]
 - Fix aws s3 overview dashboard. {pull}23045[23045]
 - Fix bad `network.direction` values in Fortinet/firewall fileset. {pull}23072[23072]
+- Add support for organization and custom prefix in AWS/CloudTrail fileset. {issue}23109[23109] {pull}23126[23126]
 
 *Heartbeat*
 
@@ -968,4 +969,3 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
 
 
 
-

x-pack/filebeat/module/aws/cloudtrail/config/s3.yml

@@ -2,16 +2,16 @@ type: s3
 queue_url: {{ .queue_url }}
 file_selectors:
 {{ if .process_cloudtrail_logs }}
-  - regex: '^AWSLogs/\d+/CloudTrail/'
+  - regex: 'AWSLogs/\d+/CloudTrail/'
     expand_event_list_from_field: 'Records'
 {{ end }}
 
 {{ if .process_digest_logs }}
-  - regex: '^AWSLogs/\d+/CloudTrail-Digest/'
+  - regex: 'AWSLogs/\d+/CloudTrail-Digest/'
 {{ end }}
 
 {{ if .process_insight_logs }}
-  - regex: '^AWSLogs/\d+/CloudTrail-Insight/'
+  - regex: 'AWSLogs/\d+/CloudTrail-Insight/'
     expand_event_list_from_field: 'Records'
 {{ end }}