You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
When processing files larger than 2GB, the system throws an error indicating that the numeric value is out of range for an integer.
To Reproduce
Steps to reproduce the behavior:
Upload a file larger than 2GB to the specified S3 bucket.
Ensure that the S3EventBridgeNotification is configured to trigger when a new file is uploaded.
Once the Data Prepper pipeline is triggered, the error shows up in the logs.
Expected behavior
The system should handle files larger than 2GB without throwing an error.
Additional context
404 [s3-source-sqs-1] ERROR org.opensearch.dataprepper.plugins.source.s3.parser.S3EventBridgeNotificationParser - SQS message with message ID:8947a26b-2500-4859-bfd1-05353e0fc232 has invalid body which cannot be parsed into EventBridgeNotification. Numeric value (2409161669) out of range of int (-2147483648 - 2147483647)
at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 523] (through reference chain: org.opensearch.dataprepper.plugins.source.s3.S3EventBridgeNotification["detail"]->org.opensearch.dataprepper.plugins.source.s3.S3EventBridgeNotification$Detail["object"]->org.opensearch.dataprepper.plugins.source.s3.S3EventBridgeNotification$Object["size"])
The text was updated successfully, but these errors were encountered:
Describe the bug
When processing files larger than 2GB, the system throws an error indicating that the numeric value is out of range for an integer.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
The system should handle files larger than 2GB without throwing an error.
Additional context
The text was updated successfully, but these errors were encountered: