Ensure callback is called once in doWhilst with S3 operations

[williamlardier]

The trigger seem to be
aws/aws-sdk-js#1678
... causing the callback to be called twice, which is a problem in these loops.

Issue: ZENKO-4925

Rationale of the change in the ticket comments.

[bert-e]

Hello williamlardier,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
/after_pull_request	Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval	Bypass the pull request author's approval	⭐
/bypass_build_status	Bypass the build and test status	⭐
/bypass_commit_size	Bypass the check on the size of the changeset TBA	⭐
/bypass_incompatible_branch	Bypass the check on the source branch prefix	⭐
/bypass_jira_check	Bypass the Jira issue check	⭐
/bypass_peer_approval	Bypass the pull request peers' approval	⭐
/bypass_leader_approval	Bypass the pull request leaders' approval	⭐
/approve	Instruct Bert-E that the author has approved the pull request.		✍️
/create_pull_requests	Allow the creation of integration pull requests.
/create_integration_branches	Allow the creation of integration branches.
/no_octopus	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity	Change review acceptance criteria from one reviewer at least to all reviewers
/wait	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
/help	Print Bert-E's manual in the pull request.
/status	Print Bert-E's current status in the pull request TBA
/clear	Remove all comments from Bert-E from the history TBA
/retry	Re-start a fresh build TBA
/build	Re-start a fresh build TBA
/force_reset	Delete integration branches & pull requests, and restart merge process from the beginning.
/reset	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

[bert-e]

Incorrect fix version

The Fix Version/s in issue ZENKO-4925 contains:

• 2.10.6

• 2.6.69

• 2.7.65

• 2.8.45

• 2.9.23

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

• 2.10.7

• 2.6.70

• 2.7.66

• 2.8.46

• 2.9.24

Please check the Fix Version/s of ZENKO-4925, or the target
branch of this pull request.

[francoisferrand]

if I understand correctly, this would happen if/because the headObject callback is invoked twice : both for the actual response and a socket timeout which happens at about the same time.

→ it seems to me a simpler fix would be just:

 callback => this.s3.headObject({
                Bucket: bucketName,
                Key: key,
                VersionId: versionId,
            }, jsutil.once((err, data) => {
                [...]
           })

However, I am wondering if there may not be a real underlying issue : like the socket timeout (on client side, i.e. test) being shorter than the timeout we have in zenko.... and maybe this could explain the (other) issues we have with these tests, what do you think?

Can we check the timeout (or infirm this hypothesis), and/or should we ignore retryable AWS errors (retry after a timeout) ?

[williamlardier]

I am not sure it was a timeout but worth reviewing that indeed, let me check

[williamlardier]

The current client-side timeout seem to be infinite, so I don't think we are affected by the timeout from client side:

const scalityS3Client = new S3({
    ...
    // disable node sdk retries and timeout to prevent InvalidPart
    // and SocketHangUp errors. If retries are allowed, sdk will send
    // another request after first request has already deleted parts,
    // causing InvalidPart. Meanwhile, if request takes too long to finish,
    // sdk will create SocketHangUp error before response.
    maxRetries: 0,
    httpOptions: { timeout: 0 },
});

[williamlardier]

Will be merged with the upcoming Zenko release - keeping this open till then.