Interrupt altool's CFNetworking retry loop

[priitlatt]

Sometimes app-store-connect publish action gets stuck while binary is being uploaded using altool. From verbose altool logs it can be seen that uploading fails because specified hostname could not be found, and then internally altool just keeps on retrying the operation indefinitely.

Example of altool verbose log output

2024-03-13 14:26:15.227 DEBUG: [ContentDelivery.Uploader] Wrote part 1 to temp file '/var/folders/w2/rrf5p87d1bbfyphxc7jdnyvh0000gn/T/com.apple.cds.vbtx/com.apple.cds_6A4F9402-15C4-49DA-8C2C-57950EFCD52F/Part-1.tmp'.
2024-03-13 14:26:15.227 DEBUG: [ContentDelivery.Uploader] Adding upload task for part 1.
2024-03-13 14:26:15.228 DEBUG: [ContentDelivery.Uploader] Upload tasks created. Waiting for them to finish uploading.
2024-03-13 14:26:15.236 DEBUG: [ContentDelivery.Uploader] LOST 0 bytes for part 1.
2024-03-13 14:26:15.236 DEBUG: [ContentDelivery.Uploader] Adding upload task 3 for part 1.
2024-03-13 14:26:15.236 *** Error: ERROR: [ContentDelivery.Uploader] WILL RETRY PART 1. Failed with error: Error Domain=NSURLErrorDomain Code=-1003 "A server with the specified hostname could not be found." UserInfo={_kCFStreamErrorCodeKey=8, NSUnderlyingError=0x6000039494a0 {Error Domain=kCFErrorDomainCFNetwork Code=-1003 "(null)" UserInfo={_kCFStreamErrorDomainKey=12, _kCFStreamErrorCodeKey=8, _NSURLErrorNWResolutionReportKey=Resolved 0 endpoints in 6ms using unknown from cache, _NSURLErrorNWPathKey=satisfied (Path is satisfied), interface: en0, ipv4, dns}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalUploadTask <89821E5C-11AB-4F69-B1CB-69AAC3D6FBB8>.<2>, _NSURLErrorRelatedURLSessionTaskErrorKey=(
    "LocalUploadTask <89821E5C-11AB-4F69-B1CB-69AAC3D6FBB8>.<2>"
), NSLocalizedDescription=A server with the specified hostname could not be found., NSErrorFailingURLStringKey=https://northamerica-1.object-storage.apple.com/itmspod12-assets-massilia-200001/PurpleSource221%2Fv4%2F9c%2F70%2F3c%2F9c703cfe-477c-c38d-a274-508c2ee1a2b2%2F7FnMpSoadCNA8WVLq8VxeS_wSsKqcEDWTeNWS3RMilg_U003d-1710339974733?uploadId=a5eba160-e145-11ee-a95f-5a73240ad8a5&Signature=X6I5Qk04%2Ba4RXRIQJW3%2B6MMUuXk%3D&AWSAccessKeyId=MKIAP7F9QNTEY48OTE7F&partNumber=1&Expires=1710944775, NSErrorFailingURLKey=https://northamerica-1.object-storage.apple.com/itmspod12-assets-massilia-200001/PurpleSource221%2Fv4%2F9c%2F70%2F3c%2F9c703cfe-477c-c38d-a274-508c2ee1a2b2%2F7FnMpSoadCNA8WVLq8VxeS_wSsKqcEDWTeNWS3RMilg_U003d-1710339974733?uploadId=a5eba160-e145-11ee-a95f-5a73240ad8a5&Signature=X6I5Qk04%2Ba4RXRIQJW3%2B6MMUuXk%3D&AWSAccessKeyId=MKIAP7F9QNTEY48OTE7F&partNumber=1&Expires=1710944775, _kCFStreamErrorDomainKey=12}
2024-03-13 14:26:15.237 DEBUG: [ContentDelivery.Uploader] Created new upload task (0x13264e8f0) for part 1.
2024-03-13 14:26:15.237 DEBUG: [ContentDelivery.Uploader] Saving uploader state (CDUploaderStateUploadAssetDescription) for identifier 'com.apple.cds_6A4F9402-15C4-49DA-8C2C-57950EFCD52F'.
2024-03-13 14:26:15.239 DEBUG: [ContentDelivery.Uploader] There is one part remaining to upload.
2024-03-13 14:26:15.240 DEBUG: [ContentDelivery.Uploader] LOST 0 bytes for part 1.
2024-03-13 14:26:15.240 DEBUG: [ContentDelivery.Uploader] Adding upload task 4 for part 1.
2024-03-13 14:26:15.240 *** Error: ERROR: [ContentDelivery.Uploader] WILL RETRY PART 1. Failed with error: Error Domain=NSURLErrorDomain Code=-1003 "A server with the specified hostname could not be found." UserInfo={_kCFStreamErrorCodeKey=8, NSUnderlyingError=0x600003949dd0 {Error Domain=kCFErrorDomainCFNetwork Code=-1003 "(null)" UserInfo={_kCFStreamErrorDomainKey=12, _kCFStreamErrorCodeKey=8, _NSURLErrorNWResolutionReportKey=Resolved 0 endpoints in 0ms using unknown from cache, _NSURLErrorNWPathKey=satisfied (Path is satisfied), interface: en0, ipv4, dns}}, _NSURLErrorFailingURLSessionTaskErrorKey=LocalUploadTask <88A22F39-3A2A-40A0-8EA2-80DB60C997E3>.<3>, _NSURLErrorRelatedURLSessionTaskErrorKey=(
    "LocalUploadTask <88A22F39-3A2A-40A0-8EA2-80DB60C997E3>.<3>"
...

As verbose log output is not guaranteed we cannot rely on this output to detect whether the upload has gotten stuck or not. Instead add a background system log checker which looks for similar entries for altool process starting from the moment when the subprocess was launched.

In case the logs indicate that altool uploading might be stuck, then the same retry logic is applied as was already introduced for failed uploads in #163.

Updated actions:

• app-store-connect publish

[mikhail-tokarev]

LGTM