Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File upload failed even after retry #1916

Open
3 tasks
jeffschwMSFT opened this issue Jan 30, 2024 · 2 comments
Open
3 tasks

File upload failed even after retry #1916

jeffschwMSFT opened this issue Jan 30, 2024 · 2 comments

Comments

@jeffschwMSFT
Copy link
Member

jeffschwMSFT commented Jan 30, 2024

Build

https://dev.azure.com/dnceng/internal/_build/results?buildId=2365835

Error:

Fail to upload '/mnt/vss/_work/1/a/IntermediateArtifacts/MonoRuntimePacks/Shipping/Microsoft.NETCore.App.Ref.9.0.0-preview.2.24080.4.symbols.nupkg' due to 'Bad Request'.
Microsoft.VisualStudio.Services.WebApi.VssServiceResponseException: Bad Request
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.HandleResponseAsync(HttpResponseMessage response, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.FileContainer.Client.FileContainerHttpClient.UploadFileAsync(Int64 containerId, String itemPath, Stream fileStream, Byte[] contentId, Int64 fileLength, Boolean isGzipped, Guid scopeIdentifier, CancellationToken cancellationToken, Int32 chunkSize, Int32 chunkRetryTimes, Boolean uploadFirstChunk, Object userState)
   at Microsoft.VisualStudio.Services.FileContainer.Client.FileContainerHttpClient.UploadFileAsync(Int64 containerId, String itemPath, Stream fileStream, Guid scopeIdentifier, CancellationToken cancellationToken, Int32 chunkSize, Boolean uploadFirstChunk, Object userState, Boolean compressStream)
   at Microsoft.VisualStudio.Services.Agent.Worker.Build.FileContainerServer.UploadAsync(IAsyncCommandContext context, Int32 uploaderId, CancellationToken token) in /home/vsts/work/1/s/src/Agent.Worker/Build/FileContainerServer.cs:line 250
Detail upload trace for file that fail to upload: IntermediateArtifacts/MonoRuntimePacks/Shipping/Microsoft.NETCore.App.Ref.9.0.0-preview.2.24080.4.symbols.nupkg
Chunk '2' attempt '1' of file 'IntermediateArtifacts/MonoRuntimePacks/Shipping/Microsoft.NETCore.App.Ref.9.0.0-preview.2.24080.4.symbols.nupkg' fail to send request to server. Error: Microsoft.VisualStudio.Services.Common.VssServiceException: TF203009: The size of the file that you uploaded does not match the size that the client computer provided.
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.HandleResponseAsync(HttpResponseMessage response, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.FileContainer.Client.FileContainerHttpClient.UploadFileAsync(Int64 containerId, String itemPath, Stream fileStream, Byte[] contentId, Int64 fileLength, Boolean isGzipped, Guid scopeIdentifier, CancellationToken cancellationToken, Int32 chunkSize, Int32 chunkRetryTimes, Boolean uploadFirstChunk, Object userState).
##[error]File upload failed even after retry.

Build leg reported

No response

Pull Request

No response

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "##[error]File upload failed even after retry",
    "BuildRetry": false,
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

  • Feature changes/additions
  • Bug fixes
  • Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Report

Build Definition Step Name Console log Pull Request
2604790 dotnet-runtime 🔒 Publish intermediate artifacts Log
2603532 dotnet-winforms Pack, Sign, and Publish Log
2600776 dotnet-winforms Pack, Sign, and Publish Log
2599830 dotnet-runtime 🔒 Publish intermediate artifacts Log
2594286 dotnet-runtime 🔒 Publish intermediate artifacts Log
2594212 dotnet-runtime Publish Logs Log #45620
2590694 dotnet-winforms Pack, Sign, and Publish Log
2589613 dotnet-runtime 🔒 Publish intermediate artifacts Log
2587197 dotnet-winforms Pack, Sign, and Publish Log

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
0 2 9

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng/internal/_build/results?buildId=2365835
Error message validated: [##[error]File upload failed even after retry]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 2/7/2024 2:03:27 AM UTC

@lewing
Copy link
Member

lewing commented Feb 21, 2024

This is often an from multiple lanes attempting to upload the same artifact

@markwilkie
Copy link
Member

@lewing is right - there's usually not a whole lot dnceng can do to help.

akoeplinger added a commit to dotnet/runtime that referenced this issue Nov 9, 2024
)

We're occasionally seeing errors on the official build during artifacts upload (dotnet/dnceng#1916)
This is because multiple jobs are trying to push the Microsoft.NETCore.App.Ref nuget package to the same AzDO build artifacts container, causing conflicts.

Since we'll be using the VMR in the near future and it is using a different asset selection the quick fix is to just delete the package on the jobs where it's not supposed to be uploaded rather than trying to introduce a complicated condition in msbuild.
mikelle-rogers pushed a commit to mikelle-rogers/runtime that referenced this issue Dec 10, 2024
…et#109650)

We're occasionally seeing errors on the official build during artifacts upload (dotnet/dnceng#1916)
This is because multiple jobs are trying to push the Microsoft.NETCore.App.Ref nuget package to the same AzDO build artifacts container, causing conflicts.

Since we'll be using the VMR in the near future and it is using a different asset selection the quick fix is to just delete the package on the jobs where it's not supposed to be uploaded rather than trying to introduce a complicated condition in msbuild.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants