GenerateDepsFile: The process cannot access the file '...\MyProject.deps.json' because it is being used by another process. #2902

LordMike · 2019-02-03T21:46:46Z

UPDATE: While writing this, rather long, post -- I've managed to produce a small reproduction example, see bottom

We've recently upgraded our builds to run on faster hardware with more cpu cores, and are now ~100% of the time seeing the below error in a number of repositories. The error here is from Windows, but our CI servers run on linux and have the same error (see logs below).

We run the following commandlines:

"C:\Program Files\dotnet\dotnet.exe" restore C:\Project\MyProject.sln
"C:\Program Files\dotnet\dotnet.exe" build C:\Project\MyProject.sln --configuration Debug --framework netstandard2.0 --no-restore

C:\Program Files\dotnet\sdk\2.2.101\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(129,5): error MSB4018

The "GenerateDepsFile" task failed unexpectedly. [C:\Project\MyProject.csproj]
System.IO.IOException: The process cannot access the file 'C:\Project\bin\Debug\netstandard2.0\MyProject.deps.json' because it is being used by another process. [C:\Project\MyProject.csproj]
   at System.IO.FileStream.ValidateFileHandle(SafeFileHandle fileHandle) [C:\Project\MyProject.csproj]
   at System.IO.FileStream.CreateFileOpenHandle(FileMode mode, FileShare share, FileOptions options) [C:\Project\MyProject.csproj]
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options) [C:\Project\MyProject.csproj]
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize) [C:\Project\MyProject.csproj]
   at System.IO.File.Create(String path) [C:\Project\MyProject.csproj]
   at Microsoft.NET.Build.Tasks.GenerateDepsFile.ExecuteCore() [C:\Project\MyProject.csproj]
   at Microsoft.NET.Build.Tasks.TaskBase.Execute() [C:\Project\MyProject.csproj]
   at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [C:\Project\MyProject.csproj]
   at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [C:\Project\MyProject.csproj]

Observations:

This fails on my local Windows desktop roughly 33% of the time. Rerunning the build (no cleanup) always succeeds (seems to be a race condition, so HW speeds play a role). On our Linux CI servers, it fails 100% of the time currently
The deps file in question is always the same file for a given repository / builds. The file is for a shared library between multiple projects.
A plain dotnet build MyProject.sln -f netstandard2.0 (no build script) also exhibits this issue.
A dotnet build MyProject.sln seemingly always succeeds (locally and on CI server), at least for three consecutive attempts right now.
All the projects in the MyProject.sln have netstandard2.0 - but a number of them have more targets (ie. netcoreapp)
We have more projects, f.ex. for tests, but they're in MyProject-Tests.sln. These projects only target netcoreapp2.x - we've split the solutions both for speed when developing and to be able to build artifacts without compiling the tests (since we can't build certain projects from the solution only ... that's another issue entirely) (not relevant)
Limiting the build to one process (msbuild: /m:1) seems to "solve" the problem.
At work (inaccessible currently), I performed some Procmon captures to see the difference between successful and failing runs
- It turns out, that two dotnet.exe processes attempt to create the .deps.json files
- When succeeding, there is a clear seperation between two CreateFile() calls, where the first creates the file, and the second call (from the second PID) finds the file already there and does nothing (not even reads it)
- When failing, the two processes make calls overlapping each other, leading to both of them discovering the file as missing, and both trying to create it (one obviously failing, producing the stacktrace above) -- classical concurrency issue.

So.

It should be that when two projects reference a third, shared, project .. that this shared one is built exactly once .. right?.. How come two attempts are made at writing the same .deps.json files?
How come this seemingly works fine, when building for all target frameworks?

Versions:

Local, Windows 10 x64, dotnet 2.2.101
Linux CI, docker, dotnet 2.2.103 (we build our own docker images using the dockerfile from the base microsoft/dotnet:2-sdk images)

Reproduction

Project: ci-stresstest-master.zip

In it, I have a solution with 5 projects, all empty (no actual code), but with a targeting setup that mimicks a very small subset of our projects. I've then created four seperate CI builds that did the following:

log (succeeds) dotnet restore ci-stresstest.sln + dotnet build ci-stresstest.sln --no-restore
log (succeeds) dotnet build ci-stresstest.sln
log (fails) dotnet restore ci-stresstest.sln + dotnet build ci-stresstest.sln -f netstandard2.0 --no-restore
log (fails) dotnet build ci-stresstest.sln -f netstandard2.0

Notes on repro.

We run a vanilla docker setup for CI, except for one thing:
- We mount a shared directory for nuget packages, to be able to share downloaded packages between builds

The process cannot access the file '...\Shared02.dll' because it is being used by another process.

Cleaning the workspace (git clean -xdf) between builds produces this roughly once every two builds.

Log

PS C:\Users\MichaelBisbjerg\Git\ci-stresstest> dotnet build -f netstandard2.0
Microsoft (R) Build Engine version 15.9.20+g88f5fadfbe for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Restoring packages for C:\Users\MichaelBisbjerg\Git\ci-stresstest\Program01\Program01.csproj...
  Restoring packages for C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\TopLib01.csproj...
  Restoring packages for C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\Shared02.csproj...
  Restoring packages for C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\Shared01.csproj...
  Restoring packages for C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib02\TopLib02.csproj...
  Generating MSBuild file C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib02\obj\TopLib02.csproj.nuget.g.props.
  Generating MSBuild file C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\obj\TopLib01.csproj.nuget.g.props.
  Generating MSBuild file C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\obj\Shared01.csproj.nuget.g.props.
  Generating MSBuild file C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\obj\Shared02.csproj.nuget.g.props.
  Generating MSBuild file C:\Users\MichaelBisbjerg\Git\ci-stresstest\Program01\obj\Program01.csproj.nuget.g.props.
  Generating MSBuild file C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\obj\Shared01.csproj.nuget.g.targets.
  Generating MSBuild file C:\Users\MichaelBisbjerg\Git\ci-stresstest\Program01\obj\Program01.csproj.nuget.g.targets.
  Generating MSBuild file C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\obj\Shared02.csproj.nuget.g.targets.
  Generating MSBuild file C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib02\obj\TopLib02.csproj.nuget.g.targets.
  Generating MSBuild file C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\obj\TopLib01.csproj.nuget.g.targets.
  Restore completed in 128,32 ms for C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib02\TopLib02.csproj.
  Restore completed in 128,32 ms for C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\TopLib01.csproj.
  Restore completed in 128,32 ms for C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\Shared02.csproj.
  Restore completed in 128,32 ms for C:\Users\MichaelBisbjerg\Git\ci-stresstest\Program01\Program01.csproj.
  Restore completed in 128,32 ms for C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\Shared01.csproj.
  Shared01 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\bin\Debug\netstandard2.0\Shared01.dll
CSC : error CS2012: Cannot open 'C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\obj\Debug\netstandard2.0\Shared02.dll' for writing -- 'The process cannot access the file 'C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\obj\Debug\netstandard2.0\Shared02.dll' because it is being used by another process.' [C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\Shared02.csproj]
CSC : error CS2012: Cannot open 'C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\obj\Debug\netstandard2.0\Shared01.dll' for writing -- 'The process cannot access the file 'C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\obj\Debug\netstandard2.0\Shared01.dll' because it is being used by another process.' [C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\Shared01.csproj]
  Shared02 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\bin\Debug\netstandard2.0\Shared02.dll
  TopLib02 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib02\bin\Debug\netstandard2.0\TopLib02.dll
  TopLib01 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\bin\Debug\netstandard2.0\TopLib01.dll
  TopLib01 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\bin\Debug\netstandard2.0\TopLib01.dll
  Program01 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\Program01\bin\Debug\netstandard2.0\Program01.dll

Build FAILED.

CSC : error CS2012: Cannot open 'C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\obj\Debug\netstandard2.0\Shared02.dll' for writing -- 'The process cannot access the file 'C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\obj\Debug\netstandard2.0\Shared02.dll' because it is being used by another process.' [C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\Shared02.csproj]
CSC : error CS2012: Cannot open 'C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\obj\Debug\netstandard2.0\Shared01.dll' for writing -- 'The process cannot access the file 'C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\obj\Debug\netstandard2.0\Shared01.dll' because it is being used by another process.' [C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\Shared01.csproj]
    0 Warning(s)
    2 Error(s)

Time Elapsed 00:00:01.30

Could not read state file "obj\..\TopLib01.csprojAssemblyReference.cache". The process cannot access the file '..\TopLib01.csprojAssemblyReference.cache' because it is being used by another process.

Building the solution without cleanup between builds, this happened 8 out of 17 times I built it.

Log

PS C:\Users\MichaelBisbjerg\Git\ci-stresstest> dotnet build -f netstandard2.0
Microsoft (R) Build Engine version 15.9.20+g88f5fadfbe for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Restore completed in 25,24 ms for C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib02\TopLib02.csproj.
  Restore completed in 25,22 ms for C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\Shared02.csproj.
  Restore completed in 25,19 ms for C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\Shared01.csproj.
  Restore completed in 25,24 ms for C:\Users\MichaelBisbjerg\Git\ci-stresstest\Program01\Program01.csproj.
  Restore completed in 25,19 ms for C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\TopLib01.csproj.
  Shared02 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\bin\Debug\netstandard2.0\Shared02.dll
  Shared02 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared02\bin\Debug\netstandard2.0\Shared02.dll
  Shared01 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\bin\Debug\netstandard2.0\Shared01.dll
C:\Program Files\dotnet\sdk\2.2.101\Microsoft.Common.CurrentVersion.targets(2110,5): warning MSB3088: Could not read state file "obj\Debug\netstandard2.0\TopLib01.csprojAssemblyReference.cache". The process cannot access the file 'C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\obj\Debug\netstandard2.0\TopLib01.csprojAssemblyReference.cache' because it is being used by another process. [C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\TopLib01.csproj]
  TopLib01 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\bin\Debug\netstandard2.0\TopLib01.dll
  TopLib02 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib02\bin\Debug\netstandard2.0\TopLib02.dll
  TopLib01 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\bin\Debug\netstandard2.0\TopLib01.dll
  Shared01 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\Shared01\bin\Debug\netstandard2.0\Shared01.dll
  Program01 -> C:\Users\MichaelBisbjerg\Git\ci-stresstest\Program01\bin\Debug\netstandard2.0\Program01.dll

Build succeeded.

C:\Program Files\dotnet\sdk\2.2.101\Microsoft.Common.CurrentVersion.targets(2110,5): warning MSB3088: Could not read state file "obj\Debug\netstandard2.0\TopLib01.csprojAssemblyReference.cache". The process cannot access the file 'C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\obj\Debug\netstandard2.0\TopLib01.csprojAssemblyReference.cache' because it is being used by another process. [C:\Users\MichaelBisbjerg\Git\ci-stresstest\TopLib01\TopLib01.csproj]
    1 Warning(s)
    0 Error(s)

Time Elapsed 00:00:00.41

marius-stanescu-archive360 · 2019-02-05T14:57:03Z

I have the same problem. Any solutions?

rainersigwald · 2019-02-05T18:07:18Z

The core problem here is that you're specifying a TargetFramework at the solution level. As you note, it works without that, while building all of the TargetFrameworks specified at the individual project level.

We hope to improve the experience around this -- at least providing an informative error, but maybe also doing something smarter. But at the moment, I'm not quite sure what that would be.

Can you describe why you're specifying a TargetFramework and what you expect to happen in that build, as distinct from the no-TF-specified solution build?

LordMike · 2019-02-05T18:37:41Z

For our current use case, we wanted to optimize our builds a bit, because we're targeting two frameworks to allow our developers to test the modules they're working (they have a small program.cs, and a shared library to actually run code -- only available to them when in netcoreapp2.x).

So our builds on the CI would be building for both coreapp and netstandard. Targeting just netstandard (since everything in the slution supports that anyways) did this and shaved a few seconds of our builds.

We've workarounded by not targeting the specific framework, since we found that it was just a few seconds..

However. I'm not at all sure why this is an issue.. All of the projects in question target netstandard2.0..

HenrikBach1 · 2019-02-20T06:05:28Z

Hi,

I've a case where there is no specifying of TargetFramework at the solution level where the dotnet build behaves similar in some cases where TargetFramework at the project level refers to netstandard2.0.

This behavior has just emerged without any notice in dotnet core 2.0. I've also tried to upgrade to dotnet core 2.1, but in vein.

In my case, there seems to be a race condition from my hardware setup, due to the dotnet build leaves few spawned dotnet processes.

Thank you.

Regards
Henrik

LordMike · 2019-02-20T09:09:11Z

Note that the spawned processes could be the Dotnet Build Server. You can try shutting them down by running dotnet build-server shutdown. If its that, they shouldn't be a problem (they're meant to speed up future builds). I have an anecdotal feeling that we're worse of for having the build server, but so far it's just that.

HenrikBach1 · 2019-02-20T10:02:25Z

Two out of three of them shut down with the dotnet build-server shutdown command:

myagent@3f98f3ab2e96:~/projects/azure/iotedge/scripts/linux$ ps
  PID TTY          TIME CMD
   31 pts/1    00:00:00 bash
10834 pts/1    00:00:04 dotnet
10850 pts/1    00:00:04 dotnet
10866 pts/1    00:00:06 dotnet
11617 pts/1    00:00:00 ps
myagent@3f98f3ab2e96:~/projects/azure/iotedge/scripts/linux$ dotnet build-server shutdown
Shutting down MSBuild server...
Shutting down VB/C# compiler server...
VB/C# compiler server shut down successfully.
MSBuild server shut down successfully.
myagent@3f98f3ab2e96:~/projects/azure/iotedge/scripts/linux$ ps
  PID TTY          TIME CMD
   31 pts/1    00:00:00 bash
10866 pts/1    00:00:07 dotnet
11668 pts/1    00:00:00 ps
myagent@3f98f3ab2e96:~/projects/azure/iotedge/scripts/linux$ ps 10866
  PID TTY      STAT   TIME COMMAND
10866 pts/1    SLl    0:07 /usr/share/dotnet/dotnet /usr/share/dotnet/sdk/2.2.102/MSBuild.dll /usr/share/dotnet/sdk/2.2.102/MSBuild.dll /nologo /nodemode:1 /nodeReuse:true
myagent@3f98f3ab2e96:~/projects/azure/iotedge/scripts/linux$

LordMike · 2019-02-20T12:09:51Z

That does seem like one of the build processes from the parallel build (the /nodeReuse:true option). That, I have no good answer for.

HenrikBach1 · 2019-02-22T04:54:23Z

Ok.

Problem solved by provisioning a new virtual machine.

anttirt · 2019-03-25T10:58:06Z

We've run into another similar (but not the exact same) race condition:

/usr/share/dotnet/sdk/2.2.103/Microsoft.Common.CurrentVersion.targets(2110,5):
error MSB3248: Parameter "AssemblyFiles" has invalid value
	"/build/Engine/Server/bin/Debug/netcoreapp2.2/Hydra.Engine.Server.dll".
The process cannot access the file
	'/build/Engine/Server/bin/Debug/netcoreapp2.2/Hydra.Engine.Server.dll'
because it is being used by another process. [/build/Server/Map/Hydra.Map.csproj]

I'm not sure if this is caused by a bug in the build system or if there is somehow a conflict in our projects that could be resolved by fixing the project definition. We use standard project layouts with simple SDK style csproj projects.

In the latter case, it would be amazing if the build system could detect the sources of these conflicts and warn or error on them.

victor-yarema · 2019-06-13T09:58:58Z

The core problem here is that you're specifying a TargetFramework at the solution level. As you note, it works without that, while building all of the TargetFrameworks specified at the individual project level.

I am getting this error if I simply call dotnet build Main.sln without specifying the target framework.
So I can't believe in this "core problem" explanation.

victor-yarema · 2019-06-13T10:22:33Z

Currently I am thinking about workaround to simply change build script part from something like

dotnet build Main.sln &&
echo 'Success'

To something like

{
  dotnet build Main.sln ||
  dotnet build Main.sln
} &&
echo 'Success'

There can be more tries if needed.
I already did that trick with MSBuild in year 2013 or 2014 when we were unable to get reliable build of website project (ASP.NET). We had 4 tries back then because 2 tries were not enough.
P.S. The problem with website project was different.

rainersigwald · 2019-06-13T14:32:25Z

@victor-yarema Can you please file a new issue for that? Please tag me. Because you're not specifying global properties at the solution level, you're seeing a different problem.

If you can attach a binary log of your build (a successful build log is fine), hopefully we can figure out what's going on. Please read the warnings about the information contained in the log. If you cannot upload such a log, please file the bug anyway. I can suggest additional debugging techniques.

….1 (#2902) - Microsoft.DotNet.Cli.Runtime - 5.0.100-alpha1.19465.1

theimowski · 2020-07-15T13:33:14Z

FWIW, I've experienced similar issue with GenerateDepsFile phase using .net sdk 3.1.301 - in my case one way to mitigate the issue is to explicitly build the project in question first (using dotnet build) before building projects that depend on it.

mercer · 2020-12-31T16:19:55Z

I see builds fail with p5 because of an issue similar to this
our pattern is to start several processes with
dotnet tool restore
then dotnet ef database update --no-build --verbose --context AppDbContext --project "SomeProjectThatMigratesSchemaLocally.csproj"
then dotnet run --no-restore --project ./src/SomeProject.Startup *> console.log
then execute an e2e test to see all is good
we have custom snappy linux azure vm agents with azure yaml pipelines
don't have any lead as to root cause

HenrikWM · 2021-05-04T09:14:59Z

Encountered this build-error on a project of ours. We build our .Net Core solutions using a FAKE-script and that script failed while building a solution. A manual build of the solution with dotnet build worked fine.

Turns out FAKE's FAKE.DotNet.MSBuild module adds the parameter -maxcpucount to the command dotnet.exe build as a parameter without a value.

From the build-log we see the maxcpucount parameter being used:

[...]
C:\Program Files\dotnet\sdk\5.0.104\MSBuild.dll /nologo - \
distributedlogger:Microsoft.DotNet.Tools.MSBuild.MSBuildLogger,C:\Program \
Files\dotnet\sdk\5.0.104\dotnet.dll*Microsoft.DotNet.Tools.MSBuild.MSBuildForwardingLogger,C:\Program \
Files\dotnet\sdk\5.0.104\dotnet.dll -maxcpucount -property:Configuration=Release -target:Restore -verbosity:m /bl: [...]  
Determining projects to restore...
[...]

From the documentation of MSBuild about when no default value of maxcpucount is provided:

If you include this switch without specifying a value, MSBuild will use up to the number of processors in the computer

Which explains our issue of getting errors when building via FAKE but not via manual dotnet build.

Our fix was to add MaxCpuCount = Some(Some 1) to our script's MSBuildParams to ensure no parallelization and thus preventing the read-locks on the files during a build.

Ian1971 · 2021-08-13T09:23:33Z

I was bugged by this problem for a while. Took ages to spot the cause and resulted in having to build things twice if it failed. It turned out that one projects that referenced the project that always reported the error only had TargetFramework netstandard2.0, whilst all the others were both net5.0 & netstandard2.0. Once I switched that project to <TargetFrameworks>net5.0;netstandard2.0</TargetFrameworks> the problem went away.

Chris-D-Turk · 2022-05-01T17:00:11Z

I'm currently facing this issue with the latest net6.0 sdk in the following build scenario:

Solution with one "netstandard2.1;MonoAndroid10.0" (MSBuild.Sdk.Extras/3.0.22) and two "netstandard2.1" (Microsoft.NET.Sdk) projects
The "netstandard2.1;MonoAndroid10.0" project references both netstandard-only projects
Build using dotnet build xxx.sln /p:Configuration=Debug --framework netstandard2.1

The error occurs (on the first netstandard-only project) in ~30% of all builds.

ptupitsyn · 2022-09-21T06:54:32Z

I've started facing this issue on CI (TeamCity) after introducing Source Generators to my project.

06:48:18     Restored /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite/Apache.Ignite.csproj (in 282 ms).
06:48:18     Restored /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj (in 288 ms).
06:48:18     Restored /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Tests/Apache.Ignite.Tests.csproj (in 501 ms).
06:48:18     Restored /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Benchmarks/Apache.Ignite.Benchmarks.csproj (in 528 ms).
06:48:21     Apache.Ignite.Internal.Generators -> /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/bin/Debug/netstandard2.0/Apache.Ignite.Internal.Generators.dll
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018: The "GenerateDepsFile" task failed unexpectedly. [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018: System.IO.IOException: The process cannot access the file '/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/bin/Debug/netstandard2.0/Apache.Ignite.Internal.Generators.deps.json' because it is being used by another process. [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018:    at Microsoft.Win32.SafeHandles.SafeFileHandle.Init(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize) [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018:    at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize) [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018:    at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize) [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018:    at System.IO.Strategies.FileStreamHelpers.ChooseStrategy(FileStream fileStream, String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, Int64 preallocationSize) [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018:    at System.IO.File.Create(String path) [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018:    at Microsoft.NET.Build.Tasks.GenerateDepsFile.WriteDepsFile(String depsFilePath) [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018:    at Microsoft.NET.Build.Tasks.GenerateDepsFile.ExecuteCore() [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018:    at Microsoft.NET.Build.Tasks.TaskBase.Execute() [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:21   /usr/share/dotnet/sdk/6.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(172,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/Apache.Ignite.Internal.Generators.csproj]
06:48:23     Apache.Ignite -> /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite/bin/Debug/netstandard2.1/Apache.Ignite.dll
06:48:26     Apache.Ignite.Tests -> /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Tests/bin/Debug/netcoreapp3.1/Apache.Ignite.Tests.dll
06:48:26     Apache.Ignite.Benchmarks -> /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Benchmarks/bin/Debug/netcoreapp3.1/Apache.Ignite.Benchmarks.dll
06:48:26   
06:48:26   Build FAILED.
06:48:26

The workaround for me was to add dotnet restore step before dotnet build. Sometimes there is still some file access issue during dotnet build, but it gets retried and the build passes:

All projects are up-to-date for restore.
[05:55:24 ](https://ci.ignite.apache.org/buildConfiguration/ignite3_Test_RunNetTests/6778428?buildTab=log&focusLine=488&linesState=488)    /usr/share/dotnet/sdk/6.0.100/Microsoft.Common.CurrentVersion.targets(4635,5): warning MSB3026: Could not copy "obj/Debug/netstandard2.0/Apache.Ignite.Internal.Generators.dll" to "bin/Debug/netstandard2.0/Apache.Ignite.Internal.Generators.dll". Beginning retry 1 in 1000ms. The process cannot access the file '/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/obj/Debug/netstandard2.0/Apache.Ignite.Internal.Generators.dll' because it is being used by another process.
[05:55:24 ](https://ci.ignite.apache.org/buildConfiguration/ignite3_Test_RunNetTests/6778428?buildTab=log&focusLine=489&linesState=489)      Apache.Ignite.Internal.Generators -> /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/bin/Debug/netstandard2.0/Apache.Ignite.Internal.Generators.dll
[05:55:25 ](https://ci.ignite.apache.org/buildConfiguration/ignite3_Test_RunNetTests/6778428?buildTab=log&focusLine=490&linesState=490)      Apache.Ignite.Internal.Generators -> /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Internal.Generators/bin/Debug/netstandard2.0/Apache.Ignite.Internal.Generators.dll
[05:55:27 ](https://ci.ignite.apache.org/buildConfiguration/ignite3_Test_RunNetTests/6778428?buildTab=log&focusLine=491&linesState=491)      Apache.Ignite -> /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite/bin/Debug/netstandard2.1/Apache.Ignite.dll
[05:55:30 ](https://ci.ignite.apache.org/buildConfiguration/ignite3_Test_RunNetTests/6778428?buildTab=log&focusLine=492&linesState=492)      Apache.Ignite.Tests -> /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Tests/bin/Debug/netcoreapp3.1/Apache.Ignite.Tests.dll

So clearly there is some race condition with file access.

We are only using the project (.csproj) themselves. Apparently TargetFramework definitions in solution files can cause problems while publishing projects: dotnet/sdk#2902

This branch is currently seeing intermittent build errors, which seem to be related to concurrent framework builds (see dotnet/sdk#2902). Changing the logging project to explicitly target Standard 2.1 only, to see if it resolves the build issues.

Bartleby2718 · 2022-12-13T00:10:45Z

@rainersigwald What is the recommended alternative if "specifying a TargetFramework at the solution level" is what I want? Is there anything else besides using the -maxcpucount switch?

gavinheisesc · 2023-01-03T03:35:11Z

A plain dotnet build MyProject.sln -f netstandard2.0 (no build script) also exhibits this issue.

A dotnet build MyProject.sln seemingly always succeeds (locally and on CI server), at least for three consecutive attempts right now.

@LordMike Thank you for mentioning this! There seems to be a problem with the --framework parameter when the projects just target a single framework version. This is still happening for me as of .NET 7.

Works (with <TargetFrameworks>net6.0;net7.0</TargetFrameworks> in the CSPROJs): dotnet build --no-restore -c Release -f net7.0 Solution.sln
Works (with <TargetFramework>net7.0</TargetFramework> in the CSPROJs): dotnet build --no-restore -c Release Solution.sln
DOES NOT WORK (with <TargetFramework>net7.0</TargetFramework> in the CSPROJs): dotnet build --no-restore -c Release -f net7.0 Solution.sln

mcallaghan-geotab · 2023-02-06T16:19:00Z

After switching/adding on TargetFrameworks (net6.0;net7.0), msbuild immediately starting giving file access concurency warnings with retries, and frequently started failing entirely. (similar symptoms stated above)

In this instance we only had a single solution file, and single project, targeting >1 SDK for build.

The only workaround that completely resolved the issue was to drop down CPU count to 1 to disable any parallel execution.

dotnet build --output $BUILD_ARTIFACT_DIR /maxcpucount:1

^ after doing this, ALL warnings were immediately eliminated and build hasn't failed since

alnaimi-github · 2023-09-21T20:27:26Z

eading data.json
Unhandled exception. System.IO.FileNotFoundException: Could not find file 'F:\My University\Level Two\C# File\json\json\bin\Debug\net5.0\data.json'.
File name: 'F:\My University\Level Two\C# File\json\json\bin\Debug\net5.0\data.json'
at System.IO.FileStream.ValidateFileHandle(SafeFileHandle fileHandle)
at System.IO.FileStream.CreateFileOpenHandle(FileMode mode, FileShare share, FileOptions options)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
at System.IO.StreamReader.ValidateArgsAndOpenPath(String path, Encoding encoding, Int32 bufferSize)
at System.IO.StreamReader..ctor(String path, Encoding encoding, Boolean detectEncodingFromByteOrderMarks)
at System.IO.File.InternalReadAllText(String path, Encoding encoding)
at System.IO.File.ReadAllText(String path)
at JsonDataStorage01.Program.Main(String[] args) in F:\My University\Level Two\C# File\json\json\Program.cs:line 36

F:\My University\Level Two\C# File\json\json\bin\Debug\net5.0\json.exe (process 22928) exited with code -532462766.
Press any key to close this window . . .

sykhro · 2024-05-28T09:03:00Z

Running into this with .net8 100% of the time on high core-count build servers (>64 CPUs)

rainersigwald · 2024-05-28T14:49:29Z

@sykhro can you please file a new issue with details about exactly what errors you're experiencing? This issue has accumulated several types of problem, and you may have a new problem with similar symptoms.

victor-fialkin-paylocity · 2024-06-11T13:46:37Z

Seeing the same on CI pipeline time to time. (docker, .NET 8)

/usr/share/dotnet/sdk/8.0.301/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(284,5): error MSB4018: The "GenerateDepsFile" task failed unexpectedly. [/REMOVED.csproj]
#16 21.06 /usr/share/dotnet/sdk/8.0.301/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.targets(284,5): error MSB4018: System.IO.IOException: The process cannot access the file '/bin/Debug/net8.0/REMOVED.deps.json' because it is being used by another process. [REMOVED.csproj]

ArielCCardoso · 2024-09-20T16:38:57Z

Any updates? I'm experiencing the same behavior in CI environments with containers on Kubernetes. SDKs .NET 6.0 and 8.0.

Dunge · 2024-09-27T19:28:07Z

I'm encountering a similar problem. I have a build pipeline using dotnet publish on a .sln file (because it would be annoying to create a step of each csproj independently). The sln file only has net8.0 executable, but also multitargeting class libraries.

Previously had libraries targeting netstandard2.0 and it worked perfectly. I recently transformed them to multitarget (net8.0;net427) to gain advantages of new features in all my net8.0 projects (a whole lot of them). I only have one old project still stuck on net472 (not part of the .sln mentioned above) but I need it to still be capable to be built.

I then received error NETSDK1129: The 'Publish' target is not supported without specifying a target framework. The current project targets multiple frameworks, you must specify one of the following frameworks in order to publish: net8.0, net472

So I added -f net8.0 to the dotnet publish command, and now am getting The process cannot access the file '' because it is being used by another process. errors.

So what are the solutions? Create a step for each csproj? Use the /maxcpucount:1 switch and slow down the process? Anything else?

atsushieno mentioned this issue May 14, 2019

unable to continue development atsushieno/managed-midi#42

Open

aslicerh mentioned this issue Jun 6, 2019

Build failes with IOException redhat-developer/s2i-dotnetcore#277

Closed

wli3 pushed a commit that referenced this issue Feb 7, 2020

Update dependencies from https://github.com/dotnet/cli build 20190915…

cfd11ac

….1 (#2902) - Microsoft.DotNet.Cli.Runtime - 5.0.100-alpha1.19465.1

vchirikov mentioned this issue Mar 25, 2020

Solution build with RuntimeIdentifier specified as global property can fail with concurrency problems #10987

Closed

bearpro mentioned this issue May 31, 2020

Очередная попытка перейти на dotnet 3.1 Seterator/seterator.ru#105

Merged

omajid mentioned this issue Feb 18, 2021

Enable arcade-powered-source-build for local and CI builds dotnet/command-line-api#1204

Merged

KirillOsenkov mentioned this issue Oct 13, 2021

GenerateDepsFile should not dump stack when failing to access a file in the build #14980

Open

xgqt mentioned this issue Feb 14, 2023

dotnet-utils.eclass: Introduction of a eclass for dotnet packages. gentoo/gentoo#29309

Closed

limeship-chd mentioned this issue Aug 30, 2023

[BUG] Concurrency errors when accessing same files as parallel test nx-dotnet/nx-dotnet#731

Open

This was referenced Sep 16, 2023

feat: allow extra arguments to be passed to dotnet publish command oddstr13/jellyfin-plugin-repository-manager#98

Closed

CI build consistently fails LizardByte/Themerr-jellyfin#100

Closed

Apollo3zehn pushed a commit to Apollo3zehn/FluentModbus that referenced this issue Feb 21, 2024

Try to fix dotnet/sdk#2902

7935faf

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GenerateDepsFile: The process cannot access the file '...\MyProject.deps.json' because it is being used by another process. #2902

GenerateDepsFile: The process cannot access the file '...\MyProject.deps.json' because it is being used by another process. #2902

LordMike commented Feb 3, 2019 •

edited

Loading

LordMike commented Feb 3, 2019 •

edited

Loading

marius-stanescu-archive360 commented Feb 5, 2019

rainersigwald commented Feb 5, 2019

LordMike commented Feb 5, 2019

HenrikBach1 commented Feb 20, 2019

LordMike commented Feb 20, 2019

HenrikBach1 commented Feb 20, 2019 •

edited

Loading

LordMike commented Feb 20, 2019 •

edited

Loading

HenrikBach1 commented Feb 22, 2019

anttirt commented Mar 25, 2019

victor-yarema commented Jun 13, 2019

victor-yarema commented Jun 13, 2019 •

edited

Loading

rainersigwald commented Jun 13, 2019

theimowski commented Jul 15, 2020

mercer commented Dec 31, 2020

HenrikWM commented May 4, 2021

Ian1971 commented Aug 13, 2021

Chris-D-Turk commented May 1, 2022

ptupitsyn commented Sep 21, 2022

Bartleby2718 commented Dec 13, 2022

gavinheisesc commented Jan 3, 2023

mcallaghan-geotab commented Feb 6, 2023 •

edited

Loading

alnaimi-github commented Sep 21, 2023

sykhro commented May 28, 2024

rainersigwald commented May 28, 2024

victor-fialkin-paylocity commented Jun 11, 2024 •

edited

Loading

ArielCCardoso commented Sep 20, 2024

Dunge commented Sep 27, 2024

GenerateDepsFile: The process cannot access the file '...\MyProject.deps.json' because it is being used by another process. #2902

GenerateDepsFile: The process cannot access the file '...\MyProject.deps.json' because it is being used by another process. #2902

Comments

LordMike commented Feb 3, 2019 • edited Loading

Observations:

Versions:

Reproduction

Notes on repro.

LordMike commented Feb 3, 2019 • edited Loading

The process cannot access the file '...\Shared02.dll' because it is being used by another process.

Could not read state file "obj\..\TopLib01.csprojAssemblyReference.cache". The process cannot access the file '..\TopLib01.csprojAssemblyReference.cache' because it is being used by another process.

marius-stanescu-archive360 commented Feb 5, 2019

rainersigwald commented Feb 5, 2019

LordMike commented Feb 5, 2019

HenrikBach1 commented Feb 20, 2019

LordMike commented Feb 20, 2019

HenrikBach1 commented Feb 20, 2019 • edited Loading

LordMike commented Feb 20, 2019 • edited Loading

HenrikBach1 commented Feb 22, 2019

anttirt commented Mar 25, 2019

victor-yarema commented Jun 13, 2019

victor-yarema commented Jun 13, 2019 • edited Loading

rainersigwald commented Jun 13, 2019

theimowski commented Jul 15, 2020

mercer commented Dec 31, 2020

HenrikWM commented May 4, 2021

Ian1971 commented Aug 13, 2021

Chris-D-Turk commented May 1, 2022

ptupitsyn commented Sep 21, 2022

Bartleby2718 commented Dec 13, 2022

gavinheisesc commented Jan 3, 2023

mcallaghan-geotab commented Feb 6, 2023 • edited Loading

alnaimi-github commented Sep 21, 2023

sykhro commented May 28, 2024

rainersigwald commented May 28, 2024

victor-fialkin-paylocity commented Jun 11, 2024 • edited Loading

ArielCCardoso commented Sep 20, 2024

Dunge commented Sep 27, 2024

LordMike commented Feb 3, 2019 •

edited

Loading

LordMike commented Feb 3, 2019 •

edited

Loading

HenrikBach1 commented Feb 20, 2019 •

edited

Loading

LordMike commented Feb 20, 2019 •

edited

Loading

victor-yarema commented Jun 13, 2019 •

edited

Loading

mcallaghan-geotab commented Feb 6, 2023 •

edited

Loading

victor-fialkin-paylocity commented Jun 11, 2024 •

edited

Loading