Check for usages of PhysicalFileProvider in all repos for leaks

[kichalla]

Related to issue: aspnet/Mvc#7696

Repos where we instantiate the PhysicalFileProvider currently:

• Hosting
• Configuration

Repro:
You should see the memory increasing rapidly when you run the below app. Un-comment the line which uses the finalizer to see the memory usage being stable.

using Microsoft.Extensions.FileProviders;
using Microsoft.Extensions.Primitives;
using System;
using System.Diagnostics;

namespace PhysicalFileProviderLeakInvestigation
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Process Id:" + Process.GetCurrentProcess().Id);
            Console.WriteLine("Enter any key to start watching folders...");
            Console.ReadKey();

            while (true)
            {
                var fp = new PhysicalFileProvider(@"c:\github\mvc");
                //var fp = new FinalizingPhysicalFileProvider(new PhysicalFileProvider(@"c:\github\mvc"));
                fp.Watch("*.*");
            }
        }
    }

    class FinalizingPhysicalFileProvider : IFileProvider
    {
        private readonly PhysicalFileProvider _physicalFileProvider;

        public FinalizingPhysicalFileProvider(PhysicalFileProvider physicalFileProvider)
        {
            _physicalFileProvider = physicalFileProvider;
        }

        public IDirectoryContents GetDirectoryContents(string subpath)
        {
            return _physicalFileProvider.GetDirectoryContents(subpath);
        }

        public IFileInfo GetFileInfo(string subpath)
        {
            return _physicalFileProvider.GetFileInfo(subpath);
        }

        public IChangeToken Watch(string filter)
        {
            return _physicalFileProvider.Watch(filter);
        }

        ~FinalizingPhysicalFileProvider()
        {
            _physicalFileProvider.Dispose();
        }
    }
}

cc @Tratcher

[Tratcher]

Or call Dispose...

The Finalizer may be a good mitigation for components like ConfigurationBuilder. We should look into making HostingEnvironment implement IDisposable as the primary fix for aspnet/Mvc#7696.

[kichalla]

Did a search on Universe repo's submodules:

C:\github\Universe\modules [dev ≡]> findstr /spin /c:"new PhysicalFileProvider(" *.cs
AuthSamples\samples\StaticFilesAuth\Startup.cs:97:            var files = new PhysicalFileProvider(Path.Combine(env.ContentRootPath, "PrivateFiles"));
Caching\samples\MemoryCacheFileWatchSample\Program.cs:18:            var fileProvider = new PhysicalFileProvider(Path.Combine(Directory.GetCurrentDirectory(), "WatchedFiles"));
Configuration\src\Config.FileExtensions\FileConfigurationExtensions.cs:51:            return new PhysicalFileProvider(AppContext.BaseDirectory ?? string.Empty);
Configuration\src\Config.FileExtensions\FileConfigurationExtensions.cs:72:            return builder.SetFileProvider(new PhysicalFileProvider(basePath));
Configuration\src\Config.FileExtensions\FileConfigurationSource.cs:82:                    FileProvider = new PhysicalFileProvider(directory);
Configuration\src\Config.KeyPerFile\KeyPerFileConfigurationBuilderExtensions.cs:23:                source.FileProvider = new PhysicalFileProvider(directoryPath);
Configuration\src\Config.UserSecrets\UserSecretsConfigurationExtensions.cs:138:                ? new PhysicalFileProvider(directoryPath)
Configuration\test\Config.FunctionalTests\ConfigurationTests.cs:81:            _fileProvider = new PhysicalFileProvider(_fileSystem.RootPath);
Diagnostics\test\Microsoft.AspNetCore.Diagnostics.Tests\ExceptionDetailsProviderTest.cs:66:            using (var provider = new PhysicalFileProvider(rootPath))
Diagnostics\test\Microsoft.AspNetCore.Diagnostics.Tests\ExceptionDetailsProviderTest.cs:90:            using (var provider = new PhysicalFileProvider(rootPath))
Diagnostics\test\Microsoft.AspNetCore.Diagnostics.Tests\ExceptionDetailsProviderTest.cs:261:                new PhysicalFileProvider(Directory.GetCurrentDirectory()),
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:25:            using (var provider = new PhysicalFileProvider(Path.GetTempPath()))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:35:            using (var provider = new PhysicalFileProvider(Path.GetTempPath()))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:65:            using (var provider = new PhysicalFileProvider(Path.GetTempPath()))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:92:            using (var provider = new PhysicalFileProvider(Path.GetTempPath()))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:118:            using (var provider = new PhysicalFileProvider(Path.GetTempPath()))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:130:            using (var provider = new PhysicalFileProvider(Path.GetTempPath()))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:140:            using (var provider = new PhysicalFileProvider(Path.GetTempPath()))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:154:                using (var provider = new PhysicalFileProvider(root.RootPath))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:169:                using (var provider = new PhysicalFileProvider(root.RootPath))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:182:                using (var provider = new PhysicalFileProvider(root.RootPath))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:203:                using (var provider = new PhysicalFileProvider(root.RootPath))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:225:                using (var provider = new PhysicalFileProvider(root.RootPath))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:245:                using (var provider = new PhysicalFileProvider(root.RootPath))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:262:                using (var provider = new PhysicalFileProvider(root.RootPath, ExclusionFilters.None))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:282:                using (var provider = new PhysicalFileProvider(root.RootPath))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:308:                        using (var provider = new PhysicalFileProvider(root.RootPath, physicalFilesWatcher))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:337:                        using (var provider = new PhysicalFileProvider(root.RootPath, physicalFilesWatcher))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:377:                        using (var provider = new PhysicalFileProvider(root.RootPath, physicalFilesWatcher))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:406:                        using (var provider = new PhysicalFileProvider(root.RootPath, physicalFilesWatcher))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:432:                        using (var provider = new PhysicalFileProvider(root.RootPath, physicalFilesWatcher))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:473:            using (var provider = new PhysicalFileProvider(Directory.GetCurrentDirectory()))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:504:            using (var provider = new PhysicalFileProvider(Directory.GetCurrentDirectory()))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:515:            using (var provider = new PhysicalFileProvider(Path.GetTempPath()))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:545:            using (var provider = new PhysicalFileProvider(Path.GetTempPath()))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:576:            using (var provider = new PhysicalFileProvider(Path.GetTempPath()))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:586:            using (var provider = new PhysicalFileProvider(Path.GetTempPath()))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:596:            using (var provider = new PhysicalFileProvider(Path.GetTempPath()))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:611:                using (var provider = new PhysicalFileProvider(root.RootPath))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:628:                using (var provider = new PhysicalFileProvider(root.RootPath))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:653:                using (var provider = new PhysicalFileProvider(root.RootPath))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:678:                using (var provider = new PhysicalFileProvider(root.RootPath))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:699:                using (var provider = new PhysicalFileProvider(root.RootPath))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:722:                using (var provider = new PhysicalFileProvider(root.RootPath, ExclusionFilters.None))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:739:                        using (var provider = new PhysicalFileProvider(root.RootPath, physicalFilesWatcher))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:766:                using (var provider = new PhysicalFileProvider(root.RootPath))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:785:                        using (var provider = new PhysicalFileProvider(root.RootPath, physicalFilesWatcher))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:817:                        using (var provider = new PhysicalFileProvider(root.RootPath, physicalFilesWatcher))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:842:                using (var provider = new PhysicalFileProvider(root.RootPath))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:856:                using (var provider = new PhysicalFileProvider(root.RootPath))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:870:                using (var provider = new PhysicalFileProvider(root.RootPath))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:885:                using (var provider = new PhysicalFileProvider(root.RootPath))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:904:                using (var provider = new PhysicalFileProvider(root.RootPath))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:923:                        using (var provider = new PhysicalFileProvider(root.RootPath, physicalFilesWatcher))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:947:                        using (var provider = new PhysicalFileProvider(root.RootPath, physicalFilesWatcher))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:971:                        using (var provider = new PhysicalFileProvider(root.RootPath, physicalFilesWatcher))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:1028:                        using (var provider = new PhysicalFileProvider(root.RootPath, physicalFilesWatcher))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:1069:                        using (var provider = new PhysicalFileProvider(root.RootPath, physicalFilesWatcher))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:1094:                        using (var provider = new PhysicalFileProvider(root.RootPath, physicalFilesWatcher))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:1122:                using (var provider = new PhysicalFileProvider(root.RootPath))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:1141:                        using (var provider = new PhysicalFileProvider(root.RootPath, physicalFilesWatcher))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:1172:            using (var provider = new PhysicalFileProvider(root.RootPath, physicalFilesWatcher))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:1241:                        using (var provider = new PhysicalFileProvider(root.RootPath, physicalFilesWatcher))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:1279:                        using (var provider = new PhysicalFileProvider(root.RootPath, physicalFilesWatcher))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:1306:                        using (var provider = new PhysicalFileProvider(root.RootPath, physicalFilesWatcher))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:1334:            using (var provider = new PhysicalFileProvider(root.RootPath, physicalFilesWatcher))
FileSystem\test\FS.Physical.Tests\PhysicalFileProviderTests.cs:1358:            using (var provider = new PhysicalFileProvider(root.RootPath, physicalFilesWatcher))
Hosting\src\Microsoft.AspNetCore.Hosting\Internal\HostingEnvironmentExtensions.cs:29:            hostingEnvironment.ContentRootFileProvider = new PhysicalFileProvider(hostingEnvironment.ContentRootPath);
Hosting\src\Microsoft.AspNetCore.Hosting\Internal\HostingEnvironmentExtensions.cs:53:                hostingEnvironment.WebRootFileProvider = new PhysicalFileProvider(hostingEnvironment.WebRootPath);
Hosting\src\Microsoft.Extensions.Hosting\HostBuilder.cs:137:            _hostingEnvironment.ContentRootFileProvider = new PhysicalFileProvider(_hostingEnvironment.ContentRootPath);
JavaScriptServices\src\Microsoft.AspNetCore.SpaServices.Extensions\StaticFiles\DefaultSpaStaticFileProvider.cs:46:                _fileProvider = new PhysicalFileProvider(absoluteRootPath);
MvcPrecompilation\src\Microsoft.AspNetCore.Mvc.Razor.ViewCompilation\Internal\MvcServiceProvider.cs:102:                WebRootFileProvider = new PhysicalFileProvider(_projectPath),
MvcPrecompilation\src\Microsoft.AspNetCore.Mvc.Razor.ViewCompilation\Internal\MvcServiceProvider.cs:103:                ContentRootFileProvider = new PhysicalFileProvider(_contentRoot),
StaticFiles\test\Microsoft.AspNetCore.StaticFiles.Tests\DefaultFilesMiddlewareTests.cs:56:            using (var fileProvider = new PhysicalFileProvider(Path.Combine(AppContext.BaseDirectory, baseDir)))
StaticFiles\test\Microsoft.AspNetCore.StaticFiles.Tests\DefaultFilesMiddlewareTests.cs:95:            using (var fileProvider = new PhysicalFileProvider(Path.Combine(AppContext.BaseDirectory, baseDir)))
StaticFiles\test\Microsoft.AspNetCore.StaticFiles.Tests\DefaultFilesMiddlewareTests.cs:134:            using (var fileProvider = new PhysicalFileProvider(Path.Combine(AppContext.BaseDirectory, baseDir)))
StaticFiles\test\Microsoft.AspNetCore.StaticFiles.Tests\DefaultFilesMiddlewareTests.cs:172:            using (var fileProvder = new PhysicalFileProvider(Path.Combine(AppContext.BaseDirectory, baseDir)))
StaticFiles\test\Microsoft.AspNetCore.StaticFiles.Tests\DirectoryBrowserMiddlewareTests.cs:75:            using (var fileProvider = new PhysicalFileProvider(Path.Combine(AppContext.BaseDirectory, baseDir)))
StaticFiles\test\Microsoft.AspNetCore.StaticFiles.Tests\DirectoryBrowserMiddlewareTests.cs:112:            using (var fileProvider = new PhysicalFileProvider(Path.Combine(AppContext.BaseDirectory, baseDir)))
StaticFiles\test\Microsoft.AspNetCore.StaticFiles.Tests\DirectoryBrowserMiddlewareTests.cs:154:            using (var fileProvider = new PhysicalFileProvider(Path.Combine(AppContext.BaseDirectory, baseDir)))
StaticFiles\test\Microsoft.AspNetCore.StaticFiles.Tests\DirectoryBrowserMiddlewareTests.cs:193:            using (var fileProvider = new PhysicalFileProvider(Path.Combine(AppContext.BaseDirectory, baseDir)))
StaticFiles\test\Microsoft.AspNetCore.StaticFiles.Tests\DirectoryBrowserMiddlewareTests.cs:229:            using (var fileProvider = new PhysicalFileProvider(Path.Combine(AppContext.BaseDirectory, baseDir)))
StaticFiles\test\Microsoft.AspNetCore.StaticFiles.Tests\StaticFileMiddlewareTests.cs:94:            using (var fileProvider = new PhysicalFileProvider(AppContext.BaseDirectory))
StaticFiles\test\Microsoft.AspNetCore.StaticFiles.Tests\StaticFileMiddlewareTests.cs:146:            using (var fileProvider = new PhysicalFileProvider(Path.Combine(AppContext.BaseDirectory, baseDir)))
StaticFiles\test\Microsoft.AspNetCore.StaticFiles.Tests\StaticFileMiddlewareTests.cs:176:            using (var fileProvider = new PhysicalFileProvider(Path.Combine(AppContext.BaseDirectory, baseDir)))
StaticFiles\test\Microsoft.AspNetCore.StaticFiles.Tests\StaticFileMiddlewareTests.cs:235:            using (var fileProvider = new PhysicalFileProvider(Path.Combine(AppContext.BaseDirectory, baseDir)))

Entropy:

samples\Mvc.DynamicCorsPolicyProvider\Startup.cs:42:                return new PhysicalFileProvider(hostingEnvironment.ContentRootPath);
samples\Mvc.RenderViewToString\Program.cs:66:                fileProvider = new PhysicalFileProvider(customApplicationBasePath);
samples\Mvc.RenderViewToString\Program.cs:71:                fileProvider = new PhysicalFileProvider(Directory.GetCurrentDirectory());

[pranavkm]

Might be unimportant if we get it work correctly, but not all repos call Watch on the PhysicalFileProvider. It's likely the only scenario you need to worry about disposing it.

[kichalla]

cc @sebastienros

[muratg]

Closing this. If we measure any issues, we'll fix them in the particular use location.