Port to .NET Core 3

[0xced]

Here's a pull request porting ICanHasDotnetCore to .NET Core 3 with a few side improvements. I have tried to break things in isolated commits so in case you don't want to merge it altogether you can cherry-pick commits of interest.

Here are the most important changes:

• Use Entity Framework Core instead of raw SQL
  This change makes it super easy to develop with SQLite instead of SQL Server (I did all the work on macOS using JetBrains Rider). To develop on SQLite, just add this in appsettings.Development.json:

{
  "Database": {
    "Provider": "Sqlite",
    "ConnectionString": "DataSource=/path/to/ICanHasDotnetCore.sqlite3"
  }
}

And maybe you could even use SQLite in production and save money on an SQL Server instance?

• Use NuGet.Protocol instead of NuGet.Core which is not compatible with .NET Core. This change has brought to light some packages that were incorrectly identified, the tests have been adapted accordingly.
• Improved logs overall. The logs added for Octokit allowed me to identify how to improve performance of GitHub scanning.
• Web project modernization

Updated the web project to work with Node.js 12 and Gulp 4. I also added a new Scan A NuGet package tab to get the dependency graph of a NuGet package without having to upload a file.

And here are less important changes:

• Updated NuGet packages to their latest versions
• Replaced AutoFac with built-in ASP.NET Core equivalent features
• Replace Newtonsoft.Json with System.Text.Json
• Asyncified all code
• Updated knowledge
• Fixed a potential ArgumentNullException

[CLAassistant]

All committers have signed the CLA.

[droyad]

This is no longer atomic. I used the MERGE command as it guaranteed (from my understanding) atomic insert/update of that record. With this change two threads could update the same statistic record and only the last in wins.

[0xced]

Good catch! I addressed this issue in 49561db. I used FlexLabs.EntityFrameworkCore.Upsert as EF Core does not handle upsert out of the box.

By increasing the EF Core log level to information (.MinimumLevel.Override("Microsoft.EntityFrameworkCore", LogEventLevel.Information)) and enabling sensitive data logging (optionsBuilder.EnableSensitiveDataLogging();) we can see how FlexLabs.Upsert works:

With SQLite:

-- Executing DbCommand [Parameters=[@p0='Newtonsoft.Json' (Nullable = false) (Size = 15), @p1='1' (DbType = String), @p2='Supported' (Nullable = false) (Size = 9), @p3='1' (DbType = String)], CommandType='Text', CommandTimeout='30']
INSERT INTO "PackageStatistics" AS "T" ("Name", "Count", "LatestSupportType") VALUES (@p0, @p1, @p2) ON CONFLICT ("Name") DO UPDATE SET "Count" = ( "T"."Count" + @p3 )

With SQL Server:

-- Executing DbCommand [Parameters=[@p0='Newtonsoft.Json' (Nullable = false) (Size = 450), @p1='1', @p2='Supported' (Nullable = false) (Size = 4000), @p3='1'], CommandType='Text', CommandTimeout='30']
MERGE INTO [PackageStatistics] WITH (HOLDLOCK) AS [T] USING ( VALUES (@p0, @p1, @p2) ) AS [S] ([Name], [Count], [LatestSupportType]) ON [T].[Name] = [S].[Name] WHEN NOT MATCHED BY TARGET THEN INSERT ([Name], [Count], [LatestSupportType]) VALUES ([Name], [Count], [LatestSupportType]) WHEN MATCHED THEN UPDATE SET [Count] = ( [T].[Count] + @p3 );

[droyad]

@0xced I haven't forgotten about this and will get to doing a full review. I did a quick scan and only one thing stood out to me so far (added a comment)

[0xced]

Excellent, I’ll stop force-pushing small changes while you are reviewing.

Would you consider using SQLite in production? I think for a simple two tables database (cache + stats) it would be well enough and cheaper than an SQL Server instance. Are you currently using an Azure SQL Database? Is it costly?

[0xced]

I have pushed a few more improvements. We could probably shave a few more milliseconds when scanning GitHub repositories if octokit/octokit.net#2151 gets merged and a new release of Octokit is released.

[droyad]

@0xced Thank you for this contribution. Unfortunately we've decided to shutter the site. The traffic has decreased significantly since .NET Core 3.1 was released (and previous 2.0). The ecosystem as a whole has transitioned over and the majority of use cases now work.