Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fake.DotNet.Testing.Expecto #1871

Merged
merged 6 commits into from
Apr 20, 2018
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions Fake.sln
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ Project("{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}") = "Fake.DotNet.MSBuild", "src/
EndProject
Project("{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}") = "Fake.DotNet.NuGet", "src/app/Fake.DotNet.NuGet/Fake.DotNet.NuGet.fsproj", "{93F1A71E-54E2-4C65-BB1E-1D499890317F}"
EndProject
Project("{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}") = "Fake.DotNet.Testing.Expecto", "src/app/Fake.DotNet.Testing.Expecto/Fake.DotNet.Testing.Expecto.fsproj", "{819E2756-7BEE-4FCD-BA08-7BF323405CBE}"
EndProject
Project("{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}") = "Fake.DotNet.Testing.MSpec", "src/app/Fake.DotNet.Testing.MSpec/Fake.DotNet.Testing.MSpec.fsproj", "{C1B87E1F-8D0D-4A78-9A24-9D5D38B7E9A8}"
EndProject
Project("{6EC3EE1D-3C4E-46DD-8F32-0CC8E7565705}") = "Fake.DotNet.Testing.NUnit", "src/app/Fake.DotNet.Testing.NUnit/Fake.DotNet.Testing.NUnit.fsproj", "{75C9DD21-B4EA-4117-BF4F-AFE777A80B5B}"
Expand Down Expand Up @@ -670,6 +672,18 @@ Global
{8561A35A-C2A4-43C7-A938-CB35A7747121}.Release|x64.Build.0 = Release|x64
{8561A35A-C2A4-43C7-A938-CB35A7747121}.Release|x86.ActiveCfg = Release|x86
{8561A35A-C2A4-43C7-A938-CB35A7747121}.Release|x86.Build.0 = Release|x86
{819E2756-7BEE-4FCD-BA08-7BF323405CBE}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{819E2756-7BEE-4FCD-BA08-7BF323405CBE}.Debug|Any CPU.Build.0 = Debug|Any CPU
{819E2756-7BEE-4FCD-BA08-7BF323405CBE}.Debug|x64.ActiveCfg = Debug|Any CPU
{819E2756-7BEE-4FCD-BA08-7BF323405CBE}.Debug|x64.Build.0 = Debug|Any CPU
{819E2756-7BEE-4FCD-BA08-7BF323405CBE}.Debug|x86.ActiveCfg = Debug|Any CPU
{819E2756-7BEE-4FCD-BA08-7BF323405CBE}.Debug|x86.Build.0 = Debug|Any CPU
{819E2756-7BEE-4FCD-BA08-7BF323405CBE}.Release|Any CPU.ActiveCfg = Release|Any CPU
{819E2756-7BEE-4FCD-BA08-7BF323405CBE}.Release|Any CPU.Build.0 = Release|Any CPU
{819E2756-7BEE-4FCD-BA08-7BF323405CBE}.Release|x64.ActiveCfg = Release|Any CPU
{819E2756-7BEE-4FCD-BA08-7BF323405CBE}.Release|x64.Build.0 = Release|Any CPU
{819E2756-7BEE-4FCD-BA08-7BF323405CBE}.Release|x86.ActiveCfg = Release|Any CPU
{819E2756-7BEE-4FCD-BA08-7BF323405CBE}.Release|x86.Build.0 = Release|Any CPU
{B636A082-4DB4-439D-8A37-E5214BDC00A3}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{B636A082-4DB4-439D-8A37-E5214BDC00A3}.Debug|Any CPU.Build.0 = Debug|Any CPU
{B636A082-4DB4-439D-8A37-E5214BDC00A3}.Debug|x64.ActiveCfg = Debug|x64
Expand Down Expand Up @@ -747,6 +761,7 @@ Global
{CA6EB1B3-EB3A-4063-8A6C-DE099A53A8B1} = {901F162F-8925-4390-89C5-9EE2C343F744}
{E09B72E4-D890-46A8-8D14-7367C2E23E9D} = {539D7B9A-18A1-4D79-86AB-C8B48090CA84}
{8561A35A-C2A4-43C7-A938-CB35A7747121} = {E09B72E4-D890-46A8-8D14-7367C2E23E9D}
{819E2756-7BEE-4FCD-BA08-7BF323405CBE} = {7BFFAE76-DEE9-417A-A79B-6A6644C4553A}
{B636A082-4DB4-439D-8A37-E5214BDC00A3} = {901F162F-8925-4390-89C5-9EE2C343F744}
{DE7579F2-C20F-4C35-BC04-C10362912243} = {901F162F-8925-4390-89C5-9EE2C343F744}
EndGlobalSection
Expand Down
17 changes: 17 additions & 0 deletions src/app/Fake.DotNet.Testing.Expecto/AssemblyInfo.fs
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
// Auto-Generated by FAKE; do not edit
namespace System
open System.Reflection

[<assembly: AssemblyTitleAttribute("FAKE - F# Make Running Expecto test runner")>]
[<assembly: AssemblyProductAttribute("FAKE - F# Make")>]
[<assembly: AssemblyVersionAttribute("5.0.0")>]
[<assembly: AssemblyInformationalVersionAttribute("5.0.0-beta025")>]
[<assembly: AssemblyFileVersionAttribute("5.0.0")>]
do ()

module internal AssemblyVersionInformation =
let [<Literal>] AssemblyTitle = "FAKE - F# Make Running Expecto test runner"
let [<Literal>] AssemblyProduct = "FAKE - F# Make"
let [<Literal>] AssemblyVersion = "5.0.0"
let [<Literal>] AssemblyInformationalVersion = "5.0.0-beta025"
let [<Literal>] AssemblyFileVersion = "5.0.0"
135 changes: 135 additions & 0 deletions src/app/Fake.DotNet.Testing.Expecto/Expecto.fs
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
/// Contains tasks to run [Expecto](https://github.com/haf/expecto) unit tests.
module Fake.DotNet.Testing.Expecto
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should add RequireQualifiedAccess here


open Fake.Core
open Fake.Core.Environment
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RequireQualifiedAccess was added to Environment in rc005

Copy link
Contributor Author

@jackfoxy jackfoxy Apr 15, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think more than happened to RequireQualifiedAccess Environment.

I now get this runtime error

Your environment variables look like they are set manually, but you are missing the default variables. Use the `Process.` helpers to change the 'Environment' field to inherit default values! See https://github.com/fsharp/FAKE/issues/1776#issuecomment-365431982

I had to change env variable handling.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was introduced somewhere in the beta phase. This will prevent incorrect usage of the API. Basically you almost always want to inherit the default environment variables, but you replaced them with your own map -> this error occurs.

open Fake.Core.StringBuilder
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RequireQualifiedAccess was added to StringBuilder in rc005

open Fake.Testing.Common
open System

/// CLI parameters available if you use Tests.runTestsInAssembly defaultConfig argv in your code:
type ExpectoParams =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As it is part of the Expecto-module we could

  • either call it Params (if we want to, users would then use Expecto.Params)
  • move it out of the Expecto module and leave it ExpectoParams

What do you think? We probably even should have some guideline around this and make it more uniform in fake 6

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shortening to Params.

{
/// Extra verbose output for your tests.
Debug : bool
/// Run all tests in parallel. Default is true.
Parallel : bool
/// Number of parallel workers (defaults to the number of logical processors).
ParallelWorkers : int
/// Prints out summary after all tests are finished.
Summary : bool
/// Prints out summary after all tests are finished including their source code location.
SummaryLocation : bool
/// Fails the build if focused tests exist. Default is true
FailOnFocusedTests : bool
/// Filter a specific hierarchy to run.
Filter : string
/// Filter a specific test case to run.
FilterTestCase : string
/// Filter a specific test list to run.
FilterTestList : string
/// Run only provided tests.
Run : string list
/// Doesn't run tests, print out list of tests instead.
ListTests : bool
/// Custom arguments to use in the case the helper not yet supports them
CustomArgs: string list
/// Prints the version on startup. Default is true
PrintVersion : bool
/// Working directory
WorkingDirectory : string
}

override this.ToString() =
let append (s: string) (sb: StringBuilder) = sb.Append s
let appendIfTrue value s sb =
if value then append s sb else sb
let appendIfNotNullOrWhiteSpace value s (sb: StringBuilder) =
if String.IsNullOrWhiteSpace value |> not
then sprintf "%s%s " s value |> sb.Append
else sb
let appendIfNotEqual other value s (sb: StringBuilder) =
if other = value
then sprintf "%s%A " s value |> sb.Append
else sb
let appendList list s (sb: StringBuilder) =
let filtered = list |> List.filter (String.IsNullOrWhiteSpace >> not)
if List.isEmpty filtered then sb
else
filtered |> String.separated " " |> sprintf "%s%s " s |> sb.Append
StringBuilder()
|> appendIfTrue this.Debug "--debug "
|> appendIfTrue this.Parallel "--parallel "
|> appendIfNotEqual 0 this.ParallelWorkers "--parallel-workers "
|> appendIfTrue this.FailOnFocusedTests "--fail-on-focused-tests "
|> appendIfTrue this.Summary "--summary "
|> appendIfTrue this.SummaryLocation "--summary-location "
|> appendIfTrue (not this.Parallel) "--sequenced "
|> appendIfTrue this.PrintVersion "--version "
|> appendIfTrue this.ListTests "--list-tests "
|> appendIfNotNullOrWhiteSpace this.Filter "--filter "
|> appendIfNotNullOrWhiteSpace this.FilterTestCase "--filter-test-case "
|> appendIfNotNullOrWhiteSpace this.FilterTestList "--filter-test-list "
|> appendList this.Run "--run "
|> appendList this.CustomArgs ""
|> toText

static member DefaultParams =
{
Debug = false
Parallel = true
ParallelWorkers = 0
Filter = ""
FilterTestCase = ""
FailOnFocusedTests = true
FilterTestList = ""
PrintVersion = true
Run = []
ListTests = false
// Summary = true somehow breakes Expecto TeamCity printer
Summary = false
SummaryLocation = false
CustomArgs = []
WorkingDirectory = ""
}

let run (setParams : ExpectoParams -> ExpectoParams) (assemblies : string seq) =
let details = assemblies |> String.separated ", "
use __ = Trace.traceTask "Expecto" details

let runAssembly testAssembly =
let exitCode =
let fakeStartInfo testAssembly args =
let workingDir =
if String.isNotNullOrEmpty args.WorkingDirectory
then args.WorkingDirectory else Fake.IO.Path.getDirectory testAssembly
(fun (info: ProcStartInfo) ->
{ info with
FileName = testAssembly
Arguments = string args
WorkingDirectory = workingDir
UseShellExecute = false
// Pass environment variables to the expecto console process in order to let it detect it's running on TeamCity
// (it checks TEAMCITY_PROJECT_NAME <> null specifically).
Environment = environVars() |> Map } )
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just leave it out? I think this is the default.


let execWithExitCode testAssembly argsString timeout =
Process.execSimple (fakeStartInfo testAssembly argsString) timeout

execWithExitCode testAssembly (setParams ExpectoParams.DefaultParams) TimeSpan.MaxValue

testAssembly, exitCode

let res =
assemblies
|> Seq.map runAssembly
|> Seq.filter( snd >> (<>) 0)
|> Seq.toList

match res with
| [] -> ()
| failedAssemblies ->
failedAssemblies
|> List.map (fun (testAssembly,exitCode) -> sprintf "Expecto test of assembly '%s' failed. Process finished with exit code %d." testAssembly exitCode)
|> String.concat System.Environment.NewLine
|> FailedTestsException |> raise
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
<Project Sdk="Microsoft.NET.Sdk" ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup>
<TargetFrameworks>net46;netstandard2.0</TargetFrameworks>
<DefineConstants>$(DefineConstants);NO_DOTNETCORE_BOOTSTRAP</DefineConstants>
<AssemblyName>Fake.DotNet.Testing.Expecto</AssemblyName>
<OutputType>Library</OutputType>
</PropertyGroup>
<PropertyGroup>
<DefineConstants>$(DefineConstants);NETSTANDARD;USE_HTTPCLIENT</DefineConstants>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)' == 'Release' ">
<DefineConstants>$(DefineConstants);RELEASE</DefineConstants>
</PropertyGroup>
<ItemGroup>
<Compile Include="AssemblyInfo.fs" />
<Compile Include="Expecto.fs" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\Fake.Core.Environment\Fake.Core.Environment.fsproj" />
<ProjectReference Include="..\Fake.Core.Trace\Fake.Core.Trace.fsproj" />
<ProjectReference Include="..\Fake.Core.Process\Fake.Core.Process.fsproj" />
<ProjectReference Include="..\Fake.Core.String\Fake.Core.String.fsproj" />
<ProjectReference Include="..\Fake.IO.FileSystem\Fake.IO.FileSystem.fsproj" />
<ProjectReference Include="..\Fake.Testing.Common\Fake.Testing.Common.fsproj">
<FromP2P>true</FromP2P>
</ProjectReference>
<ProjectReference Include="..\Fake.Core.Context\Fake.Core.Context.fsproj">
<FromP2P>true</FromP2P>
</ProjectReference>
</ItemGroup>
<Import Project="..\..\..\.paket\Paket.Restore.targets" />
</Project>
4 changes: 4 additions & 0 deletions src/app/Fake.DotNet.Testing.Expecto/paket.references
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
group netcore

FSharp.Core
NETStandard.Library