Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

over an hour for paket update #607

Closed
ctaggart opened this issue Feb 13, 2015 · 16 comments
Closed

over an hour for paket update #607

ctaggart opened this issue Feb 13, 2015 · 16 comments
Labels
bug performance

Comments

@ctaggart
Copy link
Contributor

paket has been churning away on an update for over an hour now and is not complete. It has been hovering at about 26% CPU. The exploring lines are slowly being printed by the minute. What can be done to improve performance?
image

We have 44 dependenies listed in paket.dependencies along with two sources:

source https://nuget.org/api/v2
source https://ci.appveyor.com/nuget/tachyus... username: ... password: ...

Current output:

PS C:\Tachyus\e> .\.paket\paket.exe update --force --hard
Paket version 0.27.16.0
found: C:\Tachyus\e\paket.dependencies
Resolving packages:
    - exploring Microsoft.SqlServer.Types 11.0.1.1
    - exploring FsControl 1.0.9
    - exploring canopy 0.9.21
    - exploring Selenium.Support 2.44.0
    - exploring Selenium.WebDriver 2.44.0
    - exploring SizSelCsZzz 0.3.36.0
  - fetching versions for Newtonsoft.Json
    - exploring Newtonsoft.Json 6.0.8
  - fetching versions for FSharp.Core
    - exploring FSharp.Core 4.0.0
    - exploring FSharp.Core 3.1.2.1
  - fetching versions for xunit
    - exploring xunit 2.0.0-rc1-build2826
    - exploring xunit.assert 2.0.0-rc1-build2826
    - exploring xunit.core 2.0.0-rc1-build2826
    - exploring xunit.extensibility.core 2.0.0-rc1-build2826
    - exploring xunit.abstractions 2.0.0-rc1-build2826
    - exploring xunit.extensibility.execution 2.0.0-rc1-build2826
  - fetching versions for Dyfrig
    - exploring Dyfrig 0.8.0
  - fetching versions for Microsoft.Net.Http
    - exploring Microsoft.Net.Http 2.2.28
  - fetching versions for Microsoft.Bcl
    - exploring Microsoft.Bcl 1.1.9
  - fetching versions for Microsoft.Bcl.Build
    - exploring Microsoft.Bcl.Build 1.0.21
  - fetching versions for Elmah.AzureTableStorage
    - exploring Elmah.AzureTableStorage 0.6.2.0
  - fetching versions for WindowsAzure.Storage
    - exploring WindowsAzure.Storage 4.3.0
  - fetching versions for Microsoft.Data.OData
    - exploring Microsoft.Data.OData 5.6.3
    - exploring Microsoft.Data.Edm 5.6.3
    - exploring System.Spatial 5.6.3
  - fetching versions for Microsoft.Data.Services.Client
    - exploring Microsoft.Data.Services.Client 5.6.3
  - fetching versions for Microsoft.WindowsAzure.ConfigurationManager
    - exploring Microsoft.WindowsAzure.ConfigurationManager 2.0.3
  - fetching versions for elmah
    - exploring elmah 1.2.2.1
  - fetching versions for elmah.corelibrary
    - exploring elmah.corelibrary 1.2.2
  - fetching versions for Elmah.Contrib.WebApi
    - exploring Elmah.Contrib.WebApi 1.0.9.0
  - fetching versions for Microsoft.AspNet.WebApi.Core
    - exploring Microsoft.AspNet.WebApi.Core 5.2.3
  - fetching versions for Microsoft.AspNet.WebApi.Client
    - exploring Microsoft.AspNet.WebApi.Client 5.2.3
  - fetching versions for ExtCore
    - exploring ExtCore 0.8.45
  - fetching versions for FAKE
    - exploring FAKE 3.17.0
  - fetching versions for FParsec
    - exploring FParsec 1.0.1
  - fetching versions for FSharp.Azure
    - exploring FSharp.Azure 1.0.1
  - fetching versions for FSPowerPack.Linq.Community
    - exploring FSPowerPack.Linq.Community 3.0.0.0
  - fetching versions for FSPowerPack.Core.Community
    - exploring FSPowerPack.Core.Community 3.0.0.0
  - fetching versions for FSharp.Configuration
    - exploring FSharp.Configuration 0.4.10
  - fetching versions for FSharp.Data
    - exploring FSharp.Data 2.1.1
  - fetching versions for Zlib.Portable
    - exploring Zlib.Portable 1.10.0
  - fetching versions for FSharp.Data.SqlClient
    - exploring FSharp.Data.SqlClient 1.5.9
  - fetching versions for FSharp.Formatting
    - exploring FSharp.Formatting 2.7.4
  - fetching versions for FSharpVSPowerTools.Core
    - exploring FSharpVSPowerTools.Core 1.7.0
  - fetching versions for FSharp.Compiler.Service
    - exploring FSharp.Compiler.Service 0.0.82
  - fetching versions for FSharpx.Collections
    - exploring FSharpx.Collections 1.9.6
  - fetching versions for Fleece
    - exploring Fleece 0.4.0
    - exploring ReadOnlyCollectionInterfaces 1.0.0
  - fetching versions for System.Json
    - exploring System.Json 4.0.20126.16343
  - fetching versions for ReadOnlyCollectionExtensions
    - exploring ReadOnlyCollectionExtensions 1.2.0
  - fetching versions for LinqBridge
    - exploring LinqBridge 1.3.0
  - fetching versions for FSharpPlus
    - exploring FSharpPlus 0.0.4
  - fetching versions for FsCheck
    - exploring FsCheck 2.0.0
    - exploring FsCheck 1.0.4
  - fetching versions for FsCheck.Xunit
    - exploring FsCheck.Xunit 1.0.4
    - exploring FsCheck.Xunit 1.0.3
    - exploring FsCheck.Xunit 1.0.2
    - exploring FsCheck.Xunit 1.0.1
    - exploring FsCheck.Xunit 1.0.0
    - exploring FsCheck.Xunit 0.4.1.0
    - exploring FsCheck.Xunit 0.4.0.2
    - exploring FsCheck.Xunit 0.4.0.1
    - exploring FsCheck.Xunit 0.4.0.0
    - exploring FsCheck.Xunit 0.3.0.0
    - exploring FsCheck.Xunit 0.2.0.0
    - exploring FsCheck.Xunit 0.1.0.0
    - exploring FsCheck 1.0.3
    - exploring FsCheck 1.0.2
    - exploring FsCheck 1.0.1
    - exploring FsCheck 1.0.0
    - exploring FsCheck 0.9.4.0
    - exploring FsCheck 0.9.3.0
    - exploring FsCheck 0.9.2.0
    - exploring FsCheck 0.9.1.0
    - exploring FsCheck 0.9.0.1
    - exploring FsCheck 0.9.0.0
    - exploring FsCheck 0.8.3.0
    - exploring FsCheck 0.8.2.0
    - exploring FsCheck 0.8.1.0
    - exploring FsCheck 0.8.0.0
    - exploring FsCheck 0.7.1
    - exploring Fleece 0.3.0
    - exploring Fleece 0.2.0
    - exploring FSharpPlus 0.0.3
    - exploring ReadOnlyCollectionExtensions 1.1.0
    - exploring ReadOnlyCollectionExtensions 1.0.0
    - exploring Fleece 0.1.0
    - exploring FSharpx.Collections 1.9.5
    - exploring FSharpx.Collections 1.9.4
    - exploring FSharpx.Collections 1.9.3
    - exploring FSharpx.Collections 1.9.2
    - exploring FSharpx.Collections 1.9.1
    - exploring FSharpx.Collections 1.9.0
    - exploring FSharpx.Collections 1.8.89.0
    - exploring FSharpx.Collections 1.8.88.0
    - exploring FSharpx.Collections 1.8.87.0
    - exploring FSharpx.Collections 1.8.86.0
    - exploring FSharpx.Collections 1.8.85.0
    - exploring FSharpx.Collections 1.8.84.0
    - exploring FSharpx.Collections 1.8.83.0
    - exploring FSharpx.Collections 1.8.82.0
    - exploring FSharpx.Collections 1.8.81.0
    - exploring FSharpx.Collections 1.8.80.0
    - exploring FSharpx.Collections 1.8.79.0
    - exploring FSharpx.Collections 1.8.78.0
    - exploring FSharpx.Collections 1.8.77.0
    - exploring FSharpx.Collections 1.8.76.0
    - exploring FSharpx.Collections 1.8.75.0
    - exploring FSharpx.Collections 1.8.74.0
    - exploring FSharpx.Collections 1.8.73.0
    - exploring FSharpx.Collections 1.8.72.0
    - exploring FSharpx.Collections 1.8.71.0
    - exploring FSharpx.Collections 1.8.70.0
    - exploring FSharpx.Collections 1.8.69.0
    - exploring FSharpx.Collections 1.8.68.0
    - exploring FSharpx.Collections 1.8.65.0
    - exploring FSharpx.Collections 1.8.64.0
    - exploring FSharpx.Collections 1.8.63.0
    - exploring FSharpx.Collections 1.8.62.0
    - exploring FSharpx.Collections 1.8.61.0
    - exploring FSharpx.Collections 1.8.60.0
    - exploring FSharpx.Collections 1.8.59.0
    - exploring FSharpx.Collections 1.8.58.0
    - exploring FSharpx.Collections 1.8.57.0
    - exploring FSharpx.Collections 1.8.56.0
    - exploring FSharpx.Collections 1.8.55.0
    - exploring FSharpx.Collections 1.8.54.0
    - exploring FSharpx.Collections 1.8.53.0
    - exploring FSharpx.Collections 1.8.52.0
    - exploring FSharpx.Collections 1.8.49.0
    - exploring FSharpx.Collections 1.8.48.0
    - exploring FSharpx.Collections 1.8.47.0
    - exploring FSharpx.Collections 1.8.46.0
    - exploring FSharpx.Collections 1.8.45.0
    - exploring FSharp.Formatting 2.7.2
    - exploring FSharp.Formatting 2.7.1
    - exploring FSharp.Formatting 2.7.0
    - exploring FSharp.Formatting 2.6.3
    - exploring FSharp.Compiler.Service 0.0.81
    - exploring FSharp.Formatting 2.6.2
    - exploring FSharp.Formatting 2.6.1
    - exploring FSharp.Formatting 2.6.0
    - exploring FSharp.Formatting 2.5.1
    - exploring RazorEngine 3.3.0
    - exploring Microsoft.AspNet.Razor 2.0.30506
    - exploring FSharp.Formatting 2.5.0
    - exploring FSharp.Formatting 2.4.38
    - exploring FSharp.Formatting 2.4.37
    - exploring FSharp.Compiler.Service 0.0.67
    - exploring FSharp.Formatting 2.4.36
    - exploring FSharp.Formatting 2.4.35
    - exploring FSharp.Formatting 2.4.34

What does exploring mean? Why are so many versioned explored?
@forki
Copy link
Member

forki commented Feb 13, 2015

Wow that's really bad perf. I wonder why CPU is used at all. Usually bad
perf is a result of the slow nuget api, but this would be seen as CPU be
basically idle. So it seems something else is going on. Can you give me a
chance to profile this?

Regarding "exploring". It basically means that we have to look at the next
version of a package since the last one doesn't fit the version
restrictions. So if you see paket exploring a lot of different versions of
the same package it basically means that you (or packages you use) defined
very narrow version restrictions to that package.
On Feb 13, 2015 10:02 PM, "Cameron Taggart" notifications@github.com
wrote:

paket has been churning away on an update for over an hour now and is
not complete. It has been hovering at about 26% CPU. The exploring lines
are slowly being printed by the minute. What can be done to improve
performance?
[image: image]
https://cloud.githubusercontent.com/assets/80104/6195014/d0f69d82-b37e-11e4-8fcf-a034c0949dd3.png

We have 44 dependenies listed in paket.dependencies along with two
sources:

source https://nuget.org/api/v2
source https://ci.appveyor.com/nuget/tachyus... username: ... password: ...

Current output:

PS C:\Tachyus\e> ..paket\paket.exe update --force --hard
Paket version 0.27.16.0
found: C:\Tachyus\e\paket.dependencies
Resolving packages:
- exploring Microsoft.SqlServer.Types 11.0.1.1
- exploring FsControl 1.0.9
- exploring canopy 0.9.21
- exploring Selenium.Support 2.44.0
- exploring Selenium.WebDriver 2.44.0
- exploring SizSelCsZzz 0.3.36.0

  • fetching versions for Newtonsoft.Json
    • exploring Newtonsoft.Json 6.0.8
  • fetching versions for FSharp.Core
    • exploring FSharp.Core 4.0.0
    • exploring FSharp.Core 3.1.2.1
  • fetching versions for xunit
    • exploring xunit 2.0.0-rc1-build2826
    • exploring xunit.assert 2.0.0-rc1-build2826
    • exploring xunit.core 2.0.0-rc1-build2826
    • exploring xunit.extensibility.core 2.0.0-rc1-build2826
    • exploring xunit.abstractions 2.0.0-rc1-build2826
    • exploring xunit.extensibility.execution 2.0.0-rc1-build2826
  • fetching versions for Dyfrig
    • exploring Dyfrig 0.8.0
  • fetching versions for Microsoft.Net.Http
    • exploring Microsoft.Net.Http 2.2.28
  • fetching versions for Microsoft.Bcl
    • exploring Microsoft.Bcl 1.1.9
  • fetching versions for Microsoft.Bcl.Build
    • exploring Microsoft.Bcl.Build 1.0.21
  • fetching versions for Elmah.AzureTableStorage
    • exploring Elmah.AzureTableStorage 0.6.2.0
  • fetching versions for WindowsAzure.Storage
    • exploring WindowsAzure.Storage 4.3.0
  • fetching versions for Microsoft.Data.OData
    • exploring Microsoft.Data.OData 5.6.3
    • exploring Microsoft.Data.Edm 5.6.3
    • exploring System.Spatial 5.6.3
  • fetching versions for Microsoft.Data.Services.Client
    • exploring Microsoft.Data.Services.Client 5.6.3
  • fetching versions for Microsoft.WindowsAzure.ConfigurationManager
    • exploring Microsoft.WindowsAzure.ConfigurationManager 2.0.3
  • fetching versions for elmah
    • exploring elmah 1.2.2.1
  • fetching versions for elmah.corelibrary
    • exploring elmah.corelibrary 1.2.2
  • fetching versions for Elmah.Contrib.WebApi
    • exploring Elmah.Contrib.WebApi 1.0.9.0
  • fetching versions for Microsoft.AspNet.WebApi.Core
    • exploring Microsoft.AspNet.WebApi.Core 5.2.3
  • fetching versions for Microsoft.AspNet.WebApi.Client
    • exploring Microsoft.AspNet.WebApi.Client 5.2.3
  • fetching versions for ExtCore
    • exploring ExtCore 0.8.45
  • fetching versions for FAKE
    • exploring FAKE 3.17.0
  • fetching versions for FParsec
    • exploring FParsec 1.0.1
  • fetching versions for FSharp.Azure
    • exploring FSharp.Azure 1.0.1
  • fetching versions for FSPowerPack.Linq.Community
    • exploring FSPowerPack.Linq.Community 3.0.0.0
  • fetching versions for FSPowerPack.Core.Community
    • exploring FSPowerPack.Core.Community 3.0.0.0
  • fetching versions for FSharp.Configuration
    • exploring FSharp.Configuration 0.4.10
  • fetching versions for FSharp.Data
    • exploring FSharp.Data 2.1.1
  • fetching versions for Zlib.Portable
    • exploring Zlib.Portable 1.10.0
  • fetching versions for FSharp.Data.SqlClient
    • exploring FSharp.Data.SqlClient 1.5.9
  • fetching versions for FSharp.Formatting
    • exploring FSharp.Formatting 2.7.4
  • fetching versions for FSharpVSPowerTools.Core
    • exploring FSharpVSPowerTools.Core 1.7.0
  • fetching versions for FSharp.Compiler.Service
    • exploring FSharp.Compiler.Service 0.0.82
  • fetching versions for FSharpx.Collections
    • exploring FSharpx.Collections 1.9.6
  • fetching versions for Fleece
    • exploring Fleece 0.4.0
    • exploring ReadOnlyCollectionInterfaces 1.0.0
  • fetching versions for System.Json
    • exploring System.Json 4.0.20126.16343
  • fetching versions for ReadOnlyCollectionExtensions
    • exploring ReadOnlyCollectionExtensions 1.2.0
  • fetching versions for LinqBridge
    • exploring LinqBridge 1.3.0
  • fetching versions for FSharpPlus
    • exploring FSharpPlus 0.0.4
  • fetching versions for FsCheck
    • exploring FsCheck 2.0.0
    • exploring FsCheck 1.0.4
  • fetching versions for FsCheck.Xunit
    • exploring FsCheck.Xunit 1.0.4
    • exploring FsCheck.Xunit 1.0.3
    • exploring FsCheck.Xunit 1.0.2
    • exploring FsCheck.Xunit 1.0.1
    • exploring FsCheck.Xunit 1.0.0
    • exploring FsCheck.Xunit 0.4.1.0
    • exploring FsCheck.Xunit 0.4.0.2
    • exploring FsCheck.Xunit 0.4.0.1
    • exploring FsCheck.Xunit 0.4.0.0
    • exploring FsCheck.Xunit 0.3.0.0
    • exploring FsCheck.Xunit 0.2.0.0
    • exploring FsCheck.Xunit 0.1.0.0
    • exploring FsCheck 1.0.3
    • exploring FsCheck 1.0.2
    • exploring FsCheck 1.0.1
    • exploring FsCheck 1.0.0
    • exploring FsCheck 0.9.4.0
    • exploring FsCheck 0.9.3.0
    • exploring FsCheck 0.9.2.0
    • exploring FsCheck 0.9.1.0
    • exploring FsCheck 0.9.0.1
    • exploring FsCheck 0.9.0.0
    • exploring FsCheck 0.8.3.0
    • exploring FsCheck 0.8.2.0
    • exploring FsCheck 0.8.1.0
    • exploring FsCheck 0.8.0.0
    • exploring FsCheck 0.7.1
    • exploring Fleece 0.3.0
    • exploring Fleece 0.2.0
    • exploring FSharpPlus 0.0.3
    • exploring ReadOnlyCollectionExtensions 1.1.0
    • exploring ReadOnlyCollectionExtensions 1.0.0
    • exploring Fleece 0.1.0
    • exploring FSharpx.Collections 1.9.5
    • exploring FSharpx.Collections 1.9.4
    • exploring FSharpx.Collections 1.9.3
    • exploring FSharpx.Collections 1.9.2
    • exploring FSharpx.Collections 1.9.1
    • exploring FSharpx.Collections 1.9.0
    • exploring FSharpx.Collections 1.8.89.0
    • exploring FSharpx.Collections 1.8.88.0
    • exploring FSharpx.Collections 1.8.87.0
    • exploring FSharpx.Collections 1.8.86.0
    • exploring FSharpx.Collections 1.8.85.0
    • exploring FSharpx.Collections 1.8.84.0
    • exploring FSharpx.Collections 1.8.83.0
    • exploring FSharpx.Collections 1.8.82.0
    • exploring FSharpx.Collections 1.8.81.0
    • exploring FSharpx.Collections 1.8.80.0
    • exploring FSharpx.Collections 1.8.79.0
    • exploring FSharpx.Collections 1.8.78.0
    • exploring FSharpx.Collections 1.8.77.0
    • exploring FSharpx.Collections 1.8.76.0
    • exploring FSharpx.Collections 1.8.75.0
    • exploring FSharpx.Collections 1.8.74.0
    • exploring FSharpx.Collections 1.8.73.0
    • exploring FSharpx.Collections 1.8.72.0
    • exploring FSharpx.Collections 1.8.71.0
    • exploring FSharpx.Collections 1.8.70.0
    • exploring FSharpx.Collections 1.8.69.0
    • exploring FSharpx.Collections 1.8.68.0
    • exploring FSharpx.Collections 1.8.65.0
    • exploring FSharpx.Collections 1.8.64.0
    • exploring FSharpx.Collections 1.8.63.0
    • exploring FSharpx.Collections 1.8.62.0
    • exploring FSharpx.Collections 1.8.61.0
    • exploring FSharpx.Collections 1.8.60.0
    • exploring FSharpx.Collections 1.8.59.0
    • exploring FSharpx.Collections 1.8.58.0
    • exploring FSharpx.Collections 1.8.57.0
    • exploring FSharpx.Collections 1.8.56.0
    • exploring FSharpx.Collections 1.8.55.0
    • exploring FSharpx.Collections 1.8.54.0
    • exploring FSharpx.Collections 1.8.53.0
    • exploring FSharpx.Collections 1.8.52.0
    • exploring FSharpx.Collections 1.8.49.0
    • exploring FSharpx.Collections 1.8.48.0
    • exploring FSharpx.Collections 1.8.47.0
    • exploring FSharpx.Collections 1.8.46.0
    • exploring FSharpx.Collections 1.8.45.0
    • exploring FSharp.Formatting 2.7.2
    • exploring FSharp.Formatting 2.7.1
    • exploring FSharp.Formatting 2.7.0
    • exploring FSharp.Formatting 2.6.3
    • exploring FSharp.Compiler.Service 0.0.81
    • exploring FSharp.Formatting 2.6.2
    • exploring FSharp.Formatting 2.6.1
    • exploring FSharp.Formatting 2.6.0
    • exploring FSharp.Formatting 2.5.1
    • exploring RazorEngine 3.3.0
    • exploring Microsoft.AspNet.Razor 2.0.30506
    • exploring FSharp.Formatting 2.5.0
    • exploring FSharp.Formatting 2.4.38
    • exploring FSharp.Formatting 2.4.37
    • exploring FSharp.Compiler.Service 0.0.67
    • exploring FSharp.Formatting 2.4.36
    • exploring FSharp.Formatting 2.4.35
    • exploring FSharp.Formatting 2.4.34

What does exploring mean? Why are so many versioned explored?

—
Reply to this email directly or view it on GitHub
#607.

@ctaggart
Copy link
Contributor Author

I cloned this repo and did a couple of performance wizard runs on it. The first one for CPU usage.

image

image

Being that performance gets slower as it goes when I run it, it feels like a memory issues. I ran the memory performance wizard too, but the analysis process is taking a while to churn through an 8 GB log file. I'll report back when I get it.

@ctaggart
Copy link
Contributor Author

The memory performance wizard brings up the same code.

image

image

cpu .vspx 9 MB
memory .vsp compressed with .7z 57 MB, 7 GB uncompressed

@ctaggart
Copy link
Contributor Author

I was trying to install xunit 2 with nuget xunit prerelease with the above. I've got a reproducible case with strange behavior.

https://github.com/ctaggart/Paket09x/tree/collections
git clone https://github.com/ctaggart/Paket09x.git -b collections

source https://nuget.org/api/v2
nuget FSharpx.Collections
nuget FsCheck.Xunit
nuget xunit prerelease

PS C:\Projects\Paket09x> .\paket.exe update
Paket version 0.27.16.0
found: C:\Projects\Paket09x\paket.dependencies
Resolving packages:
  - fetching versions for xunit
    - exploring xunit 2.0.0-rc1-build2826
    - exploring xunit.assert 2.0.0-rc1-build2826
    - exploring xunit.core 2.0.0-rc1-build2826
    - exploring xunit.extensibility.core 2.0.0-rc1-build2826
    - exploring xunit.abstractions 2.0.0-rc1-build2826
    - exploring xunit.extensibility.execution 2.0.0-rc1-build2826
  - fetching versions for FSharpx.Collections
    - exploring FSharpx.Collections 1.9.6
  - fetching versions for FsCheck.Xunit
    - exploring FsCheck.Xunit 1.0.4
    - exploring FsCheck.Xunit 1.0.3
    - exploring FsCheck.Xunit 1.0.2
    - exploring FsCheck.Xunit 1.0.1
    - exploring FsCheck.Xunit 1.0.0
    - exploring FsCheck.Xunit 0.4.1.0
    - exploring FsCheck.Xunit 0.4.0.2
    - exploring FsCheck.Xunit 0.4.0.1
    - exploring FsCheck.Xunit 0.4.0.0
    - exploring FsCheck.Xunit 0.3.0.0
    - exploring FsCheck.Xunit 0.2.0.0
    - exploring FsCheck.Xunit 0.1.0.0
    - exploring FSharpx.Collections 1.9.5
    - exploring FSharpx.Collections 1.9.4
    - exploring FSharpx.Collections 1.9.3
    - exploring FSharpx.Collections 1.9.2
    - exploring FSharpx.Collections 1.9.1
    - exploring FSharpx.Collections 1.9.0
    - exploring FSharpx.Collections 1.8.89.0
    - exploring FSharpx.Collections 1.8.88.0
    - exploring FSharpx.Collections 1.8.87.0
    - exploring FSharpx.Collections 1.8.86.0
    - exploring FSharpx.Collections 1.8.85.0
    - exploring FSharpx.Collections 1.8.84.0
    - exploring FSharpx.Collections 1.8.83.0
    - exploring FSharpx.Collections 1.8.82.0
    - exploring FSharpx.Collections 1.8.81.0
    - exploring FSharpx.Collections 1.8.80.0
    - exploring FSharpx.Collections 1.8.79.0
    - exploring FSharpx.Collections 1.8.78.0
    - exploring FSharpx.Collections 1.8.77.0
    - exploring FSharpx.Collections 1.8.76.0
    - exploring FSharpx.Collections 1.8.75.0
    - exploring FSharpx.Collections 1.8.74.0
    - exploring FSharpx.Collections 1.8.73.0
    - exploring FSharpx.Collections 1.8.72.0
    - exploring FSharpx.Collections 1.8.71.0
    - exploring FSharpx.Collections 1.8.70.0
    - exploring FSharpx.Collections 1.8.69.0
    - exploring FSharpx.Collections 1.8.68.0
    - exploring FSharpx.Collections 1.8.65.0
    - exploring FSharpx.Collections 1.8.64.0
    - exploring FSharpx.Collections 1.8.63.0
    - exploring FSharpx.Collections 1.8.62.0
    - exploring FSharpx.Collections 1.8.61.0
    - exploring FSharpx.Collections 1.8.60.0
    - exploring FSharpx.Collections 1.8.59.0
    - exploring FSharpx.Collections 1.8.58.0
    - exploring FSharpx.Collections 1.8.57.0
    - exploring FSharpx.Collections 1.8.56.0
    - exploring FSharpx.Collections 1.8.55.0
    - exploring FSharpx.Collections 1.8.54.0
    - exploring FSharpx.Collections 1.8.53.0
    - exploring FSharpx.Collections 1.8.52.0
    - exploring FSharpx.Collections 1.8.49.0
    - exploring FSharpx.Collections 1.8.48.0
    - exploring FSharpx.Collections 1.8.47.0
    - exploring FSharpx.Collections 1.8.46.0
    - exploring FSharpx.Collections 1.8.45.0
    - exploring xunit 2.0.0-beta5-build2785
    - exploring xunit 2.0.0-beta4-build2738
    - exploring xunit 2.0.0-beta-build2700
    - exploring xunit 2.0.0-alpha-build2606
    - exploring xunit 1.9.2
  - fetching versions for FsCheck
    - exploring FsCheck 2.0.0
    - exploring FsCheck 1.0.4
Locked version resolutions written to C:\Projects\Paket09x\paket.lock
2 seconds - ready.

@forki
Copy link
Member

forki commented Feb 14, 2015

After a quick look I think there are two things that produce this strange
behavior.

  1. there is a issue with the prerelease version requirement for xunit.
    Somehow the fscheck.xunit dependency to xunit conflicts. I think the
    resolver is too strict about that and that's why it tries out all
    combinations.

  2. a memory issue seems to be in the code which needs to be analyzed
    further
    On Feb 14, 2015 7:15 AM, "Cameron Taggart" notifications@github.com wrote:

I was trying to install xunit 2 with nuget xunit prerelease with the
above. I've got a reproducible case with strange behavior.

https://github.com/ctaggart/Paket09x/tree/collections
git clone https://github.com/ctaggart/Paket09x.git -b collections

source https://nuget.org/api/v2
nuget FSharpx.Collections
nuget FsCheck.Xunit
nuget xunit prerelease

PS C:\Projects\Paket09x> .\paket.exe update
Paket version 0.27.16.0
found: C:\Projects\Paket09x\paket.dependencies
Resolving packages:

  • fetching versions for xunit
    • exploring xunit 2.0.0-rc1-build2826
    • exploring xunit.assert 2.0.0-rc1-build2826
    • exploring xunit.core 2.0.0-rc1-build2826
    • exploring xunit.extensibility.core 2.0.0-rc1-build2826
    • exploring xunit.abstractions 2.0.0-rc1-build2826
    • exploring xunit.extensibility.execution 2.0.0-rc1-build2826
  • fetching versions for FSharpx.Collections
    • exploring FSharpx.Collections 1.9.6
  • fetching versions for FsCheck.Xunit
    • exploring FsCheck.Xunit 1.0.4
    • exploring FsCheck.Xunit 1.0.3
    • exploring FsCheck.Xunit 1.0.2
    • exploring FsCheck.Xunit 1.0.1
    • exploring FsCheck.Xunit 1.0.0
    • exploring FsCheck.Xunit 0.4.1.0
    • exploring FsCheck.Xunit 0.4.0.2
    • exploring FsCheck.Xunit 0.4.0.1
    • exploring FsCheck.Xunit 0.4.0.0
    • exploring FsCheck.Xunit 0.3.0.0
    • exploring FsCheck.Xunit 0.2.0.0
    • exploring FsCheck.Xunit 0.1.0.0
    • exploring FSharpx.Collections 1.9.5
    • exploring FSharpx.Collections 1.9.4
    • exploring FSharpx.Collections 1.9.3
    • exploring FSharpx.Collections 1.9.2
    • exploring FSharpx.Collections 1.9.1
    • exploring FSharpx.Collections 1.9.0
    • exploring FSharpx.Collections 1.8.89.0
    • exploring FSharpx.Collections 1.8.88.0
    • exploring FSharpx.Collections 1.8.87.0
    • exploring FSharpx.Collections 1.8.86.0
    • exploring FSharpx.Collections 1.8.85.0
    • exploring FSharpx.Collections 1.8.84.0
    • exploring FSharpx.Collections 1.8.83.0
    • exploring FSharpx.Collections 1.8.82.0
    • exploring FSharpx.Collections 1.8.81.0
    • exploring FSharpx.Collections 1.8.80.0
    • exploring FSharpx.Collections 1.8.79.0
    • exploring FSharpx.Collections 1.8.78.0
    • exploring FSharpx.Collections 1.8.77.0
    • exploring FSharpx.Collections 1.8.76.0
    • exploring FSharpx.Collections 1.8.75.0
    • exploring FSharpx.Collections 1.8.74.0
    • exploring FSharpx.Collections 1.8.73.0
    • exploring FSharpx.Collections 1.8.72.0
    • exploring FSharpx.Collections 1.8.71.0
    • exploring FSharpx.Collections 1.8.70.0
    • exploring FSharpx.Collections 1.8.69.0
    • exploring FSharpx.Collections 1.8.68.0
    • exploring FSharpx.Collections 1.8.65.0
    • exploring FSharpx.Collections 1.8.64.0
    • exploring FSharpx.Collections 1.8.63.0
    • exploring FSharpx.Collections 1.8.62.0
    • exploring FSharpx.Collections 1.8.61.0
    • exploring FSharpx.Collections 1.8.60.0
    • exploring FSharpx.Collections 1.8.59.0
    • exploring FSharpx.Collections 1.8.58.0
    • exploring FSharpx.Collections 1.8.57.0
    • exploring FSharpx.Collections 1.8.56.0
    • exploring FSharpx.Collections 1.8.55.0
    • exploring FSharpx.Collections 1.8.54.0
    • exploring FSharpx.Collections 1.8.53.0
    • exploring FSharpx.Collections 1.8.52.0
    • exploring FSharpx.Collections 1.8.49.0
    • exploring FSharpx.Collections 1.8.48.0
    • exploring FSharpx.Collections 1.8.47.0
    • exploring FSharpx.Collections 1.8.46.0
    • exploring FSharpx.Collections 1.8.45.0
    • exploring xunit 2.0.0-beta5-build2785
    • exploring xunit 2.0.0-beta4-build2738
    • exploring xunit 2.0.0-beta-build2700
    • exploring xunit 2.0.0-alpha-build2606
    • exploring xunit 1.9.2
  • fetching versions for FsCheck
    • exploring FsCheck 2.0.0
    • exploring FsCheck 1.0.4
      Locked version resolutions written to C:\Projects\Paket09x\paket.lock
      2 seconds - ready.

—
Reply to this email directly or view it on GitHub
#607 (comment).

@forki forki added the bug label Feb 14, 2015
@ctaggart
Copy link
Contributor Author

Why are all the FSharpx.Collections being explored? Without prerelease on xunit it doesn't explore them all, just the one.

PS C:\Projects\Paket09x> .\paket.exe update
Paket version 0.27.16.0
found: C:\Projects\Paket09x\paket.dependencies
Resolving packages:
  - fetching versions for FSharpx.Collections
    - exploring FSharpx.Collections 1.9.6
  - fetching versions for FsCheck.Xunit
    - exploring FsCheck.Xunit 1.0.4
  - fetching versions for xunit
    - exploring xunit 1.9.2
  - fetching versions for FsCheck
    - exploring FsCheck 2.0.0
    - exploring FsCheck 1.0.4
Locked version resolutions written to C:\Projects\Paket09x\paket.lock

May be prerelease here is causing some sort of recursion problem.

@forki
Copy link
Member

forki commented Feb 14, 2015

As I wrote above it's detecting a conflict between prerelease requirement
and non-prerelease requirement. This is too strict and I'm going to fix it
this weekend.
On Feb 14, 2015 9:21 AM, "Cameron Taggart" notifications@github.com wrote:

Why are all the FSharpx.Collections being explored? Without prerelease on
xunit it doesn't explore them all, just the one.

PS C:\Projects\Paket09x> .\paket.exe update
Paket version 0.27.16.0
found: C:\Projects\Paket09x\paket.dependencies
Resolving packages:

  • fetching versions for FSharpx.Collections
    • exploring FSharpx.Collections 1.9.6
  • fetching versions for FsCheck.Xunit
    • exploring FsCheck.Xunit 1.0.4
  • fetching versions for xunit
    • exploring xunit 1.9.2
  • fetching versions for FsCheck
    • exploring FsCheck 2.0.0
    • exploring FsCheck 1.0.4
      Locked version resolutions written to C:\Projects\Paket09x\paket.lock

May be prerelease here is causing some sort of recursion problem.

—
Reply to this email directly or view it on GitHub
#607 (comment).

@forki
Copy link
Member

forki commented Feb 14, 2015

Quick question in order to tackle part 2) of this. What profiler tools do you use?

@isaacabraham
Copy link
Contributor

Could look at the use of nessos stream library, which has some large performance gains over seq operations. Not sure about set though.

From: Cameron Taggartmailto:notifications@github.com
Sent: ‎14/‎02/‎2015 05:14
To: fsprojects/Paketmailto:Paket@noreply.github.com
Subject: Re: [Paket] over an hour for paket update (#607)

The memory performance wizard brings up the same code.

image

image

cpu .vspx 9 MB
memory .vsp compressed with .7z 57 MB, 7 GB uncompressed

Reply to this email directly or view it on GitHub:
#607 (comment)

forki added a commit that referenced this issue Feb 14, 2015
@forki
Prerelease requirements in paket.dependencies should override package…
ff5235a 
… dependencies - references #607
@forki
Copy link
Member

forki commented Feb 14, 2015

Ok I released a hotfix which should relax the prerelease requirements. Please check if this works for you.

I will still keep this issue open since I really want to analyze the memory issue.

@forki
Copy link
Member

forki commented Feb 14, 2015

Regarding streams: I started with lists, then went over to sets since I needed uniqueness. Then I changed the order in the set. So in the I think we need a priority queue for the open dependencies and a set for the closed dependencies.

@isaacabraham
Copy link
Contributor

Yeah - I am thinking more "streams rather than lots of chained Seq. operations" than replacements for Set operations.

@ctaggart
Copy link
Contributor Author

Quick question in order to tackle part 2) of this. What profiler tools do you use?

I used the profiler that comes with Visual Studio 2013 Ultimate. Select Analyze > Performance and Diagnostis > Performance Wizard.

@forki
Copy link
Member

forki commented Feb 15, 2015

it looks like we are no longer tail recursive. I will try to fix that.

Did the quickfix help?

@ctaggart
Copy link
Contributor Author

Yes, thank you!

@forki
Copy link
Member

forki commented Feb 16, 2015

closing this, but I'm still refactoring the code

@forki forki closed this as completed Feb 16, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug performance
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@forki @ctaggart @isaacabraham