Searching for Chocolatey packages very slow #248
Comments
|
What happens if you run |
|
No. It took about 3 seconds when cold and about 1 second when warm. While BCU search for chocolatey packages tooks somewhere around a minute. |
|
Is it consistently this slow in BCU? The only thing I can think of is one
of the applications bein installed in a folder with a lot of files, and the
app size scan taking a long time. Do you happen to have Visual Studio
installed or know anything about profiling/debugging applications?
…On Sat, Dec 21, 2019 at 8:48 PM Can Altıparmak ***@***.***> wrote:
No. It took about 3 seconds when cold and about 1 second when warm. While
BCU search for chocolatey packages tooks somewhere around a minute.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#248?email_source=notifications&email_token=ADRZC4BNA5RV7UYPNPC6663QZZXJDA5CNFSM4J4ARWA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHPCLRA#issuecomment-568206788>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADRZC4ANNT62BNZO5OFZR5LQZZXJDANCNFSM4J4ARWAQ>
.
|
|
Yes, I have Visual Studio and have experience in debugging, but not any in profiling with it though. I am going to build a debug version of BCU tomorrow, and then try profiling. Is there anything I should know? |
|
Oh, nice. Profiling in VS is pretty easy, just look for the start profiling
option in the debug menu. Try to start profiling and then stop as soon as
the list loading finishes, then save the results. Repeat after turning off
chocolatey scan and then compare the two profiling results to see what
causes the slowdown.
…On Tue, Dec 24, 2019 at 7:29 PM Can Altıparmak ***@***.***> wrote:
Yes, I have Visual Studio and have experience in debugging, but not any in
profiling with it though. I am going to build a debug version of BCU
tomorrow, and then try profiling. Is there anything I should know?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#248?email_source=notifications&email_token=ADRZC4BN3NFB6YQF7CGUGMLQ2JIITA5CNFSM4J4ARWA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHTQNQI#issuecomment-568788673>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADRZC4A6N47BEPE5KQ2P4TDQ2JIITANCNFSM4J4ARWAQ>
.
|
|
I couldn't build it. I have VS 2019, got the kloctools and build successfully. Now UninstallTools fails with the following. I checked the documentation and it seems these methods do not exist. Both v4.15 and master fails on me. |
|
My bad, I forgot to push the last commit for kloctools. You can pull the
latest version from https://sourceforge.net/projects/kloctoolslibrary/
…On Sat, Dec 28, 2019 at 9:44 AM Can Altıparmak ***@***.***> wrote:
I couldn't build it. I have VS 2019, got the kloctools and build
successfully. Now UninstallTools fails with the following. I checked the
documentation and it seems these methods do not exist.
8>C:\Users\can\_work\temp\Bulk-Crap-Uninstaller\source\UninstallTools\Junk\Finders\Drive\CommonDriveJunkScanner.cs(57,70,57,93): error CS1061: 'DirectoryInfo' does not contain a definition for 'GetNameWithoutExtension' and no accessible extension method 'GetNameWithoutExtension' accepting a first argument of type 'DirectoryInfo' could be found (are you missing a using directive or an assembly reference?)
8>C:\Users\can\_work\temp\Bulk-Crap-Uninstaller\source\UninstallTools\Junk\Finders\Drive\CommonDriveJunkScanner.cs(88,157,88,180): error CS1061: 'FileSystemInfo' does not contain a definition for 'GetNameWithoutExtension' and no accessible extension method 'GetNameWithoutExtension' accepting a first argument of type 'FileSystemInfo' could be found (are you missing a using directive or an assembly reference?)
Both v4.15 and master fails on me.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#248?email_source=notifications&email_token=ADRZC4EW26HFJXOJGFJDWODQ24GYVA5CNFSM4J4ARWA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHYFNQI#issuecomment-569398977>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADRZC4FWNGYUNNBWKDOBE6LQ24GYVANCNFSM4J4ARWAQ>
.
|
|
Thanks. That was fast. It builds now. Here it is. I am looking into it too, but I couldn't figure anything out of it. |
|
It looks like it's waiting for chocolatey to finish. Can you change your
build type from Release to Debug, run in profiler again, and this time
check which line of code exactly takes the longest time to finish? It
should be listed in the summary page.
…On Sat, Dec 28, 2019 at 12:08 PM Can Altıparmak ***@***.***> wrote:
Thanks. That was fast. It builds now.
Here it is. I am looking into it too, but I couldn't figure anything out
of it.
Report.zip
<https://github.com/Klocman/Bulk-Crap-Uninstaller/files/4006967/Report.zip>
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#248?email_source=notifications&email_token=ADRZC4AXV5YFEMEPMMTY24LQ24XSLA5CNFSM4J4ARWA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHYHPFA#issuecomment-569407380>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADRZC4H6STEMLQX6MLTFTZLQ24XSLANCNFSM4J4ARWAQ>
.
|
|
With a debug build I still cannot see source code from other projects (i.e. |
|
I added some performance logging in recent commits, it should help narrow the issue if you still have it. The changes will be included in the upcoming update. The logs are written to the log file inside BCU directory. |
|
It looks like it's Chocolatey being slow after all. The issue is that it's difficult to test this because it's slow only the first time it scans for the applications, no matter if it's triggered by BCU or manually by command line. Then other times it's randomly very slow when started from command line. So far it looks like the issue is on the Chocolatey side, assuming it's not just normal operation that takes time. If you can confirm that running choco list from command line after a fresh system boot is slow I'll close the issue since it should be opened on the chocolatey repository instead. |
There may be a problem on Chocolatey side. However, It is never slow for me when run with |
|
BCU runs the exact same command
https://github.com/Klocman/Bulk-Crap-Uninstaller/blob/master/source/UninstallTools/Factory/ChocolateyFactory.cs#L68
It has to then ask for details for all applications, so maybe that's where
it gets slow.
…On Mon, Jan 27, 2020 at 5:16 PM Can Altıparmak ***@***.***> wrote:
- choco list, cold *01:03.101* -- about a minute after reboot
- choco list, warm *01:01.222* -- subsequent runs still slow
- choco list -l -nocolor -y -r, cold *00:05.859* - less than 6 seconds
immediately after reboot
- choco list -l -nocolor -y -r, warm *00:01.177* - then consistently
about a second
There may be a problem on Chocolatey side. However, It is never slow for
me when run with --localonly. So I think it should not be slow when
called from BCU.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#248?email_source=notifications&email_token=ADRZC4HRX4EBEKK6JPP7N7TQ74CEDA5CNFSM4J4ARWA2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKACTJI#issuecomment-578824613>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADRZC4DQQFKXY56DY2JBYALQ74CEDANCNFSM4J4ARWAQ>
.
|
Yes, that is where it gets slow. Thank you, extended logging is very useful. As you can see every chocolatey command took over a second. Totaling up to a nearly 2 minutes of time with many packages. It seems chocolatey has a |
|
I'm thinking about utilizing the chocolatey.lib library so that it would be possible to get rid of parsing. However, it would add dependency of chocolatey.lib and related log4net. Is it okay for you? |
|
I'm thinking about utilizing the chocolatey.lib library so that it would
be possible to get rid of parsing. However, it would add dependency of
chocolatey.lib and related log4net. Is it okay for you?
It sounds like a bit of an overkill, but it should be fine if it's light
enough in size and is noticeably faster.
…On Tue, Sep 21, 2021 at 3:19 PM Zafer Balkan ***@***.***> wrote:
I'm thinking about utilizing the chocolatey.lib library so that it would
be possible to get rid of parsing. However, it would add dependency of
chocolatey.lib and related log4net. Is it okay for you?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#248 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADRZC4GZH47GTJ3OU6CGLLDUDCA6XANCNFSM4J4ARWAQ>
.
Triage notifications on the go with GitHub Mobile for iOS
<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android
<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.
|
|
Well, I wrote a benchmark application to compare Also, when I use BCUninstaller, it takes more than 4 minutes to query all chocolatey installations -currently 68 packages. That's why I felt the urge to improve the query performance. Process creation overhead is really a PITA. By the way, you can give my Benchmark Repo a shot for performance issues. It would be great to get a review from you. |
|
@Klocman why not try Possibly it would not be as fast as utilizing |
|
@c6p, maybe you can run a benchmark or even send a PR. It would be nice to see the difference. And yes, removing the overhead of process creation makes a difference almost every time. |
|
@zbalkan since my comment in May 2020, I have switched over to scoop from chocolatey. Seeing your discussion I decided to chime in. I will try to make a PR, though I cannot benchmark it anymore. I will let you know when I've made the changes in my repo, could you benchmark both? |
|
@c6p Of course, if you send a PR, I could try it. |
|
@zbalkan sorry for the wait, I just found the time. Here it is, could you benchmark it? |
|
@c6p That's great. But I don't have access to my laptop right now. I might try tomorrow. However, you can check my benchmark project. You can do that yourself too since it will cover %80 of your use case. You can find the link above. |
|
@zbalkan Since I do not use chocolatey anymore, I don't have many packages, I have installed 2 to test, choco lists 5 packages. Thanks for the benchmark, it is very nice. Single call strategy is not as fast as library strategy as expected, but I think it is pretty acceptable, since quite possibly it will run under a second, irrespective of number of packages a user has.
Your benchmark would give us a better idea of trade-offs. |
|
@c6p The benchmark takes about 10 hours and here are the latest results. BenchmarkDotNet=v0.13.1, OS=Windows 10.0.18363.1801 (1909/November2019Update/19H2)
AMD A10-9600P RADEON R5, 10 COMPUTE CORES 4C+6G, 1 CPU, 4 logical and 4 physical cores
[Host] : .NET Framework 4.8 (4.8.4400.0), X64 RyuJIT [AttachedDebugger]
Job-AJPCRR : .NET Framework 4.8 (4.8.4400.0), X64 RyuJIT
IterationCount=100 LaunchCount=1 RunStrategy=Monitoring
WarmupCount=5
|
|
@c6p You probably need to make a PR. Although my method has significant improvement in time, the chocolatey.lib adds some odd complexities that occur time to time, which can be called Heisenbug. I need to check the actual problem and probably make the PR to Chocolatey repository first. Your suggestion is simpler and smooth. |
|
@zbalkan Done. Though, your solution does not depend on output of another program and may be more stable in the long term if you can resolve the issues. |
|
It turns out that the problem is about the target frameworks, as mentioned in the chocolatey/choco#2393. Bulk Crap Uninstaller targets dotnet 5 while Chocolatey targets 4. My solution is not available until chocolatey.lib is upgraded, meaning that the PR by @c6p is the only applicable solution for now. |
|
Thank you! I'll take c6p's solution for now since it's already a big improvement. If using chocolatey.lib becomes viable in the future then we can switch to that if need be. |
If I disable searching for chocolatey packages, startup time is acceptable. But with chocolatey enabled it is very slow and sometimes hangs at startup.
Version: 4.15.0.42702
Chocolatey v0.10.15
Windows 10 Pro - version 1909, build 18363.535
The text was updated successfully, but these errors were encountered: