Reworking the Organs tab

[oleg68]

There are several issues related to working with a list of the organs and organ packages.

• Organs list disappeared #786
• Problem with Organs cache and File menu options #886
• Deletion conflict with organ packages in Settings #950

After implementing #948 there will be one tab Organs.

The basic idea is to have the single list of registered organs with additional features.

I want to get the end users' opinions what information they want to see in the common list and what actions they want to have in this window.

Especially I ask @paolo2504 and @larspalo to share their ideas here.

[paolo2504]

Oleg, i reviewed #948 past days and i agree with almost everything. HOwever, i believe these changes, maybe not difficult to apply, are many so i'll review quietly the all stuff to be able to give you a reasonable anwser.
At first, however, i wish to summarize the first couple of things that arise in my mind,

1. to be able to identify the cache files and the organs they belong to
2. automatic deletion of cache/cbm files when organs are deleted/unregistered from Settings (no matter they are package organs or not)
3. to be able to install organ packages no matter compression used and extension (rar/zip/gzip); otherwise, i guess that the function "install package" is useless. I mean: most samplesets available on the net (P.Grabowsky,countrebombarde,odfgrandorgue and so on) are compressed with normal or high ratio and come with .zip or .rar extension (.exe on odfgrandorgue French site, in this case you have to execute to unpack and create the folder structure. Note: i have still not found how to directly unpack these files...).
   However, GO only supports 0-ratio compression (no compression) for files with .orgue extension. This limits, imho, the real advantage to use the function "install organ package".
   Note: i believe that this functionality, if fixed and optimized, would be of great help as opposite to "File/open". If you fix this function, then you can have File/Load, Install package (or even toggle it because GO does authomatically recognize .orgue files even under the Samplesets folder set in Options tab).
4. Option File/open recent , to be honest, was never used by me and could be removed

[larspalo]

1,2. is my general thoughts of how I think it should be done, not so much how it currently works.
3. Read the links I provided again carefully. Almoast no new format "organ packages" are available, sadly enough. (I have provided PiteaMHS and BureaChoir) Most other "packages" of GO samplesets only qualify as "old style" samplesets and no, they cannot be installed, nor do they qualify as "organ packages" in GO terms. In time I'll either re-work some of my older samplesets or re-record them completely and distribute them as new style GO organ packages.

[vpoguru]

The GO packages format is fantastic! Gernot Wurst (PROSPECTUM) recently published the Zoeblitz sample set as GO organ package.

[jason-barba]

( @larspalo I have re-worked my extension of your Kalvträsk extension to an organ package if you are interested. )

[larspalo]

@jason-barba Thanks, but I'll intend to also re-work the samples for that. Ideally, I'd like to completely re-record Kalvträsk organ, as I really enjoy that small gem and think it's worth the best of the best...

[jason-barba]

Great, looking forward to that. It is the one organ/sample set I use on my VPO :-)

[paolo2504]

Lars, to demonstrate how much I appreciate the “organ package” feature I say I’ll rework my samplesets as soon as the grandorguetool for Windows will be working.
At present it does not.
I have also tried winrar and 7zip (using 0-ratio compression that means store only) but GO is getting error on parsing content (“no end record found”).
As an alternative, I would greatly appreciate (in addition to have a working GrandOrgueTool) let GO read and manage directly the rar/zip files that we all find on internet.
Thanks.

[paolo2504]

Yes and … yes !
This afternoon I’ll work on this ..😃

[jason-barba]

That's really weird than. Two identical files should be handled the same by the program.

[paolo2504]

Got it ! I'll keep you on thorns for a little bit more... It was really tricky thing... i'm ashamed..

[jason-barba]

You should be ashamed. I stroggled with GrandOrgueTool a while a go. @larspalo help me with the subtle difference between .\path\filename and path\filename. It was a lot of paths in the ODF I used ...

[oleg68]

I have also tried winrar and 7zip (using 0-ratio compression that means store only) but GO is getting error on parsing content (“no end record found”).

Even the 0-ratio formats of winrar and 7zip formats differ the one used in .orgue

As an alternative, I would greatly appreciate (in addition to have a working GrandOrgueTool) let GO read and manage directly the rar/zip files that we all find on internet.

It may be difficult with the license restrictions related to using rar format.

[larspalo]

@oleg68 Do you think we should add the information on how to build an .orgue package manually (#955 (reply in thread)) to the wiki page https://github.com/GrandOrgue/grandorgue/wiki/Organ-package-creation-(GrandOrgueTool) and split that page into two halves, the first with using GrandOrgueTool and the second with the manual instructions?

[larspalo]

For example, in Burea_funeral... all reference are like Pipe001=.\Subbas16\036-C.wav , not Pipe001=Subbas16\036-C.wav as you suggest.

They all (the paths) need to be changed like I suggested. I of course noticed that while testing the errors @jason-barba got with GrandOrgueTool.

Pay very close attention to exactly how the .zip is created. The whole sampleset should be zipped in the root with no additional subfolder appearing first (ie. not having first BureaFuneralChapel as the first directory in the .zip file - the whole root directory of the sampleset must be directly available in the zip file and there the .organ file and organindex.ini file must be present (in the root). Next thing is the archive only - no compression extra with -0. I repeat the point nbr 4 above:

4. Pack the whole sampleset root directory into a zip archive. For instance one can (in the root directory) issue the following command:
   zip -r ../BureaFuneralChapel . -0

I attach two pictures of how the .zip archive layout should be, and not be so that you can compare. I'm pretty sure your .zip layout is wrong.

[paolo2504]

Nono, my zip layout is correct: today, after much testing , I finally understood what you say.
However, it’s not clear where and how the .organ file should be changed. Basically I don’t change it: I only build and add to the root the organ index file.
Pls, which modifications have to be made to the .organ file ?

Also, you mentioned some testing by user Jason but, please, keep in mind I don’t have any documentation on grandorguetool, I cannot run it with option -h so I don’t really have many information.
from post you so gently created for us in wiki section i now know that grandorguetool is able to wavpack files… again, I had to download wavpack r5.40 (latest) and build myself a batch script to do the job…
You speak of -o1 -o2 etc… parameters but I don’t know what they do…

finally, I am really interested in knowing how fast is packing with grandorguetool because the speed of wavpack, as far as I see, is too much slow….

[larspalo]

The .organ file should pass a "Perform strict ODF" check in the options without any log issues to begin with. (Any sampleset producer should always have that option enabled) Correct any reported log issues first. Then search and replace any =.\ or =./ paths to just =.

A very common issue for many windows users is case sensitivity (or rather the in-sensitivity)... Be sure to check that, too!

Unless I have access to the whole sampleset I cannot test it myself and say for sure exactly what needs to be changed, but the above notes "should" be enough...

[paolo2504]

I read right now your reply. Thanks. I was testing and i can say now it is FIXED. Problem was that i didn't realize that many if not almost every samplesets i used come with .organ files with path with "Linux/Unix" sysntax. For example Path1/Path2/Path3 while in Windows backslash is used that is Path1\Path2\Path3. In addition many entries were preceeded by ., thus making impossible or GO locate path.

aufff... Now i can use winrar to build packages. Tomorrow i'll test my script for wavpacking samplesets and try to set best parameters to speedup process that now is really too much slow.
If it took 2 hours to pack wav files with Pekela organ (1.5gb) i cannot even imagine how much it would take for example with Friesach (17.5gb).

[jason-barba]

Great.

[oleg68]

There are some feature requests here for the known organ list. It's a time to think how the features should be presented from the GUI point of view.

After merging the Organs and Organ Packages tabs the new tab looks like

Because an organ has lots of attributes and all attributes can not fit in a single line of a table, I'm going to do the classic master-detail view.

The upper part of the page will contain the table of organs (both packaged and unpackaged in the same table). Only few properties will be shown in the table: Name and a few other.

The downer part - various fields displaying all other properties of the organ selected in the upper part.

At the bottom of the page will be the buttons for the different actions on the selected organs.

Some actions (for example, scan cache) will not correspond the current organ.

My questions:

1. What columns should be in the table of organs?
2. What other properties should be shown in the downer part?
3. What actions should be available as buttons?

[larspalo]

1. I think that most of the info that can be found in the (loaded) organ properties dialog instead could be removed. Thus, we should keep: Church (name), MIDI, Organ package in the upper part, I think. It should perhaps be called: "Registered organs/packages"
2. For the lower information details view we could for instance have: Path, Id, Caches, Combinations/Settings. The info column doesn't seem to have anything there mostly at the moment. Thinking of it, I don't think the lower part should be implemented as a list at all, but as a panel with different widgets (maybe with separate listboxes for caches and settings files related to that organ)
3. The buttons for re-arranging the (upper) list order should be re-worked in the implementation so that the moved item will be visible in the list at all times and not disappearing like now. Ideally I'd like the possibility to delete an organ (together with everything related to it!) separately from the option of deleting any specifically selected cache or settings/combination file. The handling of deleting organs in a package vs not maybe needs some thought though (offer the option to remove the whole package or not). A fast and simple option for cleaning/removing all detected orphaned files/organs/packages would be a great addition too.

[paolo2504]

As suggested by Oleg I put here a new request.
I would like to have Ctrl/Shift enabled to make multiple selection when deleting organs / packages in the Settings dialog.
thanks.

[paolo2504]

I don't know if this is the right place to post enhancement request... anyway..

I would like to have ascending/descending sort capability on any column in the Settings/Organs tab.
regards.
paolo

[paolo2504]

question: are you eventually in mind to renew also the GO system bar itself ? In that case, i would personally appreciate having a couple of new icons on the left : Load Organs and Install Organ Package for example (just before 'Set' and Memory level.

thanks

[oleg68]

This topic only relates to the Organs tab. Please open a separate topic for discussing enchancements the main toolbar

[ahall41]

• Make it easier to move organs up/down the list by dragging, rather than repeatedly hitting Up.
• If you have a Top button, there should also be one for Bottom.
• Purge all MISSING organs
• The ability to edit the labels might be useful
• I use Open | Recent a lot. Please keep it!

[oleg68]

The work is in progress now. This is an intermediate variant of the Organs tab:

[paolo2504]

Please, what organ hash, package hash, package id, package info stand for ?
tks

[oleg68]

Organ Hash corresponds the name of the cache and the preset files.
Package id is used for matching between Organs and Packages now.
Package hash seems is not used now.

[oleg68]

@paolo2504

You have submitted most of issues about the Organs list.

Could you try the intermediate version of GrandOrgue with the Organs tab reworked https://github.com/oleg68/GrandOrgue-official/actions/runs/2602469146?

[paolo2504]

Ok.

…

Il giorno 3 lug 2022, alle ore 18:04, Oleg Samarin ***@***.***> ha scritto:  @paolo2504 You have submitted most of issues about the Organs list. Could you try the intermediate version of GrandOrgue with the Organs tab reworked https://github.com/oleg68/GrandOrgue-official/actions/runs/2602469146? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[paolo2504]

Here below attached my thoughts in 12 points (at the beginning I copied last q&a between Oleg and myself).

hints on new reworked setting in GrandOrgue.txt

[paolo2504]

Oleg, after further review i found a couple of issues on delete functions. Here follows detail.

a) Delete cache: cache files are deleted WITHOUT confirmation
b) Delete preset: here confirmation Yes/No is correctly shown but the word 'delete' is missing in the message
c) Delete: caches are deleted only AFTER clicking on Ok to exit the Settings dialog. Actually, cache files are correctly deleted. I just wonder if
the deletion after click on Ok button is by design (note that Delete cache and Delete preset are carried on immediately.
Shortly speaking:
Delete cache and preset -> cache files are deleted and a log mesagge dialog is shown;
Delete -> nothing happens until Ok button is selected: after that cache files are deleted and the log message is shown.
Note: when Delete is selected, organs disappear from the list. If then you press Cancel, caches are not deleted and reentering Settings
organs are still there. This is correct.
d) Delete: .cache and .cmb files are deleted, .idx ones not. I think this has to be fixed.
Note: .idx files are created only with package organs. I found this little bug only because i only use packages otherwise i would have not
noticed it.

[oleg68]

Oleg, after further review i found a couple of issues on delete functions. Here follows detail.

a) Delete cache: cache files are deleted WITHOUT confirmation

Because the cache files can be easy restored without a big problem

b) Delete preset: here confirmation Yes/No is correctly shown but the word 'delete' is missing in the message

already fixed

c) Delete: caches are deleted only AFTER clicking on Ok to exit the Settings dialog. Actually, cache files are correctly deleted. I just wonder if the deletion after click on Ok button is by design (note that Delete cache and Delete preset are carried on immediately. Shortly speaking: Delete cache and preset -> cache files are deleted and a log mesagge dialog is shown; Delete -> nothing happens until Ok
button is selected: after that cache files are deleted and the log message is shown. Note: when Delete is selected, organs disappear from the list. If then you press Cancel, caches are not deleted and reentering Settings organs are still there. This is correct.

It is by design: otherwise Cancel could not revere the side effect of Delete.

d) Delete: .cache and .cmb files are deleted, .idx ones not. I think this has to be fixed. Note: .idx files are created only with package organs. I found this little bug only because i only use packages otherwise i would have not noticed it.

Yes, it is to be fixed.

[paolo2504]

I forgot mention that even Top, Down, Up functions are now ok. Good job.

[oleg68]

@paolo2504 @larspalo Thank you for your feedbacks.

We should decide now, what is to change before #1141 may be merged.

The main points relate to deletion of cache and presets:

• The buttons are enabled regardless the actual existance of the files. May be to check? Just caching would not be sufficient: we also have to remember changes after deletion and relocations.
• These buttons are the only that act immediately. In contrast all other actions that are performed only after pressing to OK and are not performed after Cancel. May be we should not delete cache and presets until OK?
• Lack of deletion of orphaned caches and presets.

A possible solution is to discovering remember all existing preset and cache files on opening this dialog and marking what are to keep or not to keep. The result may be used for proper enabling/disabling the button. Theis task is not so simple: we need to take into account previous pressing to Delete and Relocate buttons. Pressing the delete cache/presets button only is to change the mark. The actual deletion will be only after exiting the dialog with OK. If several organs are selected, the delete cache/presets is to be enabled if ANY of the organ has existing cache/presets file.

@larspalo @paolo2504, is this idea acceptable? I would implement it. It will take about a week for me.

[larspalo]

I absolutely think this is acceptable.

[oleg68]

Another to do list:

• Add explanation of organ fields to the help
• Separate the list of packages to delete in the confirmation question with new lines instead of commas
• Add the Del hotkey for deletion
• Automatically focuse a line in the organ list and to show the properties after open and after delete
• Delete index files after deleting packages

[larspalo]

Very good further improvements!

[oleg68]

@larspalo

Nor is the additional organs from a package removed when I remove the main organ of the package, but if I then remove the additional organ it asks me if I want to remove the main organ too - which I already had done. This could perhaps be improved upon.

There is a many-to-one relationship between organs and organ packages. There is no any main organ. After you delete all the organs from the package, you will be asked, if you want to remove the package, not the main organ. And you do not have an option not to delete the package: only not to delete organs that reference this package. Any orphaned packages are removed automatically.

[larspalo]

Right, it was poorly expressed by me - in that case, one of the organs (that I consider "main") had the same name as the package. However, it still is a possible confusion point we somehow need to clarify and document well.

[paolo2504]

Oleg, a couple of thoughts still on deletion.

1. Deleted Packages are not actually deleted.
   Lars correctly says there is a many-to-one relationship between organs and organ packages. After you delete all the organs from the package, you will be asked, if you want to remove the package, not the main organ. This is absolutely correct, nothing to say on this.
   Nevertheless, i would note that packages are authomatically reloaded/registered if you put them (i mean, the orgue files) in the folder that you defined as "organ packages" in the Paths tab.
   This is causing a misleading behaviour imho; if you delete a package in Settings/Organs then you reopen Settings they are still there. The same with File/Load. Definitely, .cache, .cmb, presets, .idx (when fixed) are correctly deleted but the packages are still there.
   Solution: eliminate the option to auto recognize package organs.
   I know it is an early feature in GO but i don'ts see at present the advantage of maintaining it. Eliminating this feature you should also avoid the other issue regarding having many big .orgue files in the Package path: in this case, the first time you launch GO it could take hours to complete. This is the reason i always set an empty folder for packages and keep them in another location.

2. i personally woud enjoy having a new column "Package name" in the upper side of Settings/Organs tab.
   This to allow see which organs belong to which package. At present the organ name can be see only in the fields below.
   It could be usefull for example when deleting organs to see how many organs still belong to which package.

[larspalo]

@paolo2504 @oleg68 I think that we definitely should keep the automatic registration of packages from the packages folder - it's by far the easiest way for new/casual user of GO to start playing with a certain .orgue package without any issues.

For more advanced users, such as yourself perhaps?, they should already know that if they simply remove the package from the packages folder and place them elsewhere and instead "install" the package - they can keep listing only what organs from a package they want.

I'm quite satisfied with seeing what organ belong to which package down below in the information textfields. The only issue I ran into is if the package was registered with an earlier version of GO they didn't contain that information, but it's easily fixed with just re-registering it.

All such issues described can easily be addressed with proper documentation of expected behavior and best practices.

[paolo2504]

Thanks for reply Lars. Regarding automatic registration i know you love it. Me too at the beginning.. I don't think it's a matter of being or not adv user... It is a fact that if you put tens of big packages in a folder and set that folder as Package path, then you get into a big trouble first time you launch GO. I tested this many times before understanding the reason, and every time i ended up in killing the running GO application from task manager. As an old developer, i don't like this and i would prefer to find a solution in the coding itself. Anyway you're right saying the solution is simple: just keep packages in a folder and NOT set it as Package folder (that's a nonsense by me).

Just a final note. With automatic registration package organs deleted with the new Delete button only disappear until the Settings tab is closed but still show up when Settings is reopened. A non-advanced user should not understand well this behaviour.
regards.

[oleg68]

@larspalo @paolo2504 I've finished working on the Organs tab. Could you try the new version from https://github.com/oleg68/GrandOrgue-official/actions/runs/2682478835 ?

1. Nevertheless, i would note that packages are authomatically reloaded/registered if you put them (i mean, the orgue files) in the folder that you defined as "organ packages" in the Paths tab.

Now Delete detects this case and brings the warning box.

[paolo2504]

Three notes.

1. DEL key doesn't work (i understood it was enabled)
2. .idx files are now deleted but only if .orgue files are outside the Package organ folder. For example, DEMO organ idx is delete (but it
   doesn't reside in the packages folder...); also, package organs saved outside correctly get .idx files deleted.
3. Error 5: access denied is logged when deleting open package organs. In this case, both .cache and .idx files are not removed.
   I suggest checking if any organ is open and if it is one of those going to be deleted: in this case not to proceed with deleting
   that specific organ.
   note: problem is that caches and idxs become orphan files because their matching organs don't show up anymore in
   Settings.

note: .cache files of samplesets placed in the Sampleset folder are correctly deleted even if the organs are open.
Definitely, the access denied error arise only with package organs. We have two cases: 1) p.organs in package folder; 2) p.organs outside package folder. In both cases, .cache and .idx files are not removed if they are open during delete (error5 arises); what differs is that case 1) .idx and .cache files are rebuilt next time GO restarts (due to the auto discovery) while in case 2) the organs are removed from Settings and .idx and .cache become orphaned files.

[oleg68]

1. Confirm. It works on Linux and OsX, but not on Windows. I'll research.
2. The .idx file is deleted when you press OK file, but it appears again when GrandOrgue starts next time, because the package is autoregistered. It is desired behaviour.

the organs are removed from Settings and .idx and .cache become orphaned files.

When you enter the Organ Settings and exit with OK the orphaned files will be deleted.

I'd not like to change anything with deleting an open packaged organ.

[paolo2504]

I know cache files are deleted after clicking Ok.
problem is that if you try to delete an organ that is open you get error 5 access denied if it’s a package (no error if it’s a sampleset).

When GO is restarted, idx is recreated but this happens only for packages that are located in the package folder. Only those packages are auto registered.
Problem arises when package are outside the package folder. In this case, idx and cache are never recreated. As you have deleted them, they don’t appear anymore in Settings/Organs. Definitely the cache and idx files become orphan.

You can reproduce this way:

1. Install a package that is located outside the package folder
2. Load it
3. Open Settings/organs and delete that organ
4. press Ok and you get the error5 access denied in the log window.
5. Close Go and restart; then open Settings/organs. Correctly , the organ is not there because only packages in the defined folder are auto registered.
6. Now check the GrandorgueCache folder and you’ll find the orphaned files.

what I suggest Is simple: check if organs going to be deleted are open ; in that case, if they’re located in the package folder do nothing and proceed otherwise ask user to close the organ before deleting it (or force close before deletion).

[oleg68]

@paolo2504 I finished working on reorganising the Organs tab. The official intermediate version is 3.7.0-0.3 available for download at https://github.com/GrandOrgue/grandorgue/actions/runs/2762988899

The only problem that still persists is the error when deleting the organ that is currently open from the cache file. I created a separate issue for managing cache of the open organ on Windows: #1147. It requires a more complex solution.

The workaround of the cache orphaned files is to close the current organ or to restart GrandOrgan, to go to File->Settings and to press OK. The old orpaned cache files will be deleted.

[rousseldenis]

@paolo2504 I finished working on reorganising the Organs tab. The official intermediate version is 3.7.0-0.3 available for download at https://github.com/GrandOrgue/grandorgue/actions/runs/2762988899

The only problem that still persists is the error when deleting the organ that is currently open from the cache file. I created a separate issue for managing cache of the open organ on Windows: #1147. It requires a more complex solution.

@oleg68 Is there a mean to know which organ is open and to deactivate the organ selection for delete?

[oleg68]

Now it is not possible without a lot of coding: the Settings dialog does not know which organs are open.