Command line parameters

[softnorbi]

Hello Viktor,
I've just found your wonderful tool and I'm glad you've started developing a lightweight replacement for Geotagger but open source which I prefer as it's more future proof. My use of GTN would be in conjunction with a plugin for Lightroom 6.14, where Adobe's map module is no longer working/supported. However, this use case would require GTN to be able to accept command line parameters (similar to Geotagger). The Lightroom plugin would give GTN the filenames to open and GTN should write the geocoordinates into them. Would that be possible or can GTN already do that? Here is a link that describes what I mean in more detail:

https://geosetter.de/en/2018/05/10/lr-plugin-run-any-command-en/

Thanks again for your development of GTN. I hope you continue to work on it and I'm looking forward to the final version!
Regards,
Norbert

[nemethviktor]

Hey, glad you like it.
I'll try to add that in, but in the meanwhile, do you have any specific syntax that would be required?
e.g.:

• geotagninja.exe -w "c:\folder name\file name1.jpg", "c:\folder name\file name2.jpg" (enclosed) or
• geotagninja.exe -w c:\folder name\file name1.jpg, c:\folder name\file name2.jpg (just split by comma) or
• dunno?

[softnorbi]

According to the instruction sent, Geosetter uses the following two parameters:

/PARAMS=file

Where "file" means a file containing other Geosetter command line parameters.....

For the specific use case with Lightroom, the LR plugin creates a temporary file containing the full path of all images that have been selected by the user in the Lightroom window, one per line. The temporary file is deleted by the plugin after the command is executed (so I'm not able to check its real content...)

/COLLECTION

switches Geosetter into "collection mode", where it takes only the selected files from above (even from different paths)

Hopefuly I understood the instructions from the plugin and geosetter correctly :-)
Althoug "/collection" might not be nessesary if GTN would perform this as standard.

[nemethviktor]

Cool okay - I'm working on something atm (namely rewriting half of the underlying logic of the code and do away with datatables as much as possible) but once that's done I'll get on with this. (might be a bit of time though)

[nemethviktor]

Okay, initial thoughts, now that I've started working on this.
The good news is that what you're after is very likely possible. I might have something for you at some point next week.

Guess what's coming then....

I pointed it out somewhere else a while ago but contrary to what may seem obvious I'm not a C# developer and so this is my very first attempt at coding anything...ever. I'm a SQL/database dev. This isn't entirely a problem per se but GTN isn't really geared up to be a split "UI and CMD" thing. I could introduce parameters to the original exe file but I feel that would be a little odd, especially in the context you'd like. The source of the issue is that the code is monolithic because I'm a n00b and unaware of what's deemed good coding habits. Admittedly I'd still need to rewrite a lot of the methods regardless because they are dependent on the "main Form" being available. So I'm making a GTNCMD version.

A bit of the challenge is that - especially over the past two months - the code has evolved a lot to account for Custom Rules and City allocation logic, without which I think the product is just crap. The API is what it is (free, and it's a bit of a case of getting what one pays for) and I think power users would want that functionality as part of their workflows. On top of that @Urmel has kindly provided the basis for moving from a certain coding logic to a more reliable/sensible one, which also happens to be a lot more verbose (in terms of code) than the old one.

Now hypothetically the way this should be done from what I understand is that there should be basically 3 projects, a common class library set and then the UI and the CMD relying on the class libraries to avoid duplication and redundancy. This won't be the case here, certainly not initially. Admittedly I am not proficient in writing abstract classes or interfaces and so a lot of the code will redundantly reused (and to make things worse, with small modifications) in GTNCMD - this introduces the problem of having to maintain 2 repos and that since one is dependent on UI and the other is not there is likely to be some divergence in the code as time moves on. Also CMD won't need stuff that relates to (for example) time-shifting, amongst other things.

I'm therefore thinking, and admittedly looking at either of you (@softnorbi or @Urmel ) on the assumption that at least one of you know/s more about C# than I do, that once the initial version of CMD is out, someone could help create the common libraries/dll files and then the two projects could be redirected to read from those?

[softnorbi]

Thanks again for the effort you have made with this project! Unfortunately my active programming time was 35 years ago and only in C without object orientation and classes. So I'm sorry that I can't be of much help here with C# and therefore can rather only give my input from the user's point of view :-(

[Urmel]

Good evening @softnorbi, @nemethviktor,

let's recap on what actually is the use case you want to have supported @softnorbi.

1. You would like to select images in Lightroom
2. Lightroom (by some plugin) starts the geo tagging tool
3. In the tool you geotag just these images and close the tool again
4. Back in Lightroom, you have the updated meta data available

GeoSetter supports this by allowing to be started with command line params (description of some these can be found in GeoSetter -> Help -> White Papers --> Command Line). Using /Collection and giving it a list of files, you get to edit these passed files even if they are in different folders.

Now regarding how GTN could do this - some thoughts:

• Add cmd line param to start GTN in "File List" mode
• Depending on how it should be passed, GTN could either just accepts a list of files or a config file that contains this file list
• When this FileListMode is active, we do not start out with parsing the current dir into DEs, but just parse the file list into DEs and give that to the list view.
• Pain will be that I think, we have some positions, where the path to the image files is assumed to be the same for all in the view - we need to changed these (which would be good practice anyways)
• Furthermore, we would need to disable using some actions (track sync?) or have them behave depening on the file list versus dir mode (the directory location text box for example)

[softnorbi]

Hello @Urmel,
Thank you for your contribution to this project!
As far as I can tell, your summary seems correct and logical to me. Also the hint concerning the different file paths.

[nemethviktor]

Ah ok, that's easier to do :) - for some reason I thought you wanted a built in API of sorts . My bad. ☕
Either of us will have a look next week if that's ok.

[nemethviktor]

@Urmel thanks for the quick reply!

I think we don't need to replicate too much functionality. My understanding is that a file list (most likely sitting in a txt file that contains a one-file-per-line-list) needs to be passed into GTN. For each of those files, if they have coordinates then they get tagged and saved
Things we need:

• custom rules logic (already exists in the code)
• custom cities logic (already exists in the code)

Things we don't need

• stages (DE 1/2/3) [well, maybe nr 3 but even that just as a placeholder]
• offering multiple lines of data from the API
• anything else

Settings would be read as per normal from the sqlite database/file so any settings would need to be edited in the UI as normal.
Re: DEs need to be based on fileNameWithPath rather than fileNameWithoutPath, I think that's doable. Atm the folderName is semi-hardcoded based on what's in one of the relevant textboxes but the DE.FindElementByName (might have a different name, I'm not looking at it) already works on full-path basis.

[nemethviktor]

Hmmm actually...I've re-read the whole thing and .... @softnorbi do you need either

• a way to use the full UI functionality but with a predefined file list or
• a purely automated tagging in command line mode? (Ie feed files with coordinates and have the GTN tag and save them automatically?)

If the prior then @Urmel you're right it's a simple thing to do, if the latter that's a bit trickier.

I need more coffees lol.

[softnorbi]

@nemethviktor

Sorry for not replying here earlier! The CMD-only solution doesn't make sense to me, since my main use of GTN would be to assign coordinates to images so I need your map view! Alternatively to GTN, I could do the location information to the coordinates afterwards in Lightroom as well.

So the answer to your qustion is: I need the "full UI functionality but with a predefined file list".

There's one more point I'd like to bring up before you dive in. Since the program call of GTN including map via the LR-plugin is of course very slow, it would be good if GTN could remain open after the first plugin call and only delete its own file list after saving. Then only the new file with the new paths could be transferred with the next plugin call via the plugin reference file with the path informations (one per line), without reopening everything. But I have no idea how to tell the LR plugin that GTN has cleared its internal file list (after writing the coordinates) so that the plugin can clear its own file with the listed path references.
I hope my description is somehow understandable so far? My english is not native and therfore perhaps confusing ;-)

Translated with www.DeepL.com/Translator (free version)

[Urmel]

Hi @softnorbi,

There's one more point I'd like to bring up before you dive in. Since the program call of GTN including map via the LR-plugin is of course very slow, it would be good if GTN could remain open after the first plugin call and only delete its own file list after saving. Then only the new file with the new paths could be transferred with the next plugin call via the plugin reference file with the path informations (one per line), without reopening everything. But I have no idea how to tell the LR plugin that GTN has cleared its internal file list (after writing the coordinates) so that the plugin can clear its own file with the listed path references. I hope my description is somehow understandable so far? My english is not native and therfore perhaps confusing ;-)

Ok - now it gets quite interesting :) But this might involve some tricks. But your intention is that it does not take so long when hitting the LR command the second time, right?

Pitfalls might include that the manifest file is named differently each call or that LR (or the plugin) insists to wait until the last invokation of the external programm finishes. These 2 should be testeable with a small cmd file.

P.S.: Docu for the LR plugin seems to be here: http://regex.info/blog/lightroom-goodies/run-any-command

[Urmel]

@softnorbi,

here we go... you can extract the windows bat script and let it run from the plugin wiht the same params you would use for the GTN invokation.

Could you please perform the following test case (with the bat file configured like you would configure GTN in LR) and document your results here? This should give us more insight and an idea if / how things could be accomplished.

1. Invoke the plugin (i.e. have LR run the bat file) --> please document the output, DO NOT press enter
2. Switch back to LR and see if you can interact with it
3. Try to invoke the plugin a second time (while the other invokation is still open) --> please document the output,
4. Close both invocations (by pressing enter in the command windows)
5. Invoke the plugin again --> please document the output,

Thanks.

printargs.zip

[softnorbi]

Hi @Urmel,
I tried to open printargs.bat via the LR plugin with all the arguments I use with Geosetter (or in the future with GTN). Unfortunately the window doesn't stay open so I don't see anything. Or the plugin does not process *.bat files? To check this the printargs.bat could perhaps print the necessary information into a text file. But I don't know how to do such a file print command in a patch to provide the nessessary results...:-(

[nemethviktor]

@Urmel It might be easier to install a VM if you don't otherwise have one and just get a copy of LR Classic from piratebay or wherever. Debug-at-home :D
Btw, @softnorbi what's your OS's culture setting? (your Windows....what language is it set to? which region?)

[Urmel]

Hi @nemethviktor, @softnorbi

I'd rather stick to the batch file ;) So here's an update.
printargs.zip

You can change the path it writes to in the second line. At the moment it creates a txt file in C:\Temp (the directory must exist).

@echo off
set LOG_FILE=C:\Temp\LR-Test.txt <-----------HERE
echo Logging to %LOG_FILE%

All calls / logs are just added to that file (with date / time). So you do not need to take care of handling multiple of these.

Hope this helps.

[softnorbi]

Hi @Urmel, @nemethviktor

First of all thank you for the new patch and all the effort!!

I performed the sequence as requested however it seems that the patch is stil not abe to print into the file if executed by the plugin. It works however if I start it manually. Nevertheless the plugin execution generated some error messages and files, I would like to present you together with my performed sequence:

1. I opened LR and seected 3 pictures in the LR window. Then I started the plugin which responds with the following message:
   
   and the file "LR3" without extension is created in the path C:\Users\Arno\AppData\Local\Temp
   The content of LR3 per text editor is "Das System kann den angegebenen Pfad nicht finden."

After this I went back to LR press dismiss on the error message window and select other pictures and start the pugin again, with the following response:

and the file "LR5" without extension is created in the path C:\Users\Arno\AppData\Local\Temp
Its content per text editor is also "Das System kann den angegebenen Pfad nicht finden."

In addtion I found a logfile from the plugin which might perhaps be also helpful:

run-any-command-log.txt

However the patch file itself was not able to create the expected log file.

Hope this helps somehow to better understand whats going on.

I believe the old LR 6.14 version can still be downloaded as trial version from the Adobe page, but only with an adobe account. The plugin has a trial period as well. If required I could search for the links if you need it.

[nemethviktor]

Seems like the two of you are from the same part of the world. Wunderschön :)

I believe the old LR 6.14 version can still be downloaded as trial version from the Adobe page, but only with an adobe account. The plugin has a trial period as well. If required I could search for the links if you need it.

It's on various torrent sites too, LR Classic. I'll have a look one of these days. I keep a bunch of VMs for testing and whatever other purposes, I don't mind getting one set up for experimenting further.

[softnorbi]

@nemethviktor
sorry, it came to my mind, that I fully forgot to answer your system question because of the whole result explanation! I'm indeed from Austria and my system for running LR is Windows 10 :-) Running the torrent download in a VM is for sure a good idea!

[nemethviktor]

No worries. Btw if it makes life easier for the two of you do chat in German, I don't mind. I understand some of it but not proficient enough to communicate efficiently. I'll have a look at the LR-C tomorrow but Alex can also do some explorartory steps.
The 'culture' thing is mostly relevant in terms of just ... well I was curious and also some people have odd setups that I haven't tested. So the more the merrier with culture settings I guess.

[nemethviktor]

I think I got most of it.

1. Download (and install) the plugin from http://regex.info/blog/lightroom-goodies/run-any-command (i think you have that already)
2. Within LRC under "File | Plug-in Extras | jf Run Any Command | Configure..." -> Configure the plugin as such:
   "c:\Program Files\GeoTagNinja\GeoTagNinja.exe" --collection={MANIFEST} (where {MANIFEST} is a literal string, ie just type that in with the curly brackets).
3. Edit and interact with your files as usual. Don't forget to save!
4. When all's done, close GTN. At the moment it doesn't support being called twice with an updated file list. That may come later.

Test please: https://drive.google.com/file/d/18iI77SIdrIv-joOtyT0-MzqOVtB5OgM0/view?usp=share_link

I haven't uploaded a diff yet to the codebase because I am hoping that it'd get tested a little before it's posted but I've played around with CollectionMode and seems to do what it's supposed to.

[softnorbi]

@nemethviktor and @Urmel

Here a download link from Adobe for the original LR 6.14 version. It works as trial version for a limitted time.

https://helpx.adobe.com/lightroom-classic/help/lightroom-6-end-of-support.html

[nemethviktor]

@nemethviktor Here are the sample files (Sony arw). If put coordinates into the metadata via LR-plugin GTN call. The coordinates afterwards are visible in LR and in GTN "normal mode". However if I call GTN via plugin the coordinates aren't shown in GTN.

_DSC5885.zip

@softnorbi -- right, I see the problem. Fixed now, see the usual dev-link on my google drive.
If interested the problem was that in "normal" mode we parse a folder, which has image files and possible xmp files but we nonetheless go through the whole folder. In "collection" mode we parse a hardcoded list of files (as per the "LR-2" [or whichever] file).
When I had initially tested the functionality with my own dummy text files as source-list I added the xmp files as well but LR does not, so your list of files was just the _DSC5885.arw file and so the xmp file got ignored. Code has now been amended to check if an xmp file exists regardless of the file list being specific.

[softnorbi]

@nemethviktor - I can confirm that it works now on my PC👍👏
If I understand your description, GTN didn't get the info about the existing XMP files and therefor wasn't able to update them.
For me there is still the question why LR did show the coordinates update w/o GTN XML update? Did GTN write to the *.arw file instead of the XML? If so - why did GTN ignore the GTN settings just to write into XML files?

[softnorbi]

@nemethviktor - just send you a paypal bier 🍻

[nemethviktor]

@softnorbi thanks for the beer ;)
Regarding your question re: xmp read and write:

If the user sets "write to sidecar" setting the app will write to xmp, if the user sets "update the image file", it will update the image file. If you set neither then the app will throw a warning and do nothing.

Still, you bring up a valid point that I will need to look into. Basically the TLDR is that the 'read' process is a different thing from the 'write' process and what you correctly pointed out is that they are out of sync at the moment in terms of coding logic. The 'read' process creates a so-called dictionary, which is best thought of as a simple, two-element datastore: 'imageFileName' and a nullable (possibly empty) 'sideCarFileName'. When we read a location we assign those details into the dictionary and then depending on the existence of the elements we read data from one, the other, or both. (image/sc file). The recent update to the code has ascertained that sc files get parsed even when not explicitly stated in the plugin-output-text-file because historically we would have just added any xmp file in a folder but now we don't user folders (in 'collectionMode' anyway)

On the other hand 'write' hasn't been updated and what it still does is that when the user issues a 'save' command the name of the image file gets taken from the main 'listview' (file grid) and then the 'write' process yet again looks for the 'xmp' file in the folder where the image file is. This works across folders because the 'folderName' isn't hardcoded as such anymore and that's why it had updated the xmp files for you even when it didn't show.
Before the modification of the code for your issue (where xmp files were half ignored) the flawed logic was: parse location, read image file, not read (==not find [==ignore]) xmp file, user does modifications, user saves image file, app parses modifies image & xmp file according to settings, user sees modified stuff in LR. So the write was entirely async from read.

[nemethviktor]

@Urmel 4b96db4 last check-in just now.
TLDR:

• Updated codebase to work off GUIDs rather than lvi.name or item.name or some such. I think literally 1 place remains for the old logic perhaps. This would allow for multiple identical filenames across folders, though I haven't tested but it should serve as a basis nonetheless.
• collectionMode has been added in as per previous comments. Basically everything works in collectionMode, the only restriction is that the app is either in collectionMode or it is not, there is no switching around while the app is running (needs restart to switch)
• What needs doing, apart from testing is adding in logic to allow for 1 count of GTN to run (easy, from what I gather -> https://stackoverflow.com/a/6486341/3968494) and somehow manage a push-style update via LCR as per request in previous posts -- I have no current idea how to do that. Can you ponder on it pls?
• I've updated the readme a bit with a minor howto.
• There is an entirely unrelated bug where in Edit Form if user removes all data and then copy-pastes blanks and then clicks on another file the values become 0s rather than blanks. it's just to say i'm aware of it but haven't fixed.

[Urmel]

Hi all,

@softnorbi, can you confirm that the current approach works for calling GTN once and then receiving back the udpated data in LR without closing GTN?

And another experiment would be great: could you please

• Invoke GTN from LR
• Close GTN
• Wait a bit
• Invoke GTN from LR
• Close GTN

.... this should tell us, if LR behaves differently if the process it started (GTN) finishes before starting it again.

@nemethviktor, if this all works, we could go for the approach...

• Check if another GTN instance is running
• If so, send the info about which collection file to load to GTN
  • we could do direct communication via windows messaging, pipes or non persistant memory mapped files
  • we could be using a defined file location (new temp file) where we copy the collection file to and GTN can either recognize the update or get sent a signal that tells it to re-load the collection
• If so, exit otherwise load the collection

We should have a concept for two situations:

• GTN is running in normal and not collection mode and receives such a request. Easiest solution could be to show a message box to have the user close the app and let LR start it
• The current work in GTN is not saved. We could just as whether the current work should be saved

What do you think?

Currently, I'm having trouble carving some time out to get into the code (various other have to do duties). Sorry. I'll do again as soon as I get the chance to.

[Urmel]

I'm using the latest development codebase, feel free to add to it (I'm not actually doing anything at the moment so any changes will be accepted as they wouldn't be interfering or clashing with mine)

Hi @softnorbi,
please find attached a debug build to tackle above issue. It would be great if you could just unpack the archive and give it a go. After unpacking, please check if GTN runs by just double clicking the GeoTagNinja.exe.

If it runs, please use this exe as the invokation target for the LR runanycommand plugin.

Then, please run the test with the two invokations (i.e. starting one, going back to LR and starting a second one) again. It now should show more info (among other process ID and call parameters) in the log file. The log file also is named with date/time so that the two invokations will not overwrite each other - for this, I also adapted the clean up sequence.

Thanks!
Debug-Build_230331.zip

[softnorbi]

Hi @Urmel,
Did as you told and it worked however it seems with the second attempt also creates a second instance of GTN which and needs to be closed after closing LR. I looked for the log file, but couldn't find it. It's supposed to be generated by GTN - right?
What is the file name and the folder location?

[Urmel]

Hi @softnorbi ,

Hi @Urmel, Did as you told and it worked however it seems with the second attempt also creates a second instance of GTN which and needs to be closed after closing LR. I looked for the log file, but couldn't find it. It's supposed to be generated by GTN - right? What is the file name and the folder location?

Thanks!!

Correct - I did not implement anything to prevent a second instance from opening. First I want to understand how LR does this second call. That's what the log is good for.

You can find the log files in this folder: %APPDATA%\GeoTagNinja
(just past this into the address bar of your windows explorer)

[softnorbi]

o.k. here are the log files:
GeoTagNinja_log.zip

[Urmel]

Thanks @softnorbi!

[Urmel]

Hi @softnorbi, @nemethviktor,

I created a first version of the signalling mechanism in #80. The code in the PR uses named pipes and Mutex:

• On application start, try to get hold of the GTN Mutex
  • if not possible (another instance is running), pass a test text via a named pipe and exit
• if the Mutex lock could be acquired, startup up GTN
• Create a single threaded named pipe server
  • if data (whole line) is send through the named pipe, show a message box with it

Currently, only play code: passed on test text from newly started instance to running instance, msg box to show.

Test it yourself (command prompt):
echo "Some test message" > \\.\pipe\GeoTagNinjaSingleInstance

The most challenging next step is to decide how we do the reaction when this change of collection file is requested. Because: this (as in any other potenial method) request comes in asynchronously. So you could be right at saving some data or doing a gpx track sync when it comes in...

[nemethviktor]

hi @Urmel

Thanks for that as always. since i'm not working on anything now i'll leave it un-committed if that's ok so that you can amend it if you'd like as there is nothing to clash with.
A particular thought on the issue you raised: I think the software is small enough that we can't necessarily defend against every type of user stupidity. With that in mind, we could consider doing a simple boolean flag that's something like APP_BUSY true/false.
It's set to true when any of the following conditions are met:

• any Form is open (apart from the main app's form)
• when there is anything in DE3
• when a folder is loading
• ???? anything else?

Basically then we change the existing logic to say that instead of the List of files in collectionMode being parsed immediately, they get queued up. As long as the app is busy (as per above), they remain in the queue. Once the app becomes non-busy, they get parsed. An event listener could be set up for this.
Thoughts?

[Urmel]

Hi @nemethviktor,

thanks for the input! I'm with you. Question would be where we check for open tasks. We have the option to implement it in every ending call (e.g. closing logic of the edit form, ...) or we have a separate task running that frequently checks for the "lock".

Actually, I would opt for the later: we have a "check" function that determines if the app is busy (see your points above, but not with a general boolean flag only booleans/locks for specific processes like "loading"). This function we only call from the "work queue thread". Good thing with this would be that we do not need to ensure that every UI call considers this stuff - we only have one central place checking the situation...

When queuing a request, we might give the user a message box telling him/her what still is open before the task could be done and asking if the task (e.g. switch collection) should be accepted.

Should I give it a go?

[nemethviktor]

Hey,

Yes please experiment - admittedly I've been a bit quiet recently.
I'm thinking for the "check" function, instead of an actual function consider a subscription to an event? Possibly easier.

[Urmel]

Hi @nemethviktor,
will do...

ATM I'm not getting the event idea yet. Who do you envision sends respectively receives this event and what's the semantic of it? Thanks!!

[nemethviktor]

Hmm ok, initially I was thinking about events in terms of https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/events/
That said I'm now not quite sure events are the best placed things for us.
Regardless, we could have a boolean (e.g. app_is_busy or some such, triggered by certain Forms being active or certain actions happening) during which, if true any incoming requests are denied/queued up. The queueing could be triggered via an event listener where the event is the incoming request, though it might be redundant to code and instead just add to a queue in the more traditional way.
Not sure this helps?

[Urmel]

We might have 2 situations to deal with:

1. Conditions / situations that have no foreseeable end. If you have an edit window open - who knows when you will close it...
2. (More or less) predictable tasks that are being executed. When pressing the save button, you know it will end more or less soon...

My gut feeling is that we will have locks for case 1 and we should put case 2 things also in our queue. Why? Because tasks of type 2 may not be started concurrently. Also conditions of case 1 may not be entered while a task that needs locking is currently running (try to open an edit window while the application is switching to collection mode...).

But maybe I'm thinking too complex... :)

I did a cross check in the main file and my proposal is to consider these:

• any Form is open (apart from the main app's form).
  That actually implies multiple things like track sync, etc.
• when there is anything in DE3 or a saving operation ongoing
• when a folder is loading -> lvw_FileList_LoadOrUpdate
• updates to the file list:
  • lvw_FileList_UpdateTagsFromWeb
  • btn_SaveLocation_Click
  • btn_LoadFavourite_Click
  • tsb_Refresh_lvwFileList_Click
  • btn_ts_Refresh_lvwFileList_Click
  • tsb_GetAllFromWeb_Click
  • btn_OneFolderUp_Click
  • tsb_RemoveGeoData_Click

[nemethviktor]

I partially agree - but at the same time the list you have kindof overlaps with the Forms (being open) or DE3-not-being-empty -> such as:

• lvw_FileList_UpdateTagsFromWeb - changes DE3/makes it not empty
• btn_SaveLocation_Click - DE3 already not empty
• btn_LoadFavourite_Click - opens a Form
• tsb_Refresh_lvwFileList_Click - if DE3 not empty then refer to above else just read the queue (?)
• btn_ts_Refresh_lvwFileList_Click - same as above Refresh
• tsb_GetAllFromWeb_Click - same as TagsFromWeb (?)
• btn_OneFolderUp_Click - disabled in CollectionMode
• tsb_RemoveGeoData_Click - changes DE3

So I think any Form (other than main) being open or DE3 should send new items to the queue.
Possibly one way of doing this is that we already have a list of files to be processed. So essentially that list just needs amending (being added to.) and then do a check for the contents of the list.

[Urmel]

Hey @nemethviktor ,

I agree that if for example there are already modified files / DE3 is filled, the state does not change. Question is, whether starting lets say lvw_FileList_UpdateTagsFromWeb and an update of the directory elements should be allowed concurrently. That's why I thought we must also consider all methods that also change the state of the application to "blocked".

So regarding the queueing of a form start it could go like this:

• User pushes "edit button"
• Task "open edit form" is queued
• Serializer checks its queue (every ~150ms or faster) and detects new task
• Serializer checks if no other task runs and no locks are set that prevent task from executing
• Serializer runs the task: registered method is called which does what the current method does

[Urmel]

Hi @nemethviktor ,

after some more reading, I found out that we could serialize most of the stuff using existing .Net techniques. There are SynchronizationContexts available to various scopes (cf. https://learn.microsoft.com/en-us/archive/msdn-magazine/2011/february/msdn-magazine-parallel-computing-it-s-all-about-the-synchronizationcontext).

Therefore, I would now opt for

• Taking out most of the async stuff if possible (-> see question below)
• Pushing the requests from the Named Pipe into the UI context for execution

By doing this, we i) get a nice one-after-the-other execution of what comes in via Named Pipe and what the user clicks. And we ii) do not need to fumble around too deep with additional locking. In other words, we build a virtual button that is being clicked by the Named Pipe input.

So in the end, we would need to consider for this "change collection button":

• any Form is open (apart from the main app's form).
  For starters, I would just open a msg box informing the user. In step 2, the user could suspend for some time or until the currently open form is closed.

My open question now is: did you use some of the async stuff to improve performance or is it OK to make everything serial...?
Thanks!

[nemethviktor]

I don't remember - but I think everything "big" is serial at this point. Tbh I haven't looked much at the code lately but I recall there are a few async calls however I think not on the important things. Maybe have a look at (ie search for) async calls and see if they interfere with your plan? ;)

[Urmel]

Ok - so after some implementation work and testing, it turns out that we won't get around locking if we want UI updates during the processing. The cause being that updates to the UI need the UI thread to be not blocked / occupied: to update the UI, the system sends WM_PAINT and alike messages to the UI thread. If the thread is bound in a long-running operation, these messages do not get processed.

Application.DoEvents() is a scary workaround for what we want: it basically pauses the current context (i.e. the long running / background operation) and processes the UI Thread queue. BUT: thereby, new user interactions are also processed, potentially leading to two concurrently running operations.

Example. When loading the folder, we provide an information how much percent already is loaded. Two options:

1. Do the loading in the UI Thread, which means that all interfering interactions would be blocked / postponed (good). But: UI Updates do not go through - the application appears to be hanging (bad).
2. Do the loading in a separate thread or alike, which means UI updates get processed, the user sees the progress (good). But: one needs to take care that no two tasks that could interfere with each other run concurrently (bad, but solveable).

So.... I guess back to the previous approach - a bit modified:

• Introduce a lock (I guess for starters on the directory element collection, which all tasks that open forms also use) that is checked for
• Short running operations can either inform the user that it's currenlty not possible (change folder) or wait for the lock (select image)
• Queue long running operations (e.g. save changes, load folder) and process them in a sep. thread
  • background / long running operations (as they tend to finish some unexpected time in the future) may not change the selection in the UI

Concrete with the list from above:

• lvw_FileList_UpdateTagsFromWeb - changes DE3/makes it not empty --> background, lock DE-Collection
• btn_SaveLocation_Click - DE3 already not empty --> background, lock DE-Collection
• btn_LoadFavourite_Click - opens a Form --> blocking, lock DE-Collection (but not for use)
• tsb_Refresh_lvwFileList_Click - if DE3 not empty then refer to above else just read the queue (?) --> combination of load folder & save
• btn_ts_Refresh_lvwFileList_Click - same as above Refresh --> background, lock DE-Collection
• tsb_GetAllFromWeb_Click - same as TagsFromWeb (?) --> background, lock DE-Collection
• btn_OneFolderUp_Click - disabled in CollectionMode --> ignore if DE-collection is locked
• tsb_RemoveGeoData_Click - changes DE3 --> background, lock DE-Collection

So much for now

[nemethviktor]

I was thinking a bit about this and possibly a very n00b idea but:

• either: would it be possible that we use Applicaiton.DoEvents() but with a bool flag someplace in the background like processNewThingsThatAreNotUI=false - basically telling the code to ignore things until told otherwise
• or: failing that, as a fallback option just show a message that says something like "stuff is happening, it's not broken, but there are no visual updates, soz." (a little like how the UI was locked at some point in the app while a read-operation was ongoing.)

[Urmel]

The first one would work - it's basically locking things. Instead of queueing operations, we inform that the current operation is still ongoing. Sounds good to me. We could also have a modal window like during loading the directory at the beginning with an additional status line...
Good thing about this is - when you queue things, you are not sure if the situation is that same as before the queueing ;)
The only thing we would need to queue then is the request to change the application mode / update the collection list.