Limit number of scanned items in a single run

[Omertron]

Original issue 2577 created by Omertron on 2013-01-08T13:38:26.000Z:

Running YAMJ on some low performance machine can take a very long time with big libraries. A configuration parameter should be implemented in order to limit the movies scanned for each YAMJ run.

This kind of behavior should allow to periodically schedule YAMJ (e.g during night hours) and be sure that it won't hog the system for too long time..

[Omertron]

Comment #1 originally posted by Omertron on 2013-01-08T14:01:57.000Z:

Fixed in r3469

[Omertron]

Comment #2 originally posted by Omertron on 2013-01-08T14:03:36.000Z:

This issue was closed by revision r3470.

[Omertron]

Comment #3 originally posted by Omertron on 2013-01-08T18:05:00.000Z:

If I set "mjb.check.max=2" and run the jukebox, I always get the message that the limit was reached and movies were skipped.

So not sure that this is fully functional

[Omertron]

Comment #4 originally posted by Omertron on 2013-01-08T18:30:14.000Z:

this doesn't work right.. it increases the counter when it starts to process something, but then decreases it when it finishes. you never actually hit the stop limit.

[Omertron]

Comment #5 originally posted by Omertron on 2013-01-08T19:26:00.000Z:

Can you guys both send me a log? I'm using this feature on my library and it is working

[Omertron]

Comment #6 originally posted by Omertron on 2013-01-08T21:25:51.000Z:

Here are 5 log files on a jukebox of 5 movies, I expect it to take 3 runs to complete.
First run is from an empty jukebox. 4 subsequent log files are run directly afterwards.

[Omertron]

Comment #7 originally posted by Omertron on 2013-01-08T23:48:07.000Z:

OK first of all it seems that I made a mess while merging my changes to MovieJukebox.java. There were conflicts so I had to resolve them manually and I forgot to copy/paste the step where the token is released :) I'll fix that as soon as possible.

In any case, you should use mjb.check.limit >= mjb.MaxThreadsProcess.

I knew that it would be tricky to deal with multiple threads without messing too much with the MovieJukebox class, and that's why I implemented the limit with a logic of get/release token. The methods are synchronized, so the implementation should be thread safe. However, when more than one thread is asking for a token and you are close to the limit, this can lead to false positives... I'll clearly have to think about something to avoid these false positives, but in the meanwhile using a limit higher than the number of threads should be safe most of the time.

[Omertron]

Comment #8 originally posted by Omertron on 2013-01-09T15:14:23.000Z:

According thread safeness: Synchronized is thread but there is s perfomanter method
of counter increase/decrease in thread safe behaviour: Use an AtomicInteger instead
of the "int tokens", then you can leave the synchronized methods unsynchronized.

[Omertron]

Comment #9 originally posted by Omertron on 2013-01-09T17:12:58.000Z:

This should be fixed with r3478.

Note: you can still have problems if you have trailers rescan feature enabled, since I think we have a problem with checking the days since last trailer scan (I just opened Issue 2578), and so if you have movies for which trailers can't be found, these are re-scanned with each yamj run, and are set as dirty everytime. If these movies are at least mjb.check.Max, you will never get past them.

With trailer rescan disabled (or with 2578 fixed as I have with my local copy), however, it is working.