Improved progress indication for fulltext-index operations

[btut]

Operations on the full-text index are now more fine-grained. Instead of adding a whole database at once, files are added individually so we can display a more sensible progress in the background-task menu. The progress is now also counted towards the average progress of background processes and therefore shown by the progress indicator at the top right corner.

Fixes #7961.

• Change in CHANGELOG.md described in a way that is understandable for the average user (if applicable)
• Tests created for changes (if applicable)
• Manually tested changed features in running JabRef (always required)
• Screenshots added in PR description (for UI changes)
• Checked documentation: Is the information available and up to date? If not created an issue at https://github.com/JabRef/user-documentation/issues or, even better, submitted a pull request to the documentation repository.

[tobiasdiez]

Thanks!

I don't have much time right now, so just a few initial remarks from my side

[calixtus]

Checkstyle: Unused imports in IndexingTaskManager and PdfIndexer

[btut]

This is basically a follow up to a feature PR that is not yet part of any release. Shall I add an entry in the Changelog anyways?

[calixtus]

No. Changelog should only describe changes from one release to the next.

[Siedlerchr]

lgtm

[Siedlerchr]

Seems like there are some L10n errors

[btut]

L10n passes now.

[koppor]

Looks good to me.

I merge to have a better experience for the erlier adapters.

[ilippert]

looks wonderful on JabRef 5.4--2021-08-23--cef4151
Linux 5.13.12-200.fc34.x86_64 amd64
Java 16.0.2
JavaFX 16+8

however, when indexing two files, the progress indication is not well intelligible for me. I have a file A with +7k attached files, another B with 0.5k attached files. After a while, the progress was half full (A and B both a few hundred), but after B was completed, progress was indicated as "lower" than before.

[btut]

looks wonderful on JabRef 5.4--2021-08-23--cef4151
Linux 5.13.12-200.fc34.x86_64 amd64
Java 16.0.2
JavaFX 16+8

Yay!

however, when indexing two files, the progress indication is not well intelligible for me. I have a file A with +7k attached files, another B with 0.5k attached files. After a while, the progress was half full (A and B both a few hundred), but after B was completed, progress was indicated as "lower" than before.

The indicator shows the average progress of running tasks. So if 75% of files in B are indexed and 25% the average is 50%. A little later, all files in B are indexed but let's say A is only at 30%. The indicator will show 30% because the task for B is already done and not considered for the average.
I think this is better than counting all tasks towards the progress forever, because given enough tasks that are already done and one task that just started, the average would be very high because of all the finished tasks.
Also, I think the progress indicator is more important so users can tell something is done in the background. IMO, the actual progress of the tasks is not that important.

The current behavior is also how other programs often show progress. Just look at the download-progress indicator of most browsers. If you download multiple files, once the first is done the indicator will jump back, just as it does now in JabRef.

If you have a better idea how to handle this, let me know.

[ilippert]

The indicator shows the average progress of running tasks. So if 75% of files in B are indexed and 25% the average is 50%. A little later, all files in B are indexed but let's say A is only at 30%. The indicator will show 30% because the task for B is already done and not considered for the average.

Ah, ok, get it. Thank you for the explanation.

I think this is better than counting all tasks towards the progress forever, because given enough tasks that are already done and one task that just started, the average would be very high because of all the finished tasks.

I get the idea of showing the current progress relative to what more needs to be indexed.

Also, I think the progress indicator is more important so users can tell something is done in the background. IMO, the actual progress of the tasks is not that important.

I very much agree.

The current behavior is also how other programs often show progress. Just look at the download-progress indicator of most browsers. If you download multiple files, once the first is done the indicator will jump back, just as it does now in JabRef.

If you have a better idea how to handle this, let me know.

how about always use the sum of all files across the ongoing task(s) instead of per library? say to be indexed in A: 10k, already indexed 1k; to be indexed in B 1k, already indexed 0.5k. Then 1.5/11=14%

[btut]

how about always use the sum of all files across the ongoing task(s) instead of per library? say to be indexed in A: 10k, already indexed 1k; to be indexed in B 1k, already indexed 0.5k. Then 1.5/11=14%

Then what about other background tasks (such as file download)? Their progress is inherently different (Bytes received / File Size).

[ilippert]

Then what about other background tasks (such as file download)? Their progress is inherently different (Bytes received / File Size).

ok, yes, no easy response to that.

I completely accept the incommensurabilities you point to. Thus, you have convinced me there is no simple change.

To reiterate my happiness: the progress indication is great! Thanks!

[btut]

Yay!