Add cursor-based ItemReader for MongoDB

[juchanei]

The MongoItemReader currently uses pagination with MongoDB's skip operation, which can cause unnecessary document scans when requesting next pages.

The skip() method requires the server to scan from the beginning of the input results set before beginning to return results. As the offset increases, skip() will become slower.
https://www.mongodb.com/docs/manual/reference/method/cursor.skip/#using-skip--

Therefore, I have implemented a new cursor-based MongoDB ItemReader that accesses the next documents without skipping.

[MinJunKweon]

related #3824

[fmbenhassine]

Awesome! Thank you for contributing this cursor-based implementation! I will plan this feature for Spring Batch 5.1.

Just curious, have you run a benchmark to compare the performance of this reader with the paging one?

[juchanei]

Just curious, have you run a benchmark to compare the performance of this reader with the paging one?

@fmbenhassine

As you know, pagination-based item readers perform poorly in terms of performance due to the amount of documents that need to be skipped. This amount is related to the total number of documents and the batch size, making it difficult to quantify the performance improvement of cursor-based item readers in numbers.

In my case, when benchmarking with a batch size of 10k on 0-3.5 million documents, the elapsed time was reduced by about 20-40%. While different results may occur depending on the test conditions, I am attaching the scatter chart results from my test.

[hmhuan]

Nitpicking: Should we take care of batchSize when it is 0 or negative?
As I checked the spec of spring-data-mongo

• Use {@literal 0 (zero)} for no limit. A negative limit closes the cursor after returning a single
• batch indicating to the server that the client will not ask for a subsequent one.

ref: https://github.com/spring-projects/spring-data-mongodb/blob/main/spring-data-mongodb/src/main/java/org/springframework/data/mongodb/core/query/Query.java#L539-L546

[juchanei]

@hmhuan I'm sorry for the delay in responding.
Users would expect that this code is implemented using spring-data-mongodb, given the explicit use of MongoOperations as an argument in MongoCursorItemReader. Therefore, if setBatchSize handles 0 or negative values, I think it could reduce predictability.

[fmbenhassine]

The type of the parameter should be the same as expected by the Query API. The setter should add the javadoc tag @see Query#cursorBatchSize(int) so the user knows that the value will be set on that field, and therefore knows the (default) values to use. I will take care of this change when merging the PR.

[fmbenhassine]

Excellent! Thank you for sharing those benchmarks. Great to see up to 40% improvement!

Rebased and merged as b103900. Refined in d84057c.

Thank you for your contribution!