Add pagination and partial name search to List Jobs APIs

[krithika369]

Description

This PR adds 2 new paginated APIs for listing jobs:

• /projects/{project_id}/jobs-by-page
• /models/{model_id}/versions/{version_id}/jobs-by-page

With this, the use of the existing .../jobs list APIs has been replaced by the new APIs in the SDK implementation and will also be replaced for the UI, in another PR. The non-paginated list jobs APIs have been marked deprecated and can be removed eventually.

Illustration

Modifications

• swagger.yaml
  • Deprecate existing list jobs APIs.
  • Add new /jobs-by-page APIs. These APIs accept a new search query parameter compared to the existing list jobs APIs. This parameter will do partial matches of the job name as opposed to equality matching. This can be particularly useful for searches done from the UI. (The search parameter has been named so, taking inspiration from XP - example.)
• api/api/prediction_job_api.go - Implement the paginated APIs
• api/service/prediction_job_service.go
  • Add paginator to predictionJobService.
  • Add Page, PageSize and Search parameters to ListPredictionJobQuery.
  • Add isPaginated flag to ListPredictionJobs method. When set, the DB query will be executed with the appropriate offset and limit and the pagination data sent back.
• api/storage/prediction_job_storage.go - Add Count method. The List method is updated to handle offset and limit. Both methods support partial searching of the name column.
• api/go.mod - Update MLP API dependency to consume Add package pagination mlp#94 and Add JSON tags for Paging struct mlp#98
• python/sdk/merlin/model.py - Update the list prediction job method to use the paginated backed API

Tests

SDK's list_prediction_job API (which lists the jobs for a given model version) now uses the paginated backend API, getting a maximum of 10 results at once. The performance of this has been tested locally with a dataset size of 275 (results in 28 jobs API calls). This does make the SDK method slower (0.33 seconds vs 0.85 seconds locally). If required, in the future, the page_size passed to the API call can be explicitly set to a larger value or the pagination options can be exposed to the user via the SDK.

Checklist

• Added PR label
• Added unit test, integration, and/or e2e tests
• Tested locally
• Updated documentation
• Update Swagger spec if the PR introduce API changes
• Regenerated Golang and Python client if the PR introduces API changes

Release Notes

Add new paginated APIs for listing prediction jobs.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 60.39%. Comparing base (9cdcecb) to head (bfc08e1).
Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #581      +/-   ##
==========================================
+ Coverage   60.31%   60.39%   +0.08%     
==========================================
  Files         274      274              
  Lines       21930    22017      +87     
==========================================
+ Hits        13226    13298      +72     
- Misses       7855     7863       +8     
- Partials      849      856       +7     

Flag	Coverage Δ
api-test	58.33% <ø> (+0.09%)	⬆️
sdk-test-3.10	75.54% <ø> (+0.06%)	⬆️
sdk-test-3.8	75.46% <ø> (+0.06%)	⬆️
sdk-test-3.9	75.46% <ø> (+0.06%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ariefrahmansyah]

lgtm. thanks @krithika369 !