-
Notifications
You must be signed in to change notification settings - Fork 996
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Ingestion Job management API for Feast Core #548
Commits on Apr 6, 2020
-
Added protobuf definitions for a Job Management API
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 1e1abb1 - Browse repository at this point
Copy the full SHA 1e1abb1View commit details -
Added query methods to JobRepository to query jobs by store and featu…
…reset
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for bd04076 - Browse repository at this point
Copy the full SHA bd04076View commit details -
Added hashCode() & equals() to Job model compare and hash jobs
This would allow jobs to be used as elements in HashSets and keys in HashMaps
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 643f042 - Browse repository at this point
Copy the full SHA 643f042View commit details -
Added toIngestionProto() to Job object to convert Job model to ingest…
…ion job proto
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for f4b417e - Browse repository at this point
Copy the full SHA f4b417eView commit details -
Added query methods to FeatureSetRepository to query by exact (name, …
…project) or (name,version)
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 9d62a38 - Browse repository at this point
Copy the full SHA 9d62a38View commit details -
Added listJobs() to JobService to handle request list ingestion jobs …
…requests
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 32a0bb5 - Browse repository at this point
Copy the full SHA 32a0bb5View commit details -
Added missing filter field in ListIngestionJobsRequest protobuf
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 4f55012 - Browse repository at this point
Copy the full SHA 4f55012View commit details -
Added code to setup mockups for testing JobService
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for b3aa72a - Browse repository at this point
Copy the full SHA b3aa72aView commit details -
Revert "Added hashCode() & equals() to Job model compare and hash jobs"
This reverts commit ba995bb712cbd38f0f4bef47efbe433d5ec07521 as it caused core tests to fail
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 06b84df - Browse repository at this point
Copy the full SHA 06b84dfView commit details -
Added JobServiceTest unit tests for JobService's listJobs()
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for c4b901a - Browse repository at this point
Copy the full SHA c4b901aView commit details -
Added stopJobs() to JobService to handle requests to stop jobs
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for cd0bde2 - Browse repository at this point
Copy the full SHA cd0bde2View commit details -
Added getTransitionalStates() to JobStatus to return collection of tr…
…ansitional states
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for c59035f - Browse repository at this point
Copy the full SHA c59035fView commit details -
Changed stopJobs() to throw unsupported error on transitional job sta…
…tuses
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 898045c - Browse repository at this point
Copy the full SHA 898045cView commit details -
Moved conversion of JobStatus to IngestionJobStatus proto to JobStatus.
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for f4711cd - Browse repository at this point
Copy the full SHA f4711cdView commit details -
Make findFeatureSet() match only one featureset.
Limit findFeatureSets() as feature set references as composite keys should match one and only one featureset.
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for fe744ea - Browse repository at this point
Copy the full SHA fe744eaView commit details -
Added matchFeatureSets() to SpecService to match Feature Sets from Re…
…ferences This commit is adds temporary support for FeatureSetReference to SpecService via listFeatureSets(). In the future, this should merged together with listFeatureSets() as their functionality is almost the same.
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for a79f8b4 - Browse repository at this point
Copy the full SHA a79f8b4View commit details -
Refactor JobService listJobs() to use SpecService to provide features…
…et matching
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 5b25929 - Browse repository at this point
Copy the full SHA 5b25929View commit details -
Revert findBy methods added to FeatureSetRepository as no longer used.
JobService no longer depends on FeatureSetRepository directly, instead via SpecService
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 1997bb1 - Browse repository at this point
Copy the full SHA 1997bb1View commit details -
Added restartJob() to JobManagers to restart ingestion/import jobs
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for f320e38 - Browse repository at this point
Copy the full SHA f320e38View commit details -
Added restartJob() to JobService to restart ingestion jobs
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 0ec5afb - Browse repository at this point
Copy the full SHA 0ec5afbView commit details -
Update job model (due to new extId) when restartJob() in JobService
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 83cc4c7 - Browse repository at this point
Copy the full SHA 83cc4c7View commit details -
Use assertThat() & equalTo() instead of assertEquals() in JobService
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for cf6a9c4 - Browse repository at this point
Copy the full SHA cf6a9c4View commit details -
Revert "Use assertThat() & equalTo() instead of assertEquals() in Job…
…Service" due to failed tests This reverts commit bb3cf0ae1e7cfbde7a2f997cacc661d040c0a35f.
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 8088b33 - Browse repository at this point
Copy the full SHA 8088b33View commit details -
Use hamcrest’s assertThat() and equalTo() instead of Junit's assertEq…
…uals
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for bdc2803 - Browse repository at this point
Copy the full SHA bdc2803View commit details -
Hook up JobService list, stop, restart job methods to CoreServiceImpl.
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for fafcd11 - Browse repository at this point
Copy the full SHA fafcd11View commit details -
Throw InvalidArgumentException instead when calling listJobs() with i…
…nvalid FeatureSetReference Throw InvalidArgumentException instead of UnsupportedOperationException for better semantics: UnsupportedOperationException should be reserved for operations that fail due to failed preconditions
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 7a97e1f - Browse repository at this point
Copy the full SHA 7a97e1fView commit details -
Fixed typos in javadocs: InvalidArgumentException should be IllegalAr…
…gumentException
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for d277f93 - Browse repository at this point
Copy the full SHA d277f93View commit details -
Fixed in findByFeatureSetsIn() query in JobRepository
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 255946b - Browse repository at this point
Copy the full SHA 255946bView commit details -
Renamed toIngestionProto() methods to toProto() to follow code conven…
…tion
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 09e9ddc - Browse repository at this point
Copy the full SHA 09e9ddcView commit details -
Make JobService's listJobs() a transaction to prevent DB data race co…
…nditions
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for c85da17 - Browse repository at this point
Copy the full SHA c85da17View commit details -
Moved matchFeatureSets() in SpecService to private method in JobService
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for c876e40 - Browse repository at this point
Copy the full SHA c876e40View commit details -
Make the map that maps between JobStatus and IngestionJobStatus static
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 1799c3a - Browse repository at this point
Copy the full SHA 1799c3aView commit details -
Make JobService's listJobs() to return all ingestion jobs on empty fi…
…lter
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 1bead61 - Browse repository at this point
Copy the full SHA 1bead61View commit details -
Fixed issue where the jobManager map that JobService built used wrong…
… keys
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 19994ea - Browse repository at this point
Copy the full SHA 19994eaView commit details -
Fix issue where actual Job Status is not synced with database.
Issue occurs when the job is aborted/restarted, but the JobStatus has not yet been updated by JobUpdateTask. Hence another call to abortJob() & restartJob() that should be rejected due invalid status is allowed through
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for ff73b5f - Browse repository at this point
Copy the full SHA ff73b5fView commit details -
Log stopJob() & restartJob() operations to make debugging easier
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for c207f1a - Browse repository at this point
Copy the full SHA c207f1aView commit details -
Use Runner.name() instead of runner.toString() to build JobManager map
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for c64039a - Browse repository at this point
Copy the full SHA c64039aView commit details -
Move documentation on JobService operations to CoreService protobuf d…
…efinition
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for ed4dae9 - Browse repository at this point
Copy the full SHA ed4dae9View commit details -
Added IngestJob to python sdk as native representation of IngestionJo…
…b proto
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 24c09be - Browse repository at this point
Copy the full SHA 24c09beView commit details -
Make empty filter on JobService's listJobs() select all ingestion jobs
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 2930388 - Browse repository at this point
Copy the full SHA 2930388View commit details -
Added bindings for Job management API to python sdk client.
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 5cc8abe - Browse repository at this point
Copy the full SHA 5cc8abeView commit details -
Fixed __connect_core() to connect to Feast CoreService when calling o…
…n Job API calls
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for d86cb66 - Browse repository at this point
Copy the full SHA d86cb66View commit details -
Auto reload IngestJob.status and IngestJob.external_id on get property.
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for d9874f2 - Browse repository at this point
Copy the full SHA d9874f2View commit details -
Added IngestJob.wait() to wait for job status to transtion
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 19620b2 - Browse repository at this point
Copy the full SHA 19620b2View commit details -
Added basic job api e2e test to exercise job api
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 6b47ecb - Browse repository at this point
Copy the full SHA 6b47ecbView commit details -
Reorder the operations e2etest to make sure that jobs are running aft…
…er test
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for f3471ac - Browse repository at this point
Copy the full SHA f3471acView commit details -
Added e2e test for all types exercising job api
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 1707401 - Browse repository at this point
Copy the full SHA 1707401View commit details -
Fixed typo in function arguments
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 9999db5 - Browse repository at this point
Copy the full SHA 9999db5View commit details -
Added unit tests for Ingestion Job API additions in python sdk
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for f012623 - Browse repository at this point
Copy the full SHA f012623View commit details -
Rename "ingestion" to "ingest" for more consistent naming
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 2e819b2 - Browse repository at this point
Copy the full SHA 2e819b2View commit details -
Disable support for restarting Job in a terminal state due to possibl…
…e race conditions
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for ba08c8a - Browse repository at this point
Copy the full SHA ba08c8aView commit details -
Added __str__ and __repr__ to IngestJob to render ingestjob in human …
…readable string
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 5fb774c - Browse repository at this point
Copy the full SHA 5fb774cView commit details -
Added FeatureSetRef to represent references to featursets
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 80ef3c9 - Browse repository at this point
Copy the full SHA 80ef3c9View commit details -
Admend client's list_ingest_jobs() to accept feature references directly
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for d3c9a3d - Browse repository at this point
Copy the full SHA d3c9a3dView commit details -
Fixed typo in IngestJob.store property
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 0cfc32a - Browse repository at this point
Copy the full SHA 0cfc32aView commit details -
Fixed issue with FeatureSetRef.from_str not converting version to int
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for a540157 - Browse repository at this point
Copy the full SHA a540157View commit details -
Make the grpc error message more apparent on stop_ingest_job() and re…
…start_ingest_job()
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 355b571 - Browse repository at this point
Copy the full SHA 355b571View commit details -
Added feast ingest-job list, describe, stop, restart to CLI
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 05b33a6 - Browse repository at this point
Copy the full SHA 05b33a6View commit details -
Rename Job to RetrievalJob to prevent confusion with IngestJob
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 3b8198a - Browse repository at this point
Copy the full SHA 3b8198aView commit details -
Updated e2e tests to use FeatureSetRef in list_ingest_jobs()
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 19bd141 - Browse repository at this point
Copy the full SHA 19bd141View commit details -
Fixed due e2e tests to cater to new limitations on stop_ingest_job()
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 9f6cf6a - Browse repository at this point
Copy the full SHA 9f6cf6aView commit details -
Increase timeout on test_all_types_ingest_jobs() e2e test.
Zhu Zhanyan committedApr 6, 2020 Configuration menu - View commit details
-
Copy full SHA for 87054b9 - Browse repository at this point
Copy the full SHA 87054b9View commit details
Commits on Apr 7, 2020
-
Configure IngestJob.wait() to backoff with a exponentially larger wai…
…t duration
Zhu Zhanyan committedApr 7, 2020 Configuration menu - View commit details
-
Copy full SHA for 1848dd4 - Browse repository at this point
Copy the full SHA 1848dd4View commit details -
Added print statements to debug e2e test failure.
Zhu Zhanyan committedApr 7, 2020 Configuration menu - View commit details
-
Copy full SHA for 146fb2b - Browse repository at this point
Copy the full SHA 146fb2bView commit details -
Revert "Added print statements to debug e2e test failure."
This reverts commit 146fb2b.
Zhu Zhanyan committedApr 7, 2020 Configuration menu - View commit details
-
Copy full SHA for 0fa6b93 - Browse repository at this point
Copy the full SHA 0fa6b93View commit details -
Fixed issue of test waiting for aborted job to become running causing…
… timeout.
Zhu Zhanyan committedApr 7, 2020 Configuration menu - View commit details
-
Copy full SHA for 66d3f77 - Browse repository at this point
Copy the full SHA 66d3f77View commit details