Execution Store #4827
Execution Store #4827
Conversation
WalkthroughThe changes introduce a new execution store repository within the FiftyOne framework, enhancing the management of execution stores using MongoDB. Key components include the addition of the Changes
Possibly related PRs
Suggested reviewers
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 12
Outside diff range and nitpick comments (1)
tests/unittests/execution_store_tests.py (1)
251-266: Add assertions to validate operation outcomes.The test currently checks that
insert_oneis called with specific arguments, but it does not validate the behavior after the operation. Consider adding assertions to verify the state of the system or any return values.Add assertions like:
# Verify that the operation result is as expected self.assertIsNotNone(result)
Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Files selected for processing (9)
- fiftyone/factory/repo_factory.py (2 hunks)
- fiftyone/factory/repos/execution_store.py (1 hunks)
- fiftyone/operators/executor.py (1 hunks)
- fiftyone/operators/store/init.py (1 hunks)
- fiftyone/operators/store/models.py (1 hunks)
- fiftyone/operators/store/permissions.py (1 hunks)
- fiftyone/operators/store/service.py (1 hunks)
- fiftyone/operators/store/store.py (1 hunks)
- tests/unittests/execution_store_tests.py (1 hunks)
Additional context used
Ruff
fiftyone/operators/store/permissions.py
8-8: Redefinition of unused
Dictfrom line 6Remove definition:
Dict(F811)
8-8: Redefinition of unused
Listfrom line 6Remove definition:
List(F811)
8-8: Redefinition of unused
Optionalfrom line 6Remove definition:
Optional(F811)
9-9: Redefinition of unused
BaseModelfrom line 5Remove definition:
BaseModel(F811)
fiftyone/operators/store/service.py
10-10:
tracebackimported but unusedRemove unused import:
traceback(F401)
fiftyone/operators/store/store.py
10-10:
fiftyone.factory.repo_factory.RepositoryFactoryimported but unusedRemove unused import:
fiftyone.factory.repo_factory.RepositoryFactory(F401)
tests/unittests/execution_store_tests.py
9-9:
timeimported but unusedRemove unused import:
time(F401)
11-11:
unittest.mockimported but unusedRemove unused import:
unittest.mock(F401)
12-12:
unittest.mock.patchimported but unusedRemove unused import:
unittest.mock.patch(F401)
14-14:
fiftyoneimported but unusedRemove unused import:
fiftyone(F401)
15-15:
bson.ObjectIdimported but unusedRemove unused import:
bson.ObjectId(F401)
23-23: Redefinition of unused
ExecutionStoreServicefrom line 18Remove definition:
ExecutionStoreService(F811)
Additional comments not posted (33)
fiftyone/operators/store/__init__.py (1)
1-20: LGTM!The
__init__.pyfile is well-structured and follows the standard pattern for exposing the public API of a Python package. The docstring provides a clear description of the module, and the__all__list is properly defined to include the relevant classes and service.The imports are organized and sourced from the appropriate submodules, maintaining a clean separation of concerns.
Overall, the code changes in this file are good to go!
fiftyone/operators/store/permissions.py (2)
12-17: LGTM!The
StorePermissionsclass structure looks good. The fields for roles, users, and plugins permissions are appropriately typed as optional dictionaries with default empty dictionary values. This allows flexibility in defining permissions for a store.
19-26: LGTM!The
defaultstatic method is a good addition for easy initialization ofStorePermissionswith default values. Providing full access (read and write) to the admin role by default is a reasonable choice. The method correctly returns aStorePermissionsinstance.fiftyone/factory/repo_factory.py (2)
16-19: LGTM!The new import statements for
ExecutionStoreRepoandMongoExecutionStoreRepoare necessary and follow the existing code style.
52-65: Excellent addition of theexecution_store_repo()factory method!The new static method follows the factory pattern and is consistent with the existing
delegated_operation_repo()method. It ensures efficient management ofMongoExecutionStoreRepoinstances by reusing existing instances when possible.The method signature, docstring, and implementation logic are all properly defined.
fiftyone/operators/store/models.py (2)
10-33: LGTM!The
KeyDocumentclass is well-defined with appropriate fields for representing a key in the store. The optional TTL field and the permissions dictionary provide flexibility in managing key expiration and access control. The example schema in theConfigclass is helpful for understanding the model's usage.
36-57: LGTM!The
StoreDocumentclass effectively represents a store in the execution store by inheriting fromKeyDocumentand overriding thekeyandvaluefields. Setting thekeyfield to a constant value ensures a consistent identifier for store documents. The optionalvaluefield allows storing additional metadata for the store. The example schema in theConfigclass demonstrates a well-structured permissions system for roles, users, groups, and plugins, enabling fine-grained access control.fiftyone/factory/repos/execution_store.py (5)
1-3: LGTM!The docstring provides a clear and concise description of the module's purpose.
9-13: LGTM!The
_wherehelper function correctly constructs a query dictionary based on the providedstore_nameand optionalkey. It will be useful for querying the execution store repository.
16-64: LGTM!The
ExecutionStoreRepoclass provides a solid foundation for execution store repositories. The methods cover the essential operations for managing stores and keys, such as creating stores, setting keys, getting keys, updating TTLs, deleting keys, and deleting stores. The use ofStoreDocumentandKeyDocumentmodels ensures consistent data structure and validation.
67-70: LGTM!The
MongoExecutionStoreRepoclass correctly inherits fromExecutionStoreRepoand sets theCOLLECTION_NAMEattribute to"execution_store". This class will provide a MongoDB-specific implementation of the execution store repository.
72-73: LGTM!The
__init__method correctly initializes theMongoExecutionStoreRepoinstance by calling the parent class constructor with the providedcollectionparameter. This ensures that the_collectionattribute is properly set in the parent class.fiftyone/operators/store/store.py (9)
17-20: LGTM!The
createmethod correctly instantiates and returns a newExecutionStoreinstance with the providedstore_nameand a newExecutionStoreServiceinstance.
22-29: LGTM!The constructor method correctly initializes the
ExecutionStoreinstance with the providedstore_nameandstore_service. The docstring provides a clear description of the parameters.
31-43: LGTM!The
getmethod correctly retrieves the value associated with the providedkeyfrom the store using theget_keymethod of theExecutionStoreServiceinstance. It handles the case when the key is not found by returningNone. The docstring provides a clear description of the parameters and return value.
45-53: LGTM!The
setmethod correctly sets thevalueassociated with the providedkeyin the store using theset_keymethod of theExecutionStoreServiceinstance. It also accepts an optionalttlparameter to set the time-to-live for the key. The docstring provides a clear description of the parameters.
55-61: LGTM!The
deletemethod correctly deletes the key from the store using thedelete_keymethod of theExecutionStoreServiceinstance. The docstring provides a clear description of the parameter.
63-72: LGTM!The
hasmethod correctly checks the existence of the providedkeyin the store using thehas_keymethod of theExecutionStoreServiceinstance. It returns a boolean indicating whether the key exists. The docstring provides a clear description of the parameter and return value.
74-76: LGTM!The
clearmethod correctly clears all the data in the store using theclear_storemethod of theExecutionStoreServiceinstance. The docstring provides a clear description of the method's functionality.
78-85: LGTM!The
update_ttlmethod correctly updates the TTL for the specifiedkeyusing theupdate_ttlmethod of theExecutionStoreServiceinstance. The docstring provides a clear description of the parameters.
87-96: LGTM!The
get_ttlmethod correctly retrieves the TTL for the specifiedkeyusing theget_ttlmethod of theExecutionStoreServiceinstance. It returns the TTL in milliseconds orNoneif the key does not have a TTL. The docstring provides a clear description of the parameter and return value.fiftyone/operators/store/service.py (9)
20-24: LGTM!The
__init__method correctly initializes the_repoattribute based on the providedrepoor creates a new one using theRepositoryFactory.
26-39: LGTM!The
create_storemethod correctly creates a new store using the_repowith the providedstore_nameandpermissions(or default permissions if not provided).
41-55: LGTM!The
set_keymethod correctly sets the value of a key in the specified store using the_repowith the providedstore_name,key,value, andttl.
57-67: LGTM!The
get_keymethod correctly retrieves the value of a key from the specified store using the_repowith the providedstore_nameandkey.
69-79: LGTM!The
delete_keymethod correctly deletes the specified key from the store using the_repowith the providedstore_nameandkey.
81-94: LGTM!The
update_ttlmethod correctly updates the TTL of the specified key in the store using the_repowith the providedstore_name,key, andnew_ttl.
96-108: LGTM!The
set_permissionsmethod correctly sets the permissions for the specified store using the_repowith the providedstore_nameandpermissions.
110-119: LGTM!The
list_storesmethod correctly lists all stores matching the given criteria using the_repowith the providedsearchcriteria and additional keyword arguments.
121-130: LGTM!The
delete_storemethod correctly deletes the specified store using the_repowith the providedstore_name.fiftyone/operators/executor.py (1)
833-845: LGTM!The
create_storemethod implementation looks good:
- It correctly imports the
ExecutionStoreclass.- Creates a new store by calling
ExecutionStore.create()with the providedstore_name.- Returns the created store instance.
tests/unittests/execution_store_tests.py (2)
259-265: Verify arguments ininsert_oneassertion.In the
test_operator_execute_set_keymethod, ensure that the data passed toinsert_oneincludes all necessary fields, such as TTL if applicable.Review the
insert_onecall to confirm that all essential fields are included. If TTL is a required field, consider adding it to the mocked data and the assertion.
158-158: Align parameter name inupdate_ttlmethod call.After renaming the parameter in
MockStoreRepo, ensure that the method call matches the new parameter name for clarity.Apply this diff to update the method call:
-self.svc.update_ttl(store_name=store_name, key=key, new_ttl=new_ttl) +self.svc.update_ttl(store_name=store_name, key=key, new_ttl=new_ttl)Note: If you prefer to keep the parameter name as
ttl, update the method call accordingly.Likely invalid or redundant comment.
ritch
force-pushed
the
feat/execution-store
branch
from
bf5b35b
to
19857d3
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Files selected for processing (5)
- fiftyone/factory/repo_factory.py (2 hunks)
- fiftyone/operators/store/init.py (1 hunks)
- fiftyone/operators/store/models.py (1 hunks)
- fiftyone/operators/store/service.py (1 hunks)
- tests/unittests/execution_store_tests.py (1 hunks)
Files skipped from review as they are similar to previous changes (4)
- fiftyone/factory/repo_factory.py
- fiftyone/operators/store/init.py
- fiftyone/operators/store/models.py
- fiftyone/operators/store/service.py
Additional context used
Ruff
tests/unittests/execution_store_tests.py
9-9:
timeimported but unusedRemove unused import:
time(F401)
11-11:
unittest.mockimported but unusedRemove unused import:
unittest.mock(F401)
12-12:
unittest.mock.patchimported but unusedRemove unused import:
unittest.mock.patch(F401)
19-19: Redefinition of unused
ExecutionStoreServicefrom line 15Remove definition:
ExecutionStoreService(F811)
Additional comments not posted (1)
tests/unittests/execution_store_tests.py (1)
9-9: Remove unused importtime.The
timemodule is imported but not used in the code. Consider removing it to clean up the imports.Apply this diff to remove the unused import:
-import timeLikely invalid or redundant comment.
Tools
Ruff
9-9:
timeimported but unusedRemove unused import:
time(F401)
There was a problem hiding this comment.
Actionable comments posted: 3
Outside diff range and nitpick comments (1)
tests/unittests/execution_store_tests.py (1)
146-240: Integration tests forExecutionStore.The
TestExecutionStoreIntegrationclass provides integration tests for the main operations of theExecutionStore, such as setting a key, getting a key, listing keys, deleting a key, and clearing the store. The tests use a mocked collection and assert the expected interactions with the mocked collection, ensuring the correctness of the store's operations.Inquiry about the commented-out test.
I noticed that there is a commented-out test for updating a key (lines 173-196). Is this functionality not yet implemented or is there a reason for not including this test?
If you need any assistance with implementing or testing the update functionality, please let me know, and I'll be happy to help.
Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Files selected for processing (4)
- fiftyone/operators/store/models.py (1 hunks)
- fiftyone/operators/store/service.py (1 hunks)
- fiftyone/operators/store/store.py (1 hunks)
- tests/unittests/execution_store_tests.py (1 hunks)
Files skipped from review as they are similar to previous changes (2)
- fiftyone/operators/store/models.py
- fiftyone/operators/store/service.py
Additional context used
Ruff
fiftyone/operators/store/store.py
10-10:
fiftyone.factory.repo_factory.RepositoryFactoryimported but unusedRemove unused import:
fiftyone.factory.repo_factory.RepositoryFactory(F401)
tests/unittests/execution_store_tests.py
11-11:
unittest.mockimported but unusedRemove unused import:
unittest.mock(F401)
12-12:
unittest.mock.patchimported but unusedRemove unused import
(F401)
12-12:
unittest.mock.ANYimported but unusedRemove unused import
(F401)
15-15:
fiftyone.operatorsimported but unusedRemove unused import:
fiftyone.operators(F401)
17-17:
fiftyone.operators.store.models.StoreDocumentimported but unusedRemove unused import:
fiftyone.operators.store.models.StoreDocument(F401)
20-20: Redefinition of unused
ExecutionStoreServicefrom line 16Remove definition:
ExecutionStoreService(F811)
Additional comments not posted (13)
fiftyone/operators/store/store.py (11)
18-20: LGTM!The
createmethod correctly instantiates anExecutionStorewith the providedstore_nameand an instance ofExecutionStoreService.
22-29: LGTM!The constructor method correctly initializes the
ExecutionStoreinstance with the providedstore_nameandstore_service.
31-43: LGTM!The
getmethod correctly retrieves the value associated with the providedkeyfrom the store using theget_keymethod ofself._store_service. It handles the case when the key is not found by returningNone.
45-53: LGTM!The
setmethod correctly sets thevaluein the store under the providedkeywith an optionalttlusing theset_keymethod ofself._store_service.
55-64: Verify the return type ofdelete_key.The
deletemethod correctly deletes the providedkeyfrom the store using thedelete_keymethod ofself._store_service.However, the docstring mentions that the method returns a boolean indicating whether the key was deleted, but the method directly returns the result of
delete_keywithout any explicit boolean conversion. Please verify that thedelete_keymethod ofExecutionStoreServiceindeed returns a boolean to ensure consistency with the docstring.
66-75: LGTM!The
hasmethod correctly checks if the providedkeyexists in the store using thehas_keymethod ofself._store_serviceand returns the result as a boolean.
77-79: LGTM!The
clearmethod correctly clears all the data in the store by calling thedelete_storemethod ofself._store_servicewithself.store_name.
81-88: LGTM!The
update_ttlmethod correctly updates the TTL for the providedkeywith thenew_ttlvalue by calling theupdate_ttlmethod ofself._store_servicewith the appropriate arguments.
90-99: LGTM!The
get_ttlmethod correctly retrieves the TTL for the providedkeyby calling theget_ttlmethod ofself._store_servicewith the appropriate arguments and returns the result as an integer orNone.
101-107: LGTM!The
list_keysmethod correctly retrieves a list of all keys in the store by calling thelist_keysmethod ofself._store_servicewithself.store_nameand returns the result as a list of strings.
10-10: Remove unused import.The following comment from the previous review is still applicable:
coderabbitai[bot]: Remove unused import.
The import statement
from fiftyone.factory.repo_factory import RepositoryFactoryis unused in the code. Please remove it to keep the code clean and avoid potential confusion.Apply this diff to remove the unused import:
-from fiftyone.factory.repo_factory import RepositoryFactoryPlease address this issue to maintain code cleanliness.
Tools
Ruff
10-10:
fiftyone.factory.repo_factory.RepositoryFactoryimported but unusedRemove unused import:
fiftyone.factory.repo_factory.RepositoryFactory(F401)
tests/unittests/execution_store_tests.py (2)
25-46: LGTM!The
IsDateTimeclass andassert_delta_seconds_approxfunction are well-defined and serve a specific purpose in the test suite. The custom equality check fordatetime.datetimeinstances and the helper for asserting approximate time differences can be useful in time-related tests.
49-144: Comprehensive integration tests forExecutionStoreService.The
ExecutionStoreServiceIntegrationTestsclass provides a comprehensive set of integration tests for theExecutionStoreService. The tests cover various operations such as setting a key, getting a key, creating a store, deleting a key, updating TTL, deleting a store, and listing keys.The use of a mocked collection allows testing the service without relying on an actual database, and the assertions verify that the service interacts with the mocked collection as expected. The tests are well-structured and provide good coverage for the
ExecutionStoreService.
There was a problem hiding this comment.
Actionable comments posted: 1
Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Files selected for processing (5)
- fiftyone/factory/repos/execution_store.py (1 hunks)
- fiftyone/operators/store/models.py (1 hunks)
- fiftyone/operators/store/service.py (1 hunks)
- fiftyone/operators/store/store.py (1 hunks)
- tests/unittests/execution_store_tests.py (1 hunks)
Files skipped from review as they are similar to previous changes (2)
- fiftyone/operators/store/models.py
- fiftyone/operators/store/service.py
Additional context used
Ruff
fiftyone/operators/store/store.py
10-10:
fiftyone.factory.repo_factory.RepositoryFactoryimported but unusedRemove unused import:
fiftyone.factory.repo_factory.RepositoryFactory(F401)
tests/unittests/execution_store_tests.py
11-11:
unittest.mockimported but unusedRemove unused import:
unittest.mock(F401)
12-12:
unittest.mock.patchimported but unusedRemove unused import
(F401)
12-12:
unittest.mock.ANYimported but unusedRemove unused import
(F401)
15-15:
fiftyone.operatorsimported but unusedRemove unused import:
fiftyone.operators(F401)
17-17:
fiftyone.operators.store.models.StoreDocumentimported but unusedRemove unused import:
fiftyone.operators.store.models.StoreDocument(F401)
20-20: Redefinition of unused
ExecutionStoreServicefrom line 16Remove definition:
ExecutionStoreService(F811)
Additional comments not posted (13)
fiftyone/operators/store/store.py (1)
10-10: Remove unused import.The import statement
from fiftyone.factory.repo_factory import RepositoryFactoryis unused in the code. Please remove it to keep the code clean and avoid potential confusion.Apply this diff to remove the unused import:
-from fiftyone.factory.repo_factory import RepositoryFactoryLikely invalid or redundant comment.
Tools
Ruff
10-10:
fiftyone.factory.repo_factory.RepositoryFactoryimported but unusedRemove unused import:
fiftyone.factory.repo_factory.RepositoryFactory(F401)
fiftyone/factory/repos/execution_store.py (7)
1-14: LGTM!The module docstring and the private helper function
_whereare well-implemented.
- The docstring accurately describes the purpose of the module.
- The
_wherefunction constructs a query dictionary based on the providedstore_nameand optionalkeyarguments, which is a common pattern for building MongoDB queries.- The function is marked as private using the leading underscore convention, indicating that it is intended for internal use within the module.
17-31: LGTM!The
ExecutionStoreRepoclass and its__init__andcreate_storemethods are well-implemented.
- The class serves as a base class for execution store repositories, providing common functionality.
- The
__init__method initializes the class with a MongoDB collection, allowing it to work with a specific collection.- The
create_storemethod creates a new store document with the providedstore_nameand optionalpermissions.- It uses the
StoreDocumentmodel to create a new document instance and inserts it into the MongoDB collection using theinsert_onemethod.- The created
StoreDocumentinstance is returned, which is a good practice for allowing further operations on the created document.
33-37: LGTM!The
list_storesmethod is well-implemented.
- It retrieves all unique store names from the MongoDB collection using the
distinctmethod.- By specifying the "store_name" field as the argument to
distinct, it ensures that only the store names are returned, without any duplicates.- The method returns the distinct store names as a list of strings, which is a suitable format for further processing or displaying to the user.
38-69: LGTM!The
set_keymethod is well-implemented and handles both updating existing keys and inserting new ones.
- It uses the
KeyDocumentmodel to create a new document instance with the providedstore_name,key,value, and the current timestamp.- The update operations are prepared using the
$setand$setOnInsertoperators, which allows for updating existing fields and setting fields only on insert.- The upsert operation is performed using the
update_onemethod with theupsertparameter set toTrue, ensuring that a new document is inserted if it doesn't exist.- The
_wherehelper function is used to construct the query based on thestore_nameandkey, promoting code reuse and readability.- The method handles the case of a new document being inserted by setting the
created_atfield, and updates theupdated_atfield for existing documents.- The updated or inserted
KeyDocumentinstance is returned, providing flexibility for further operations.
71-75: LGTM!The
get_keymethod is well-implemented and retrieves a key document from the specified store.
- It uses the
find_onemethod of the MongoDB collection to retrieve a single document that matches the providedstore_nameandkey.- The
_wherehelper function is used to construct the query, promoting code reuse and readability.- If a matching document is found, it is parsed into a
KeyDocumentinstance using theKeyDocumentmodel.- If no matching document is found,
Noneis returned, indicating the absence of the requested key.- The method returns either a
KeyDocumentinstance orNone, providing a clear indication of the presence or absence of the requested key.
77-80: LGTM!The
list_keysmethod is well-implemented and lists all keys in the specified store.
- It uses the
findmethod of the MongoDB collection to retrieve all documents that match the providedstore_name.- The
_wherehelper function is used to construct the query, promoting code reuse and readability.- The
{"key": 1}projection is used to retrieve only the "key" field of the matching documents, optimizing the query performance by fetching only the required data.- A list comprehension is used to extract the "key" value from each document in the
keyscursor, providing a concise and efficient way to obtain the list of keys.- The method returns the list of keys, allowing for further processing or display to the user.
82-128: LGTM!The
update_ttl,delete_key, anddelete_storemethods of theExecutionStoreRepoclass, as well as theMongoExecutionStoreRepoclass, are well-implemented.
update_ttlmethod:
- It updates the TTL (Time-To-Live) of a specific key in the store.
- The expiration timestamp is calculated based on the provided
ttlusing theget_expirationmethod of theKeyDocumentmodel.- The
update_onemethod is used to update the "expires_at" field of the document that matches thestore_nameandkey.- The method returns a boolean indicating whether the update modified any documents, providing feedback on the success of the TTL update.
delete_keymethod:
- It deletes a specific key from the store.
- The
delete_onemethod is used to delete the document that matches thestore_nameandkey.- The method returns a boolean indicating whether the delete operation deleted any documents, providing feedback on the success of the key deletion.
delete_storemethod:
- It deletes an entire store and all its associated keys.
- The
delete_manymethod is used to delete all documents that match thestore_name.- The method returns the count of deleted documents, providing information on the number of keys that were removed.
MongoExecutionStoreRepoclass:
- It is a MongoDB-specific implementation of the execution store repository.
- The class inherits from the
ExecutionStoreRepoclass, leveraging its methods and functionality.- The
__init__method is overridden to call the parent class's__init__method and then create the necessary indexes by calling the_create_indexesmethod.- The
_create_indexesmethod ensures thetests/unittests/execution_store_tests.py (5)
25-27: LGTM!The
IsDateTimeclass is a simple and effective way to check if an object is an instance ofdatetime.datetime. The implementation is correct and concise.
30-31: LGTM!The
assert_delta_seconds_approxfunction is a useful utility for asserting that a time delta is approximately equal to a given number of seconds within a specified epsilon. The implementation is correct and straightforward.
34-46: LGTM!The
TeskKeyDocumentclass contains two well-written test methods that cover theget_expirationmethod of theKeyDocumentclass. The tests assert the correct behavior for both non-None and None TTL values.
49-155: LGTM!The
ExecutionStoreServiceIntegrationTestsclass contains a comprehensive set of test methods that cover various functionalities of theExecutionStoreServiceandExecutionStoreRepoclasses. The tests are well-structured, use appropriate mocks, and make thorough assertions to ensure the correctness of the tested methods.
157-228: LGTM!The
TestExecutionStoreIntegrationclass contains a comprehensive set of test methods that cover various functionalities of theExecutionStoreclass. The tests are well-structured, use appropriate mocks, and make thorough assertions to ensure the correctness of the tested methods.
ritch
force-pushed
the
feat/execution-store
branch
from
5e12362
to
9f9182b
Compare
There was a problem hiding this comment.
Actionable comments posted: 20
🧹 Outside diff range and nitpick comments (5)
fiftyone/operators/store/models.py (1)
31-35: StoreDocument structure looks good, consider adding a docstring to the 'value' fieldThe
StoreDocumentclass is well-structured and appropriately inherits fromKeyDocument. The constantkeyvalue and optionalvaluefield are good design choices.To improve clarity, consider adding a docstring to the
valuefield explaining its purpose and potential contents:class StoreDocument(KeyDocument): """Model representing a Store.""" key: str = "__store__" value: Optional[Dict[str, Any]] = None """Optional metadata or configuration for the store."""This addition would provide more context about the purpose of the
valuefield inStoreDocument.fiftyone/factory/repo_factory.py (1)
52-65: LGTM: Newexecution_store_repo()method is well-implemented.The new
execution_store_repo()method is correctly implemented and follows the existing pattern in the class. It properly uses the singleton pattern to ensure only one instance ofMongoExecutionStoreRepois created and reused.Consider expanding the docstring to provide more information about the method's purpose and return value. For example:
""" Factory method for creating or retrieving an execution store repository. Returns: ExecutionStoreRepo: An instance of the execution store repository. """fiftyone/operators/executor.py (1)
850-862: LGTM! Consider adding a return type hint.The
create_storemethod is well-implemented and documented. Here are some suggestions for improvement:
Add a return type hint to enhance code readability:
def create_store(self, store_name) -> ExecutionStore:Consider addressing the TODO comment about resolving the circular import in the future. This might involve restructuring the imports or using a type comment if the circular import cannot be resolved easily.
fiftyone/operators/store/service.py (2)
90-96: Remove reference to criteria inlist_storesdocstringThe docstring for the
list_storesmethod mentions "matching the given criteria," but the method does not accept any criteria parameters.Suggested Edit:
def list_stores(self): - """Lists all stores matching the given criteria. + """Lists all stores. Returns: a list of :class:`fiftyone.store.models.StoreDocument` """ return self._repo.list_stores()
109-115: Add a Returns section to the docstring oflist_keysmethodThe
list_keysmethod is missing aReturnssection in its docstring. Adding this section improves consistency and clarifies what the method returns.Suggested Addition:
def list_keys(self, store_name): """Lists all keys in the specified store. Args: store_name: the name of the store + Returns: + a list of :class:`fiftyone.store.models.KeyDocument` """ return self._repo.list_keys(store_name)
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (8)
- fiftyone/factory/repo_factory.py (2 hunks)
- fiftyone/factory/repos/execution_store.py (1 hunks)
- fiftyone/operators/executor.py (1 hunks)
- fiftyone/operators/store/init.py (1 hunks)
- fiftyone/operators/store/models.py (1 hunks)
- fiftyone/operators/store/service.py (1 hunks)
- fiftyone/operators/store/store.py (1 hunks)
- tests/unittests/execution_store_tests.py (1 hunks)
✅ Files skipped from review due to trivial changes (1)
- fiftyone/operators/store/init.py
🧰 Additional context used
🪛 Ruff
fiftyone/operators/store/store.py
10-10:
fiftyone.factory.repo_factory.RepositoryFactoryimported but unusedRemove unused import:
fiftyone.factory.repo_factory.RepositoryFactory(F401)
tests/unittests/execution_store_tests.py
11-11:
unittest.mockimported but unusedRemove unused import:
unittest.mock(F401)
12-12:
unittest.mock.patchimported but unusedRemove unused import
(F401)
12-12:
unittest.mock.ANYimported but unusedRemove unused import
(F401)
15-15:
fiftyone.operatorsimported but unusedRemove unused import:
fiftyone.operators(F401)
17-17:
fiftyone.operators.store.models.StoreDocumentimported but unusedRemove unused import:
fiftyone.operators.store.models.StoreDocument(F401)
20-20: Redefinition of unused
ExecutionStoreServicefrom line 16Remove definition:
ExecutionStoreService(F811)
🔇 Additional comments (5)
fiftyone/operators/store/models.py (2)
1-35: Reconsider the use of Pydantic and explore standard library alternativesThe use of Pydantic for model definitions in this file introduces a new dependency that other parts of the project have intentionally avoided. While Pydantic offers robust features, it's worth considering whether the benefits outweigh the cost of adding a new dependency.
Consider refactoring these models to use
dataclassesfrom the Python standard library, which can provide similar functionality without introducing a new dependency. This approach would align better with the rest of the project and simplify the dependency management.Here's a high-level example of how this refactoring might look:
from dataclasses import dataclass, field from typing import Optional, Dict, Any import datetime @dataclass class KeyDocument: store_name: str key: str value: Any created_at: datetime.datetime = field(default_factory=datetime.datetime.now) updated_at: Optional[datetime.datetime] = None expires_at: Optional[datetime.datetime] = None @staticmethod def get_expiration(ttl: Optional[int]) -> Optional[datetime.datetime]: # ... (same implementation as before) @dataclass class StoreDocument(KeyDocument): key: str = "__store__" value: Optional[Dict[str, Any]] = NoneThis refactoring maintains the structure and functionality of the models while eliminating the Pydantic dependency. If additional validation or serialization features are needed, consider implementing them using standard Python techniques or exploring lightweight alternatives that are already in use within the project.
To assess the impact of removing Pydantic, we can check its usage across the project:
#!/bin/bash # Search for Pydantic usage across the project rg --type python 'from pydantic import|import pydantic'This will help determine if Pydantic is used elsewhere or if this is an isolated case.
1-8: 🛠️ Refactor suggestionConsider alternatives to Pydantic and enhance the docstring
The use of Pydantic for model definitions has been a point of discussion in previous reviews. Consider using
dataclassesinstead, as it's part of the Python standard library and doesn't introduce additional dependencies. This aligns with the approach taken in other parts of the project.The file-level docstring could be more informative. Consider expanding it to briefly explain the purpose of the
KeyDocumentandStoreDocumentmodels and their role in the execution store.Here's a suggested improvement for the docstring:
""" Store and key models for the execution store. This module defines two main classes: - KeyDocument: Represents a key-value pair in the execution store. - StoreDocument: Represents a store entity, extending KeyDocument. These models provide a structured approach to managing data in the execution store. """To check if Pydantic is used elsewhere in the project:
✅ Verification successful
Pydantic usage is appropriately confined to
models.pyThe verification confirms that Pydantic is only utilized within
fiftyone/operators/store/models.py. This localized usage is suitable for defining robust data models without introducing unnecessary dependencies across the project.🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Search for other uses of Pydantic in the project rg --type python 'from pydantic import'Length of output: 88
Script:
#!/bin/bash # Search for other uses of Pydantic in the project rg --type py 'from pydantic import|import pydantic'Length of output: 126
fiftyone/factory/repo_factory.py (2)
16-19: LGTM: New imports are correctly added.The new imports for
ExecutionStoreRepoandMongoExecutionStoreRepoare correctly added and are consistent with the existing import style in the file. These imports are necessary for the newexecution_store_repo()method.
Line range hint
1-65: Summary: New execution store repository functionality added successfully.The changes to this file successfully introduce the new execution store repository functionality. The implementation is consistent with the existing code patterns and follows good practices. The new
execution_store_repo()method integrates well with theRepositoryFactoryclass and provides a clean way to manageExecutionStoreRepoinstances.These changes align with the PR objectives of introducing a new
ExecutionStorecomponent based on the existingDelegationOpsSvc(). The implementation seems solid and ready for integration with the rest of the system.To ensure the new functionality is properly integrated, please run the following verification script:
This script will help verify that the new
ExecutionStoreRepois being properly imported and used throughout the codebase, and identify any remaining TODOs related to the implementation.✅ Verification successful
ExecutionStore repository integration verified successfully.
The new
ExecutionStoreRepois properly integrated and utilized within the codebase:
RepositoryFactory.execution_store_repo()is used infiftyone/operators/store/service.py.No TODOs related to
ExecutionStorewere found, indicating the implementation is complete and requires no further actions.🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Description: Verify the usage of ExecutionStoreRepo in the codebase # Test 1: Check for imports of ExecutionStoreRepo echo "Checking for imports of ExecutionStoreRepo:" rg --type python "from fiftyone\.factory\.repos\.execution_store import ExecutionStoreRepo" # Test 2: Check for usage of RepositoryFactory.execution_store_repo() echo "Checking for usage of RepositoryFactory.execution_store_repo():" rg --type python "RepositoryFactory\.execution_store_repo\(\)" # Test 3: Check for any TODOs related to ExecutionStore echo "Checking for TODOs related to ExecutionStore:" rg --type python "TODO.*ExecutionStore"Length of output: 628
Script:
#!/bin/bash # Description: Verify the usage of ExecutionStoreRepo in the codebase using file extension patterns # Test 1: Check for imports of ExecutionStoreRepo echo "Checking for imports of ExecutionStoreRepo:" rg -g "*.py" "from fiftyone\.factory\.repos\.execution_store import ExecutionStoreRepo" # Test 2: Check for usage of RepositoryFactory.execution_store_repo() echo "Checking for usage of RepositoryFactory.execution_store_repo():" rg -g "*.py" "RepositoryFactory\.execution_store_repo\(\)" # Test 3: Check for any TODOs related to ExecutionStore echo "Checking for TODOs related to ExecutionStore:" rg -g "*.py" "TODO.*ExecutionStore"Length of output: 619
fiftyone/operators/store/store.py (1)
10-10: Unused importRepositoryFactoryis still present.The previous review comment about the unused import
from fiftyone.factory.repo_factory import RepositoryFactoryis still applicable.🧰 Tools
🪛 Ruff
10-10:
fiftyone.factory.repo_factory.RepositoryFactoryimported but unusedRemove unused import:
fiftyone.factory.repo_factory.RepositoryFactory(F401)
| "expires_at": time.time() + 60000, | ||
| } | ||
| value = self.store.get("widget_1") | ||
| assert value == {"name": "Widget One", "value": 100} |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Use self.assertEqual instead of plain assert.
Replace assert value == {"name": "Widget One", "value": 100} with self.assertEqual(value, {"name": "Widget One", "value": 100}) for clearer test assertions.
Apply this diff:
- assert value == {"name": "Widget One", "value": 100}
+ self.assertEqual(value, {"name": "Widget One", "value": 100})📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| assert value == {"name": "Widget One", "value": 100} | |
| self.assertEqual(value, {"name": "Widget One", "value": 100}) |
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (1)
- fiftyone/operators/store/service.py (1 hunks)
🔇 Additional comments (9)
fiftyone/operators/store/service.py (9)
1-15: LGTM: Imports and logger setup.The imports are relevant to the functionality of the
ExecutionStoreServiceclass, and the logger is set up correctly. The previously mentioned unusedtracebackimport has been removed, addressing the past review comment.
18-26: LGTM: Well-structured ExecutionStoreService class.The
ExecutionStoreServiceclass is well-designed with a clear purpose. The constructor correctly initializes the repo, allowing for dependency injection or using a default repo. The class provides a comprehensive set of methods for managing execution store operations.
27-36: LGTM: create_store method.The
create_storemethod is well-documented and correctly implemented. It provides a clear interface for creating a new store, delegating the actual creation to the underlying repository.
38-54: LGTM: set_key method, but verify TTL consistency.The
set_keymethod is well-implemented and documented. It correctly handles the optional TTL parameter in seconds. However, we need to verify the consistency of TTL units with theupdate_ttlmethod when we review it later.
56-66: LGTM: get_key method.The
get_keymethod is well-documented and correctly implemented. It provides a clear interface for retrieving a key's value from a specified store, delegating the actual retrieval to the underlying repository.
68-78: LGTM: delete_key method.The
delete_keymethod is well-documented and correctly implemented. It provides a clear interface for deleting a key from a specified store, delegating the actual deletion to the underlying repository. The boolean return value appropriately indicates the success or failure of the operation.
97-103: LGTM: list_stores method.The
list_storesmethod is well-documented and correctly implemented. It provides a clear interface for retrieving all stores, delegating the actual listing to the underlying repository. The method signature and documentation are consistent with its purpose of listing all stores without any filtering criteria.
105-114: LGTM: delete_store method.The
delete_storemethod is well-documented and correctly implemented. It provides a clear interface for deleting a specified store, delegating the actual deletion to the underlying repository. The return type ofStoreDocumentis appropriate, likely representing the deleted store's information.
116-125: LGTM: list_keys method.The
list_keysmethod is well-documented and correctly implemented. It provides a clear interface for retrieving all keys in a specified store, delegating the actual listing to the underlying repository. The return type ofList[str]is appropriate for representing a list of keys.
There was a problem hiding this comment.
Actionable comments posted: 4
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (3)
- fiftyone/operators/store/service.py (1 hunks)
- fiftyone/operators/store/store.py (1 hunks)
- tests/unittests/execution_store_tests.py (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- fiftyone/operators/store/service.py
- fiftyone/operators/store/store.py
🧰 Additional context used
🪛 Ruff
tests/unittests/execution_store_tests.py
11-11:
unittest.mock.patchimported but unusedRemove unused import
(F401)
11-11:
unittest.mock.ANYimported but unusedRemove unused import
(F401)
18-18: Redefinition of unused
ExecutionStoreServicefrom line 14Remove definition:
ExecutionStoreService(F811)
🔇 Additional comments (2)
tests/unittests/execution_store_tests.py (2)
1-226: Overall, the test suite provides good coverage of the ExecutionStore functionality.The tests cover various scenarios and edge cases for the ExecutionStore and related components. However, there are a few areas for improvement:
- Consider adding more edge cases, such as testing with empty stores or invalid inputs.
- Ensure consistent use of
unittestassertion methods throughout the test suite.- Add docstrings to test methods to clearly describe what each test is verifying.
- Consider using parameterized tests for similar test cases with different inputs.
Implementing these suggestions will further enhance the robustness and maintainability of the test suite.
🧰 Tools
🪛 Ruff
11-11:
unittest.mock.patchimported but unusedRemove unused import
(F401)
11-11:
unittest.mock.ANYimported but unusedRemove unused import
(F401)
18-18: Redefinition of unused
ExecutionStoreServicefrom line 14Remove definition:
ExecutionStoreService(F811)
124-125:⚠️ Potential issueFix time delta calculation in
test_update_ttl.The current calculation of
time_deltamay result in a negative value ifactual_expires_atis later thanexpected_expiration. To ensure a positive time delta, reverse the subtraction order.Apply this change to correct the calculation:
- time_delta = expected_expiration - actual_expires_at + time_delta = actual_expires_at - expected_expiration assert_delta_seconds_approx(time_delta, 0, epsilon=0.0001)Likely invalid or redundant comment.
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Outside diff range and nitpick comments (3)
tests/unittests/execution_store_tests.py (1)
9-11: Remove unused imports to improve code cleanliness.The
patchandANYimports fromunittest.mockare not used in the code. Consider removing them to keep the import statements clean.Apply this diff to remove the unused imports:
import time import unittest -from unittest.mock import patch, MagicMock, ANY, Mock +from unittest.mock import MagicMock, Mock🧰 Tools
🪛 Ruff
11-11:
unittest.mock.patchimported but unusedRemove unused import
(F401)
11-11:
unittest.mock.ANYimported but unusedRemove unused import
(F401)
fiftyone/operators/store/store.py (1)
13-13: Remove unused logger instanceThe
loggeris instantiated but never used in this class. Consider removing it to keep the code clean.Apply this diff to remove the unused logger:
-logger = logging.getLogger(__name__)fiftyone/operators/store/service.py (1)
46-53: Ensure consistent parameter documentation in docstringsThe parameter types and default values in the docstrings are not consistently specified. To enhance clarity and maintain consistency, consider explicitly stating the parameter types and default values using standard conventions.
For example, update the
ttlparameter in theset_keymethod:Args: store_name: the name of the store key: the key to set value: the value to set - ttl (None): an optional TTL in seconds + ttl (Optional[int], default=None): an optional TTL in seconds Returns: a :class:`fiftyone.store.models.KeyDocument`Similarly, review other method docstrings to ensure parameter types and default values are clearly specified.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
📒 Files selected for processing (4)
- fiftyone/factory/repos/execution_store.py (1 hunks)
- fiftyone/operators/store/service.py (1 hunks)
- fiftyone/operators/store/store.py (1 hunks)
- tests/unittests/execution_store_tests.py (1 hunks)
🧰 Additional context used
🪛 Ruff
fiftyone/operators/store/service.py
19-19: Undefined name
ExecutionStoreRepo(F821)
tests/unittests/execution_store_tests.py
11-11:
unittest.mock.patchimported but unusedRemove unused import
(F401)
11-11:
unittest.mock.ANYimported but unusedRemove unused import
(F401)
🔇 Additional comments (8)
tests/unittests/execution_store_tests.py (2)
1-225: Overall, well-structured and comprehensive test suite.The test suite covers a wide range of scenarios for the execution store functionality, including key operations, store management, and integration tests. The use of mocking is appropriate for isolating the tests from external dependencies.
A few minor improvements have been suggested to enhance code quality:
- Removing unused imports
- Correcting a class name typo
- Using unittest assertion methods consistently
- Fixing a time delta calculation
- Ensuring consistency in timestamp handling
After addressing these points, the test suite will be even more robust and maintainable. Great job on creating a thorough set of tests!
🧰 Tools
🪛 Ruff
11-11:
unittest.mock.patchimported but unusedRemove unused import
(F401)
11-11:
unittest.mock.ANYimported but unusedRemove unused import
(F401)
123-124:⚠️ Potential issueCorrect the time delta calculation in
test_update_ttl.The calculation of
time_deltaasexpected_expiration - actual_expires_atmay result in a negative value ifactual_expires_atis later thanexpected_expiration. To ensure a positive time delta, reverse the subtraction.Apply this diff to correct the calculation:
- time_delta = expected_expiration - actual_expires_at + time_delta = actual_expires_at - expected_expirationThis will make
time_deltapositive ifactual_expires_atis later thanexpected_expiration.Likely invalid or redundant comment.
fiftyone/operators/store/store.py (2)
52-61: LGTMThe
setmethod is implemented correctly, and the TTL handling is consistent across methods.
38-51: Implementation ofgetmethod is correctThe
getmethod correctly retrieves the value for a given key and returnsNoneif the key is not found.fiftyone/factory/repos/execution_store.py (4)
10-14: Previous comment still applies: Use type hints for_wherefunction
25-29: Previous comment still applies: Add error handling for database insertion increate_storemethod
76-80: Previous comment still applies: Add type hints to theget_keymethod parameters and return type
113-131: Previous comment still applies: Correct index existence checks in_create_indexesmethod
| {"store_name": "widgets", "key": "widget_2"}, | ||
| ] | ||
| keys = self.store_repo.list_keys("widgets") | ||
| assert keys == ["widget_1", "widget_2"] |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Use unittest assertion methods consistently.
Replace plain assert statements with appropriate unittest assertion methods for better error messages and test output.
Apply these changes throughout the test classes:
- assert keys == ["widget_1", "widget_2"]
+ self.assertEqual(keys, ["widget_1", "widget_2"])
- assert stores == ["widgets", "gadgets"]
+ self.assertEqual(stores, ["widgets", "gadgets"])
- assert value == {"name": "Widget One", "value": 100}
+ self.assertEqual(value, {"name": "Widget One", "value": 100})
- assert keys == ["widget_1", "widget_2"]
+ self.assertEqual(keys, ["widget_1", "widget_2"])
- assert deleted
+ self.assertTrue(deleted)Also applies to: 149-149, 193-193, 205-205, 214-214
| "created_at": time.time(), | ||
| "updated_at": time.time(), | ||
| "expires_at": time.time() + 60000, |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Use datetime.datetime.now() instead of time.time().
For consistency with other parts of the code that use datetime objects, consider using datetime.datetime.now() for timestamp fields.
Apply this diff:
- "created_at": time.time(),
- "updated_at": time.time(),
- "expires_at": time.time() + 60000,
+ "created_at": datetime.datetime.now(),
+ "updated_at": datetime.datetime.now(),
+ "expires_at": datetime.datetime.now() + datetime.timedelta(seconds=60000),This change ensures consistency in timestamp handling across the codebase.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| "created_at": time.time(), | |
| "updated_at": time.time(), | |
| "expires_at": time.time() + 60000, | |
| "created_at": datetime.datetime.now(), | |
| "updated_at": datetime.datetime.now(), | |
| "expires_at": datetime.datetime.now() + datetime.timedelta(seconds=60000), |
| class TeskKeyDocument(unittest.TestCase): | ||
| def test_get_expiration(self): | ||
| ttl = 1 | ||
| now = datetime.datetime.now() | ||
| expiration = KeyDocument.get_expiration(ttl) | ||
| time_delta = expiration - now | ||
| assert_delta_seconds_approx(time_delta, ttl) | ||
| assert isinstance(expiration, datetime.datetime) | ||
|
|
||
| def test_get_expiration_none(self): | ||
| ttl = None | ||
| expiration = KeyDocument.get_expiration(ttl) | ||
| assert expiration is None |
There was a problem hiding this comment.
Correct class name typo and improve assertions.
- There's a typo in the class name: "Tesk" should be "Test".
- Use
unittestassertion methods instead of plainassertstatements for better error messages.
Apply these changes to improve the test class:
-class TeskKeyDocument(unittest.TestCase):
+class TestKeyDocument(unittest.TestCase):
def test_get_expiration(self):
ttl = 1
now = datetime.datetime.now()
expiration = KeyDocument.get_expiration(ttl)
time_delta = expiration - now
- assert_delta_seconds_approx(time_delta, ttl)
- assert isinstance(expiration, datetime.datetime)
+ self.assertAlmostEqual(time_delta.total_seconds(), ttl, delta=EPSILON)
+ self.assertIsInstance(expiration, datetime.datetime)
def test_get_expiration_none(self):
ttl = None
expiration = KeyDocument.get_expiration(ttl)
- assert expiration is None
+ self.assertIsNone(expiration)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| class TeskKeyDocument(unittest.TestCase): | |
| def test_get_expiration(self): | |
| ttl = 1 | |
| now = datetime.datetime.now() | |
| expiration = KeyDocument.get_expiration(ttl) | |
| time_delta = expiration - now | |
| assert_delta_seconds_approx(time_delta, ttl) | |
| assert isinstance(expiration, datetime.datetime) | |
| def test_get_expiration_none(self): | |
| ttl = None | |
| expiration = KeyDocument.get_expiration(ttl) | |
| assert expiration is None | |
| class TestKeyDocument(unittest.TestCase): | |
| def test_get_expiration(self): | |
| ttl = 1 | |
| now = datetime.datetime.now() | |
| expiration = KeyDocument.get_expiration(ttl) | |
| time_delta = expiration - now | |
| self.assertAlmostEqual(time_delta.total_seconds(), ttl, delta=EPSILON) | |
| self.assertIsInstance(expiration, datetime.datetime) | |
| def test_get_expiration_none(self): | |
| ttl = None | |
| expiration = KeyDocument.get_expiration(ttl) | |
| self.assertIsNone(expiration) |
| class ExecutionStoreService: | ||
| """Service for managing execution store operations.""" | ||
|
|
||
| def __init__(self, repo: Optional["ExecutionStoreRepo"] = None): |
There was a problem hiding this comment.
Resolve undefined 'ExecutionStoreRepo' in type hint
The type hint for repo in the __init__ method references ExecutionStoreRepo, but ExecutionStoreRepo is imported inside the method. This causes an undefined name error during static analysis (line 19). To fix this issue, import ExecutionStoreRepo under an if TYPE_CHECKING: block at the top of the file.
Apply this diff to resolve the error:
+from typing import TYPE_CHECKING
+
+if TYPE_CHECKING:
+ from fiftyone.factory.repo_factory import ExecutionStoreRepo
+
import logging
from typing import Optional, List
from fiftyone.operators.store.models import StoreDocument, KeyDocumentThis ensures that the type hint is recognized by type checkers without affecting the runtime imports.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def __init__(self, repo: Optional["ExecutionStoreRepo"] = None): | |
| from typing import TYPE_CHECKING | |
| if TYPE_CHECKING: | |
| from fiftyone.factory.repo_factory import ExecutionStoreRepo | |
| import logging | |
| from typing import Optional, List | |
| from fiftyone.operators.store.models import StoreDocument, KeyDocument | |
| def __init__(self, repo: Optional["ExecutionStoreRepo"] = None): |
🧰 Tools
🪛 Ruff
19-19: Undefined name
ExecutionStoreRepo(F821)
| update_fields = { | ||
| "$set": { | ||
| k: v | ||
| for k, v in key_doc.to_mongo_dict().items() | ||
| if k | ||
| not in {"_id", "created_at", "expires_at", "store_name", "key"} | ||
| }, | ||
| "$setOnInsert": { | ||
| "store_name": store_name, | ||
| "key": key, | ||
| "created_at": now, | ||
| "expires_at": expiration if ttl else None, | ||
| }, | ||
| } |
There was a problem hiding this comment.
Potential issue: TTL is not updated when the key exists in set_key method
When updating an existing key using set_key, the expires_at field is not updated because it's excluded from the $set operation. As a result, the TTL remains unchanged, even if a new ttl value is provided.
To ensure that the TTL is updated when a new ttl is provided, consider including expires_at in the $set fields by removing it from the exclusion list.
Apply this diff to fix the issue:
update_fields = {
"$set": {
k: v
for k, v in key_doc.to_mongo_dict().items()
if k
- not in {"_id", "created_at", "expires_at", "store_name", "key"}
+ not in {"_id", "created_at", "store_name", "key"}
},
"$setOnInsert": {
"store_name": store_name,
"key": key,
"created_at": now,
- "expires_at": expiration if ttl else None,
},
+ }
+ if ttl is not None:
+ update_fields["$set"]["expires_at"] = expiration
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| update_fields = { | |
| "$set": { | |
| k: v | |
| for k, v in key_doc.to_mongo_dict().items() | |
| if k | |
| not in {"_id", "created_at", "expires_at", "store_name", "key"} | |
| }, | |
| "$setOnInsert": { | |
| "store_name": store_name, | |
| "key": key, | |
| "created_at": now, | |
| "expires_at": expiration if ttl else None, | |
| }, | |
| } | |
| update_fields = { | |
| "$set": { | |
| k: v | |
| for k, v in key_doc.to_mongo_dict().items() | |
| if k | |
| not in {"_id", "created_at", "store_name", "key"} | |
| }, | |
| "$setOnInsert": { | |
| "store_name": store_name, | |
| "key": key, | |
| "created_at": now, | |
| }, | |
| } | |
| if ttl is not None: | |
| update_fields["$set"]["expires_at"] = expiration |
What changes are proposed in this pull request?
The new
ExecutionStoreincluding several abstractions.This was based on the
DelegationOpsSvc().How is this patch tested? If it is not, please explain why.
There are two suites: unit and integration tests, plus an example python panel in fiftyone-plugins.
Release Notes
Is this a user-facing change that should be mentioned in the release notes?
notes for FiftyOne users.
(Details in 1-2 sentences. You can just refer to another PR with a description
if this PR is part of a larger change.)
What areas of FiftyOne does this PR affect?
fiftyonePython library changesSummary by CodeRabbit
Release Notes
New Features
Tests
ExecutionStoreServiceand related functionalities to ensure reliability and correctness.