Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scalability fixes - Load model once per user #944

Merged
merged 18 commits into from
Dec 2, 2023

Commits on Nov 9, 2023

  1. Scalability fixes - Load model once per user

    Implementing code changes for improving scalability as per the requirements in this issue# 950 in e-mission-docs.
    
    Initial approach involves utilizing Singleton design pattern concept on instance variables to check whether model has already been loaded before attempting to load the model.
    
    Thus this should prevent model from being loaded if it is already present in the instance variable of the current class instance of BuiltinModelStorage.
    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    408cad3 View commit details
    Browse the repository at this point in the history
  2. Missing imports and self keyword

    Added import for logging.
    Added self parameter to pass reference to current instance object.
    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    7d5b5d2 View commit details
    Browse the repository at this point in the history

Commits on Nov 17, 2023

  1. Refactoring label inference pipeline for scalability issues

    Changes after refactoring label inference pipeline to load model only once. Will be merging with previous PR branch.
    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Nov 17, 2023
    Configuration menu
    Copy the full SHA
    f924776 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    a7668f5 View commit details
    Browse the repository at this point in the history
  3. Removed singleton approach

    Starting from scratch to implement refactored label pipeline approach
    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Nov 17, 2023
    Configuration menu
    Copy the full SHA
    d973da3 View commit details
    Browse the repository at this point in the history
  4. Merge branch 'refactor-label-pipeline' into label-scalability

    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Nov 17, 2023
    Configuration menu
    Copy the full SHA
    d042e1b View commit details
    Browse the repository at this point in the history
  5. Changed results_list to result_dict

    Using a dictionary of lists of predictions returned for each trip instead of a simple list.
    This will help keep predictions of each trip separate but keep all the algorithms_wise predictions of each trip together in a list.
    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Nov 17, 2023
    Configuration menu
    Copy the full SHA
    fb278ff View commit details
    Browse the repository at this point in the history
  6. Fixed function call to predict_labels_with_n in TestRunGreedyModel

    Updated function call to predict_labels_with_n() as per latest changes to include trip_list and user_id
    
    Fixed typo in eacilp for trip.get_id()
    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Nov 17, 2023
    Configuration menu
    Copy the full SHA
    30c02a4 View commit details
    Browse the repository at this point in the history
  7. Fixed function signature in TestRunGreedyModel

    eamur.predict_labels_with_n() now takes a trip instead of a trip_list.
    Passed a trip_list in previous commit but forgot to update parameter name.
    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Nov 17, 2023
    Configuration menu
    Copy the full SHA
    b15a0e0 View commit details
    Browse the repository at this point in the history
  8. Updated TestRunGreedyModel + Debug Cleanup

    Receive a prediction list of (prediction, n) for each trip in trip_list.
    
    Then for each of the predictions in the prediction list, run the assertion tests.
    
    Also, cleaned up some debug print statements from earlier commits / PRs.
    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Nov 17, 2023
    Configuration menu
    Copy the full SHA
    54a42a8 View commit details
    Browse the repository at this point in the history
  9. Cleanup print statements

    Cleaning up previously used debug statements
    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Nov 17, 2023
    Configuration menu
    Copy the full SHA
    7f70be9 View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2023

  1. Added debug statements to track time taken for operations.

    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Nov 21, 2023
    Configuration menu
    Copy the full SHA
    6c2c9c0 View commit details
    Browse the repository at this point in the history
  2. Cleaned up some comments and debug statements.

    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Nov 21, 2023
    Configuration menu
    Copy the full SHA
    433b40a View commit details
    Browse the repository at this point in the history

Commits on Nov 22, 2023

  1. Fixed failing analysis pipeline tests involved in Label Inference

    A couple of pipeline tests were failing - TestLableInferencePipeline, TestExpectationPipeline.
    Failed since they were calling placeholder_predict_n functions in eacili inferrers.
    Had to update these to now receive trip_list instead of single trip.
    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    9032ce0 View commit details
    Browse the repository at this point in the history

Commits on Nov 27, 2023

  1. Removed user_id parameter

    Fetching from trip_list itself.
    Added assertion to ensure identical and only one unique user_id present for all trips.
    Assertion fails if multiple user_ids found for the trip_list.
    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    7950981 View commit details
    Browse the repository at this point in the history
  2. Removed user_id parameter from Tests

    Removed from Test files and inferrers.py which contained placeholder_predictor functions.
    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    425966a View commit details
    Browse the repository at this point in the history

Commits on Dec 1, 2023

  1. Moved model loading step out of predict_labels_with_n

    Refactored code to pass in the trip model directly to predict_labels_with_n() in eamur.
    
    Moved the load model step to eacil.inferrers by using load_model() of eamur.
    
    Modified TestRunGreedyModel to use this refactored function.
    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    64232ab View commit details
    Browse the repository at this point in the history
  2. Passed user_id in TestRunGreedyModel.py

    Failing test due to missed user_id parameter.
    Passed user_id to rectify.
    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Dec 1, 2023
    Configuration menu
    Copy the full SHA
    3372604 View commit details
    Browse the repository at this point in the history