Commits on Oct 16, 2024

1. Add retries to evaluator client …

   * During evaluation we are seeing ocassional timeouts on the server due to
     the HTTP read/write timeout occuring. It looks like this might happen
     because ChatGPT takes a really long time to respond occassionally.
   
   * Update StreamGenerate and GenerateCells to return DeadlineExceeded to
     indicate a server timeout
   
   * Implement a unaryinterceptor to automatically retry requests
     based on the status code.
   
   * The retry is pretty simplistic; its a fixed backoff.

   

   jlewi committed Oct 16, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for 8aabb0e
   • Browse repository at this point

   Copy the full SHA

   8aabb0e View commit details

   Browse the repository at this point in the history