Mark recovery_in_prog_ to false whenever recovery thread joins (#11890)

Summary:
Make the `RecoverFromRetryableBGIOError` function always mark `recovery_in_prog_` to false when it returns.
Otherwise, in below code snippet, when db closes and the `error_handler_.CancelErrorRecovery()` call successfully joined the recovery thread, the immediately following while loop will incorrectly think the error recovery is still in progress and loops in `bg_cv_.Wait()`.

https://github.com/facebook/rocksdb/blob/1c871a4d8682ea260ba3b18ed43cd525a2141733/db/db_impl/db_impl.cc#L542-L545

This is the issue #11440

Pull Request resolved: #11890

Reviewed By: anand1976

Differential Revision: D49624216

Pulled By: jowlyzhang

fbshipit-source-id: ee10cf6527d95b8dd4705a326eb6208d741fe002

db/error_handler.cc

@@ -676,6 +676,8 @@ void ErrorHandler::RecoverFromRetryableBGIOError() {
     EventHelpers::NotifyOnErrorRecoveryEnd(db_options_.listeners, bg_error_,
                                            Status::ShutdownInProgress(),
                                            db_mutex_);
+
+    recovery_in_prog_ = false;
     return;
   }
   DBRecoverContext context = recover_context_;
@@ -689,6 +691,7 @@ void ErrorHandler::RecoverFromRetryableBGIOError() {
       EventHelpers::NotifyOnErrorRecoveryEnd(db_options_.listeners, bg_error_,
                                              Status::ShutdownInProgress(),
                                              db_mutex_);
+      recovery_in_prog_ = false;
       return;
     }
     TEST_SYNC_POINT("RecoverFromRetryableBGIOError:BeforeResume0");