prov/shm: Fix revert PR ofiwg#9377 to revert correct lock

I mistakenly reverted the wrong lock on PR#9377, and caused a performance regression. This change makes the ep->lock be domain dependent, and properly revers util_cntr ep_list_lock to always be mutex. This restores most of the performance regression. Signed-off-by: Seth Zegelstein <szegel@amazon.com>
a-szegel · Oct 23, 2023 · d091fd4 · d091fd4
1 parent d81247e
commit d091fd4
Show file tree

Hide file tree

Showing 2 changed files with 4 additions and 5 deletions.
diff --git a/prov/util/src/util_cntr.c b/prov/util/src/util_cntr.c
@@ -390,10 +390,7 @@ int ofi_cntr_init(const struct fi_provider *prov, struct fid_domain *domain,
 			return ret;
 	}
 
-	ofi_genlock_init(&cntr->ep_list_lock,
-			 cntr->domain->threading == FI_THREAD_DOMAIN ||
-			 cntr->domain->threading == FI_THREAD_COMPLETION  ?
-			 OFI_LOCK_NOOP : OFI_LOCK_MUTEX);
+	ofi_genlock_init(&cntr->ep_list_lock, OFI_LOCK_MUTEX);
 	ofi_atomic_inc32(&cntr->domain->ref);
 
 	/* CNTR must be fully operational before adding to wait set */

diff --git a/prov/util/src/util_ep.c b/prov/util/src/util_ep.c
@@ -248,7 +248,9 @@ int ofi_endpoint_init(struct fid_domain *domain, const struct util_prov *util_pr
 		ofi_ep_bind_eq(ep, util_domain->eq);
 
 	/* TODO Figure out how to optimize this lock for rdm and msg endpoints */
-	ret = ofi_genlock_init(&ep->lock, OFI_LOCK_MUTEX);
+	ret = ofi_genlock_init(&ep->lock,
+			       ep->domain->threading != FI_THREAD_SAFE ?
+			       OFI_LOCK_NOOP : OFI_LOCK_MUTEX);
 	if (ret)
 		return ret;