Skip to content

Latest commit

 

History

History
211 lines (155 loc) · 4.31 KB

BNDCG.md

File metadata and controls

211 lines (155 loc) · 4.31 KB
Raw

TFSimilarity.retrieval_metrics.BNDCG

Binary normalized discounted cumulative gain.

Inherits From: RetrievalMetric, ABC

TFSimilarity.retrieval_metrics.BNDCG(
    name: str = ndcg,
    k: int = 5,
    distance_threshold: float = math.inf,
    **kwargs
) -> None

This is normalized discounted cumulative gain where the relevancy weights are binary, i.e., either a correct match or an incorrect match.

The NDCG is a score between [0,1] representing the rank weighted results. The DCG represents the sum of the correct matches weighted by the log2 of the rank and is normalized by the 'ideal DCG'. The IDCG is computed as the match_mask, sorted descending, weighted by the log2 of the post sorting rank order. This metric takes into account both the correctness of the match and the position.

The normalized DCG is computed as:

$$ nDCG_p} = \frac{DCG_{p}}{IDCG_{p} $$

The DCG is computed for each query using the match_mask as:

$$ DCG_p} = \sum_{i=1}^{p} \frac{match_mask_{i}}{\log_{2}(i+1) $$

The IDCG uses the same equation but sorts the match_mask descending along axis=-1.

Additionally, all positive matches with a distance above the threshold are set to 0, and the closest K matches are taken.

Args

name Name associated with the metric object, e.g., precision@5
canonical_name The canonical name associated with metric, e.g., precision@K
k The number of nearest neighbors over which the metric is computed.
distance_threshold The max distance below which a nearest neighbor is considered a valid match.
average 'micro', 'macro' Determines the type of averaging performed on the data.

  • 'micro': Calculates metrics globally over all data.

  • 'macro': Calculates metrics for each label and takes the unweighted mean.

Attributes

name

Methods

compute

View source

compute(
    *,
    query_labels: <a href="../../TFSimilarity/callbacks/IntTensor.md">TFSimilarity.callbacks.IntTensor```
</a>,
    lookup_distances: <a href="../../TFSimilarity/callbacks/FloatTensor.md">TFSimilarity.callbacks.FloatTensor```
</a>,
    match_mask: <a href="../../TFSimilarity/utils/BoolTensor.md">TFSimilarity.utils.BoolTensor```
</a>,
    **kwargs
) -> <a href="../../TFSimilarity/callbacks/FloatTensor.md">TFSimilarity.callbacks.FloatTensor```
</a>

Compute the metric

Computes the binary NDCG. The query labels are only used when the averaging is set to "macro".

Args
query_labels A 1D array of the labels associated with the embedding queries.
lookup_distances A 2D array where the jth row is the distances between the jth query and the set of k neighbors.
match_mask A 2D mask where a 1 indicates a match between the jth query and the kth neighbor and a 0 indicates a mismatch.
Returns
A rank 0 tensor containing the metric.

get_config

View source

get_config()