Skip to content

Commit

Permalink
fix tutorial part-3
Browse files Browse the repository at this point in the history
  • Loading branch information
yuantailing committed Dec 20, 2018
1 parent f9c70fc commit ff97954
Showing 1 changed file with 13 additions and 13 deletions.
26 changes: 13 additions & 13 deletions tutorial/3-detection.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -449,23 +449,23 @@
"\n",
"Our Python evaluation API in `pythonapi/eval_tools.py` and C++ evaluation API `cppapi/eval_tools.hpp` work as follows, which is similar to PASCAL VOC.\n",
"\n",
" 1. Check the detection (DT) file has the same number of lines as the grount truth (GT) file. Otherwise, return error.\n",
" 1. Check the detection (DT) file has the same number of lines as the ground truth (GT) file. Otherwise, return error.\n",
" 1. Check each line of the DT file is valid JSON, and conform to the submission format. Otherwise, return error.\n",
" 1. Use the ignore list (IG) in the annotations.\n",
" 1. Remove non-Chinese character instances from GTs.\n",
" 1. For each size, we deal with DTs, GTs, and IGs of each image in following steps, respectively.\n",
" 1. Move GTs which are not fit to current size range to IG. (For size 'all', this step always has no effect.)\n",
" 1. Match DTs with GTs greedily, order by $IOU(DT, GT)$ in descending order. For any given confidence score $c_0$, matched DTs which confidence score are greater than $c_0$ are true positives (TPs), while other matched DTs are false positives (FPs).\n",
" 1. Remove unmatched DTs which can match IGs. They will have no effect to the evaluation.\n",
" 1. Remove unmatched DTs which are not fit to current size range. (For size 'all', this step always has no effect.)\n",
" 1. For each size, we deal with DTs, GTs, and IGs of each image in the following steps, respectively.\n",
" 1. Move GTs which are not fit current size range to IG. (For size 'all', this step always has no effect.)\n",
" 1. Match DTs with GTs greedily. We consider DTs in score order. For each DT, GTs are ordered by $IOU(DT, GT)$ in descending order. For any given confidence score $c_0$, matched DTs which confidence score are greater than $c_0$ are true positives (TPs), while other matched DTs are false positives (FPs).\n",
" 1. Remove unmatched DTs which can match IGs. They will have no effect on the evaluation.\n",
" 1. Remove unmatched DTs which are not fit current size range. (For size 'all', this step always has no effect.)\n",
" 1. Remaining DTs are FPs, Remaining GTs are false negatives (FNs).\n",
" 1. For each size, we compute metrics in following steps, respectively.\n",
" 1. Take all TPs, FNs and FPs to compute `AP`.\n",
" 1. For each character category, take TPs, FNs and FPs in specified category to compute average precision (AP). Compute mean of these APs weighted by number of character instances in corresponding category, denote the mean value as `macro-mAP`, also called `mAP`.\n",
" 1. For each image, we take a minimum confidence score $c_0$ which leads to $num(TPs) + num(FNs) \\leq num(GTs)$, respectively. Then,\n",
" 1. for each attribute, compute `recall` by counting TPs which $score > c_0$ and belong to specified attribute for each the image, respectively.\n",
" 1. for each character category, compute `recall` by counting TPs which $score > c_0$ and belong to specified category for each image, respectively.\n",
" 1. For each image, take TPs, FNs and FPs in specified image to compute AP. Compute the mean of these APs, denote this mean value as `micro-mAP`.\n",
" 1. Take all TPs, FNs, and FPs to compute `AP`.\n",
" 1. For each character category, take TPs, FNs, and FPs in the specified category to compute average precision (AP). Compute mean of these APs weighted by the number of character instances in the corresponding category, denote the mean value as `macro-mAP`, also called `mAP`.\n",
" 1. For each image, we take a minimum confidence score $c_0$ which leads to $num(TPs) + num(FPs) \\leq num(GTs)$, respectively. Then,\n",
" 1. for each attribute, compute `recall` by counting TPs which $score > c_0$ and belong to the specified attribute for each image, respectively.\n",
" 1. for each character category, compute `recall` by counting TPs which $score > c_0$ and belong to the specified category for each image, respectively.\n",
" 1. For each image, take TPs, FNs, and FPs in the specified image to compute AP. Compute the mean of these APs, denote this mean value as `micro-mAP`.\n",
"\n",
"When matching a DT with a GT, we require they have the identical character category, and $IOU(DT, GT) > 0.5$. When matching a DT with a IG, we require $\\exists ig \\subseteq IG$, s.t. $IOU(DT, ig) > 0.5$. Of which $IOU(A, B) = \\frac{Area(A \\cap B)}{Area(A \\cup B)}$. Otherwise, they are not matched.\n",
"\n",
Expand Down Expand Up @@ -497,7 +497,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.4.3"
"version": "3.6.3"
}
},
"nbformat": 4,
Expand Down

0 comments on commit ff97954

Please sign in to comment.