How to evaluate algorithm metrics using the test set of the DIVO dataset?

[wlf-darkmatter]

I have trained my model using the content of the DIVO train set, and now I would like to assess my model's performance metrics using the ground truth from the test set. However, the test dataset only contains det.txt information without any id information. How can I evaluate my model's performance using the test set in this situation?

[shengyuhao]

I have trained my model using the content of the DIVO train set, and now I would like to assess my model's performance metrics using the ground truth from the test set. However, the test dataset only contains det.txt information without any id information. How can I evaluate my model's performance using the test set in this situation?

Hi, we have released 5 scenes of the test set, you can download them from this link.

[wlf-darkmatter]

Thanks so much, I have received gt that you offered. But there might be some error annotations where the width (w) and height (h) in some anchor boxes' x_l, y_t, w, h are negative.

I resolved the format as ["frame", "id", "x_l", "y_t", "w", "h", "prob", "cls", "vis"]

example data in Circle_View1/gt.txt of lines=4430-4486, where abnormal lines =4433-4483

1253,4,839,1055,18,22,1,1,1
1254,4,836,1061,12,16,1,1,1
1255,4,833,1068,7,10,1,1,1
# -----------below is abnormal---------------
1261,4,815,1080,-24,-1,1,1,1
1262,4,812,1080,-30,-1,1,1,1
1263,4,809,1080,-36,-1,1,1,1
1264,4,807,1080,-42,-1,1,1,1
1265,4,804,1080,-48,-1,1,1,1
1266,4,802,1080,-54,-1,1,1,1
1267,4,799,1080,-60,-1,1,1,1
1268,4,796,1080,-66,-1,1,1,1
1269,4,794,1080,-72,-1,1,1,1
1270,4,791,1080,-78,-1,1,1,1
1271,4,789,1080,-85,-1,1,1,1
1272,4,786,1080,-91,-1,1,1,1
1273,4,784,1080,-97,-1,1,1,1
1274,4,781,1080,-103,-1,1,1,1
1275,4,779,1080,-109,-1,1,1,1
1276,4,776,1080,-116,-1,1,1,1
1277,4,774,1080,-122,-1,1,1,1
1278,4,771,1080,-128,-1,1,1,1
1279,4,769,1080,-134,-1,1,1,1
1280,4,766,1080,-140,-1,1,1,1
1281,4,764,1080,-147,-1,1,1,1
1282,4,761,1080,-153,-1,1,1,1
1283,4,759,1080,-159,-1,1,1,1
1284,4,756,1080,-165,-1,1,1,1
1285,4,754,1080,-171,-1,1,1,1
1286,4,751,1080,-178,-1,1,1,1
1287,4,749,1080,-184,-1,1,1,1
1288,4,746,1080,-190,-1,1,1,1
1289,4,744,1080,-196,-1,1,1,1
1290,4,741,1080,-202,-1,1,1,1
1291,4,739,1080,-209,-1,1,1,1
1292,4,736,1080,-215,-1,1,1,1
1293,4,734,1080,-221,-1,1,1,1
1294,4,731,1080,-227,-1,1,1,1
1295,4,729,1080,-234,-1,1,1,1
1296,4,727,1080,-240,-1,1,1,1
1297,4,724,1080,-246,-1,1,1,1
1298,4,722,1080,-253,-1,1,1,1
1299,4,719,1080,-259,-1,1,1,1
1300,4,717,1080,-265,-1,1,1,1
1301,4,715,1080,-272,-1,1,1,1
1302,4,712,1080,-278,-1,1,1,1
1303,4,710,1080,-284,-1,1,1,1
1304,4,707,1080,-291,-1,1,1,1
1305,4,705,1080,-297,-1,1,1,1
1306,4,703,1080,-304,-1,1,1,1
1307,4,700,1080,-310,-1,1,1,1
1308,4,698,1080,-316,-1,1,1,1
1309,4,695,1080,-323,-1,1,1,1
1310,4,693,1080,-329,-1,1,1,1
1311,4,691,1080,-336,-1,1,1,1
# -----------upper is abnormal---------------
1541,4,31,853,130,223,1,1,1
1542,4,33,849,132,226,1,1,1
1543,4,35,845,134,230,1,1,1

[shengyuhao]

Thanks! We have removed these error annotations and updated them. All files are available at this link.

[youonlytrackonce]

Thanks so much, I have received gt that you offered. But there might be some error annotations where the width (w) and height (h) in some anchor boxes' x_l, y_t, w, h are negative.

I resolved the format as ["frame", "id", "x_l", "y_t", "w", "h", "prob", "cls", "vis"]

example data in Circle_View1/gt.txt of lines=4430-4486, where abnormal lines =4433-4483

1253,4,839,1055,18,22,1,1,1
1254,4,836,1061,12,16,1,1,1
1255,4,833,1068,7,10,1,1,1
# -----------below is abnormal---------------
1261,4,815,1080,-24,-1,1,1,1
1262,4,812,1080,-30,-1,1,1,1
1263,4,809,1080,-36,-1,1,1,1
1264,4,807,1080,-42,-1,1,1,1
1265,4,804,1080,-48,-1,1,1,1
1266,4,802,1080,-54,-1,1,1,1
1267,4,799,1080,-60,-1,1,1,1
1268,4,796,1080,-66,-1,1,1,1
1269,4,794,1080,-72,-1,1,1,1
1270,4,791,1080,-78,-1,1,1,1
1271,4,789,1080,-85,-1,1,1,1
1272,4,786,1080,-91,-1,1,1,1
1273,4,784,1080,-97,-1,1,1,1
1274,4,781,1080,-103,-1,1,1,1
1275,4,779,1080,-109,-1,1,1,1
1276,4,776,1080,-116,-1,1,1,1
1277,4,774,1080,-122,-1,1,1,1
1278,4,771,1080,-128,-1,1,1,1
1279,4,769,1080,-134,-1,1,1,1
1280,4,766,1080,-140,-1,1,1,1
1281,4,764,1080,-147,-1,1,1,1
1282,4,761,1080,-153,-1,1,1,1
1283,4,759,1080,-159,-1,1,1,1
1284,4,756,1080,-165,-1,1,1,1
1285,4,754,1080,-171,-1,1,1,1
1286,4,751,1080,-178,-1,1,1,1
1287,4,749,1080,-184,-1,1,1,1
1288,4,746,1080,-190,-1,1,1,1
1289,4,744,1080,-196,-1,1,1,1
1290,4,741,1080,-202,-1,1,1,1
1291,4,739,1080,-209,-1,1,1,1
1292,4,736,1080,-215,-1,1,1,1
1293,4,734,1080,-221,-1,1,1,1
1294,4,731,1080,-227,-1,1,1,1
1295,4,729,1080,-234,-1,1,1,1
1296,4,727,1080,-240,-1,1,1,1
1297,4,724,1080,-246,-1,1,1,1
1298,4,722,1080,-253,-1,1,1,1
1299,4,719,1080,-259,-1,1,1,1
1300,4,717,1080,-265,-1,1,1,1
1301,4,715,1080,-272,-1,1,1,1
1302,4,712,1080,-278,-1,1,1,1
1303,4,710,1080,-284,-1,1,1,1
1304,4,707,1080,-291,-1,1,1,1
1305,4,705,1080,-297,-1,1,1,1
1306,4,703,1080,-304,-1,1,1,1
1307,4,700,1080,-310,-1,1,1,1
1308,4,698,1080,-316,-1,1,1,1
1309,4,695,1080,-323,-1,1,1,1
1310,4,693,1080,-329,-1,1,1,1
1311,4,691,1080,-336,-1,1,1,1
# -----------upper is abnormal---------------
1541,4,31,853,130,223,1,1,1
1542,4,33,849,132,226,1,1,1
1543,4,35,845,134,230,1,1,1

Hi,

what tracker did you train?

Were you be able to evaluate by using test scripts in the repos such as :
DIVOTrack/Single_view_Tracking/CenterTrack/experiments/

Thanks.