Questions on verification_RFW and training procedure

[lizhenstat]

Hi,
Thanks for your great work and sharing of the code on these two papers !
It takes me days to read the paper and go through the repository and I have a few questions:

(2)
Do you have the code for training the features (asian_females, asian_males, black_females, black_males, indian_females, indian_males,...). Since I have a hard time finding something like train.py (e.g. the loss function and training process).
(I suppose the released code is mainly on image pre-processing and result analysis)
(Since BFW dataset is not as large as other face dataset and it may possible for me to train it from scratch on one GPU)

(3)
I am little confused about how the BFW is used in two papers, as I understand:

in paper Face Recognition: Too Bias, or Not Too Bias? , the train and test model are as follows:
train: CASIA_webface trained using Sphereface loss
test: LFW
where does BFW dataset not used in training in this set of experiments?

in paper Balancing Biases and Preserving Privacy on Balanced Faces in the Wild the train, test model are as follows:
tain:
(1) MS1M trained using Arcface loss --> to get 512-dim embedding (f_in in Fig.6)
(2) BFW dataset is used to train the encoder and two classifiers in Fig 6
test: 4-folds used for training and 1-fold used for testing (using the best threshold chosen)

is that right?

(4)
There are some difference from "bfw-v0.1.5-datatable.csv" and the TABLE-2 in paper 2:
for example: there are 921379 records in TABLE-2 while ther are 923898 records from the csv file?
and there is no "{dir_meta}thresholds.pkl" file.

Thanks for your time and any help would be appreciated !

[visionjo]

Thank you for your reply; also, for expressing your interest. All much appreciated. I have been a little busy this week, but by the EOD Friday, I will get back to you on this! Just wanted to give you a heads up in meantime. Cheers, Joe

…

-- Joseph Robinson, Ph.D. Email: ***@***.******@***.***> Website: www.jrobsvision.com<http://www.jrobsvision.com/> Follow me on ***@***.***>! Cell: +1 (978) 764-0003

________________________________ From: lizhen ***@***.***> Sent: Tuesday, August 10, 2021 4:05 AM To: visionjo/facerec-bias-bfw ***@***.***> Cc: Subscribed ***@***.***> Subject: [visionjo/facerec-bias-bfw] Questions on verification_RFW and training procedure (#61) Hi, Thanks for your great work and sharing of the code on these two papers ! It takes me days to read the paper and go through the repository and I have a few questions: (1) from readme.txt in folder Accuracy_for_RFW, we should put "verification_RFW.py in the following folder: ./src/eval/" in insightFace repo but I cannot find this repo ( I assume it is because insightFace may have changed over the past year, can you tell me which version of insightFace is compatible with verification_RFW.py since I want to test that, thanks) (2) Do you have the code for training the features (asian_females, asian_males, black_females, black_males, indian_females, indian_males,...). Since I have a hard time finding something like train.py (e.g. the loss function and training process). (I suppose the released code is mainly on image pre-processing and result analysis) (Since BFW dataset is not as large as other face dataset and it may possible for me to train it from scratch on one GPU) Thanks for your time and any help would be appreciated ! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fvisionjo%2Ffacerec-bias-bfw%2Fissues%2F61&data=04%7C01%7Crobinson.jo%40northeastern.edu%7C35d3fad0dca843720b6308d95bd5a3db%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637641795404517748%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=yQ6cOOLmwRRdpr725kSdZ0%2FqjYTx3oGnDxzVbbOIa%2FM%3D&reserved=0>, or unsubscribe<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABAGNZMPTBJ5WTLQO6XNC2TT4DMVDANCNFSM5B3TPMSA&data=04%7C01%7Crobinson.jo%40northeastern.edu%7C35d3fad0dca843720b6308d95bd5a3db%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637641795404527744%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=L2vwjqyYqUZXhxWjjRneJeDMS0EYDh6%2BBmAbN93imvw%3D&reserved=0>. Triage notifications on the go with GitHub Mobile for iOS<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Crobinson.jo%40northeastern.edu%7C35d3fad0dca843720b6308d95bd5a3db%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637641795404537735%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=yxr6YxgPdegxznt5Nx52KX0Mk60%2BFhfesn1wqIWbAgk%3D&reserved=0> or Android<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26utm_campaign%3Dnotification-email&data=04%7C01%7Crobinson.jo%40northeastern.edu%7C35d3fad0dca843720b6308d95bd5a3db%7Ca8eec281aaa34daeac9b9a398b9215e7%7C0%7C0%7C637641795404537735%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2FLaFZ1%2BR%2BTMVKnPhTwbYZdfNt4rkTI7I%2FnDWdsPXe%2FE%3D&reserved=0>.

[lizhenstat]

@visionjo hi, thanks for your reply!
Besides, I also found the pictures in the following folder:
"jrobby\bfw\bfw-cropped-aligned\asian_females\n009042"
the following three pictures(0015_01.jpg, 0034_01.jpg, 0394_01.jpg) belong to Guo Jing Jing while folder "n009042" is of Wu Min Xia, both of them are famous Diving Athlete in China,

I think there are many picutures of them together from the internet then there maybe some confuse.

Here is the snapshot of the folder "n009042":

[lizhenstat]

@visionjo Hi, any update here? thanks!

[visionjo]

Thanks for pointing this out. We are currently integrating the data as part of the fiftyone datasets, and as part of this an updated version of data will be available. This bug will be addressed and released with this ASAP (within weeks). Thanks again and do post additional items that come up 👌

[visionjo]

I'll provide the features and answer your original question sooner (a night this week). My apologies, I lost track of this. Thanks for reaching back out.

[lizhenstat]

@visionjo Hi, thanks, can you notice me if you update the dataset. Thanks a lot !

[jrobinson-vs]

@lizhenstat, I have been busy with our other dataset (Families In the Wild (FIW)), as the challenge just ended and we are preparing the white paper. With my time limited to spare time around full-time work, progress has been a bit slow here. Nonetheless, as mentioned prior, we are returning to BFW shortly (within the month), and have a few plans (including releasing the next version of data). Feel free to PR any changes (whether to data or code)-- it would be highly appreciated :)

In any case, I will check back shortly and get back to you on this. Thanks for reaching out!