Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

the best models case insensitive? #7

Closed
xiaoyubing opened this issue May 6, 2019 · 11 comments
Closed

the best models case insensitive? #7

xiaoyubing opened this issue May 6, 2019 · 11 comments

Comments

@xiaoyubing
Copy link

the best models case insensitive?can you give case sensitive best models?

@ku21fan
Copy link
Contributor

ku21fan commented May 9, 2019

Yes. the model is case-insensitive.
We will give case-sensitive models soon (maybe in 1 week?).

@ku21fan
Copy link
Contributor

ku21fan commented May 14, 2019

I just uploaded the case-sensitive model. please check here
But it could have low accuracy because the training data and validation data contains some case insensitive labeling.
In detail:

  1. Some fonts always generate the upper case even though the ground truth label have lower cases. Some example of MJSynth or SynthText might have been generated by these fonts.
  2. The training data of IIIT5k and SVT are used as the validation data. Their label is always the upper case, even though the ground truth label has the lower case. please check here

If you want a more accurate case sensitive model, fine-tuning this model with your own data or more cleaned case sensitive data.

Best.

@ku21fan ku21fan closed this as completed May 14, 2019
@xiaoyubing
Copy link
Author

thanks

@ku21fan
Copy link
Contributor

ku21fan commented May 15, 2019

I forgot to notify one thing.
use case-sensitive model with --sensitive option.

@solitaire2015
Copy link

@ku21fan Thanks for your great job. I have a question:
I test the pre-trained model of case-sensitive on IIIT5K 3000 by filtering special characters and change to upper case, I got a good accuracy: 90%. I wonder if the model is trained only on MJSynth + SYNText? The performance is incredible. Could you please explain why?

Best.

@ku21fan
Copy link
Contributor

ku21fan commented Jul 16, 2019

@solitaire2015
Yes, the model is trained only on MJSynth + SynthText.
But, in case --sensitive mode, we used all of SynthText.

After validation checks for each bounding box,
[a] SynthText has about 5.5 million images which do not contain special characters (only alphanumeric).
[b] SynthText has about 1.5 million images which contain special characters.

Our model was trained on [a] with case insensitive mode,
and trained on [a] and [b] with --sensitive mode
For now, only [a] is uploaded in our google / baidu drive, and I will upload [b] soon.

Best.

@solitaire2015
Copy link

@ku21fan That's great, thank you very much.

@ku21fan
Copy link
Contributor

ku21fan commented Jul 16, 2019

@solitaire2015 I just uploaded [b] as 'ST_spe.zip' at here
Download, unzip, and move it into the data_lmdb_release/training/
And then try to train with ST_spe :)

@solitaire2015
Copy link

@ku21fan I trained the model on --sensitive mode with the [a] + [b]. here are the training accuracy and loss:
[272000/300000] Loss: 0.06758 elapsed_time: 135756.64656
acer , gt: acer , True
Endless , gt: ENDLESS , False
BBQ , gt: BBQ , True
The , gt: The , True
THE , gt: THE , True
[272000/300000] valid loss: 2.01074 accuracy: 64.745, norm_ED: 1728.75
best_accuracy: 66.390, best_norm_ED: 1609.72

I trained this model on two 1080Ti, selected the best combination, TPS + ResNet + BiLSTM + Attn, for some reasons I add one convolutional layer at the end of ResNet feature extractor to keep the output width to be 25(26 before), I test the model on IIIT5k 3000, after filtering special characters and converting lower case to upper case, I got total accuracy: 88.5%, a little bit lower than yours 90%.

I investigate some failure cases of IIIT5K, some special characters are treated as alphanumeric characters, I think it's the reason why the sensitive model is more accurate than the insensitive model.

Best.

@alwc
Copy link

alwc commented Jul 20, 2019

Hi @solitaire2015, did you use grayscale or rgb mode for your input images?

@solitaire2015
Copy link

@alwc I use grayscale images.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants