mtl-text-recognition

multi-task learning for text recognition with joint ctc-attention.

Update features

• support variable length of squence training and inference (fixed height).
• Chinese character recognition.
• Joint CTC-Attention
  
  
  
  

Getting Started

Dependency

• This work was tested with PyTorch 1.1.0, CUDA 9.0, python 3.6 and centos7
• requirements : pytorch, lmdb, pillow, torchvision, nltk, natsort

pip3 install torch==1.1.0
pip3 install lmdb pillow torchvision nltk natsort

Run demo with pretrained model(中文+英文字符版本，使用config中的chn.txt文件)

1. Download pretrained model(crnn) from baidu code:un8d
   pretrained crnn model configuration:

--output_channel 512 \
--hidden_size 256 \
--Transformation None \
--FeatureExtraction ResNet \
--SequenceModeling BiLSTM \
--Prediction CTC 

run demo with pretrained mdoel.

# change config.py file and run:
CUDA_VISIBLE_DEVICES=0 python3 infer.py ${image_path}

2. CTC-Attention pretrained model will release later.

prediction results

demo images	None-ResNet512-BiLSTM256-CTC	None-ResNet768-BiLSTM384-CTC
	同达电动车配件	同达电动车配件
	微信14987227	微信14987227
	快乐大本营20190629期:张艺兴李荣浩惊喜同台合唱彭昱畅破音三连引	快乐大本营20190629期:张艺兴李荣浩惊喜同台合唱彭昱畅破音三连引
	每周三中午12:00	每周三中午12-00
	整套征兵甄别程序的一个部分	整套征兵甄别程序的一个部分
	再热烈的鼓掌	再热烈的鼓掌
	厂外恒升拆车件	广州恒升拆车件
	我想说你为什么	我想说你为什么
	我抱吧他在你怀里一直在哭	我抱吧他在你怀里一直在哭
	如果没有这个阿姨的话	如果没有这个阿姨的话
	因为我觉得有你了我才有安全感	因为我觉得有你了我才有安全感
	丫OUI<U	YOUKU
	麦油娜孕其智商下线	轰迪娜马期智商下线
	因为后一仁看《《二》的巡演	团为册长加看飞广海》的巡演
	铂爵	铂爵
	婆婆担心张晋怒斥蔡少芬	婆婆担心张晋怒斥蔡少芬
	若风作为队长带队获得/PL5总冠军	若风作为队长带队获得/PL5总冠军
	向佐向郭碧婷求婚戊功	向佐向郭碧婷求婚成功
	严屹宽骨亲	严屹宽母宗
	哇品会	唯品会
	YOUI<U独播	YOUKU独播

Training and evaluation

1. Train CRNN model

CUDA_VISIBLE_DEVICES=0 python train.py \
	--train_data data/synch/lmdb_train \
	--valid_data data/synch/lmdb_val \
	--select_data / --batch_ratio 1 \
	--sensitive \
  	--num_iter 400000 \
  	--output_channel 512 \
  	--hidden_size 256 \
	--Transformation None \
  	--FeatureExtraction ResNet \
  	--SequenceModeling BiLSTM \
  	--Prediction CTC \
  	--experiment_name none_resnet_bilstm_ctc \
  	--continue_model saved_models/pretrained_model.pth

2. Train CTC-Attention model

CUDA_VISIBLE_DEVICES=0 python train.py \
	--train_data data/synch/lmdb_train \
	--valid_data data/synch/lmdb_val \
	--select_data / --batch_ratio 1 \
  	--sensitive \
  	--num_iter 400000 \
  	--output_channel 512 \
	--hidden_size 256 \
	--Transformation None \
  	--FeatureExtraction ResNet \
  	--SequenceModeling BiLSTM \
  	--Prediction CTC \
  	--mtl \
  	--without_prediction \
  	--experiment_name none_resnet_bilstm_ctc \
  	--continue_model saved_models/pretrained_model.pth

Acknowledgements

1. This implementation has mainly been based on this great repository: deep-text-recognition-benchmark
2. SynthText Generation has mainly been based on TextRecognitionDataGenerator