Convolutional pyramid of bidirectional character sequences for the recognition of handwritten words

Convolutional pyramid of bidirectional character sequences for the recognition of handwritten words

Mhiri, Mohamed and Desrosiers, Christian and Cheriet, Mohamed

Pattern Recognition Letters 2018

Abstract : Handwritten word recognition is a challenging task due to the large intra-class variability of handwritten shapes and the complexity of modeling and segmenting sequences of overlapping characters. This work proposes a novel approach based on deep convolutional neural networks (CNNs), which does not require the explicit segmentation of characters and can learn a suitable representation for handwritten data in an automated way. The proposed approach uses a CNN to learn the mapping from word images to a robust representation, called pyramid of bidirectional character sequences. This novel representation encodes sub-sequences of characters in a hierarchical manner, considering both forward and backward directions. An efficient inference technique is then employed to find the most likely word in a lexicon, based on the CNN output probabilities. By implicitly modeling the distribution of character sub-sequences in the data, our approach can transfer knowledge across words containing the same sub-sequences. The proposed approach achieves a word error rate of 8.83% on the IAM database and 6.22% on the RIEMS database, outperforming recent methods for this task.