Integrated text detection and recognition in natural images

Text detection and recognition in natural images have conventionally been seen in the prior art as autonomous tasks executed in a strictly sequential processing chain with limited information sharing between sub-systems. This approach is flawed because it introduces (1) redundancy in extracting the same text properties multiple times and (2) error by prohibiting verification of hard (often binarized) detection results at later stages. We explore the possibilities for integration of detection and recognition modules by a feedforward multidimensional information stream. Integration involves suitable characterization of the text string at detection and application of the knowledge to ease recognition by a given OCR system. The choice of characterization properties generally depends on the OCR system, although some of them have proven universally applicable. We show that the proposed integration measures enable more robust recognition of text in complex, unconstrained natural environments. Specifically, integration by the proposed measures (1) eliminates textual input irregularities that recognition engines cannot handle and (2) adaptively tunes the recognition stage for each input image. The former function boosts correct detections, while the latter mainly reduces the number of false positives. Our validation experiments on a set of low-quality natural images show that adaptively tuning the OCR stage to the typical text-to-background transitions in the input image (gradient significance profiling) allows to attain an improvement of 29% in the precision-recall performance, mostly through boosting precision.