Multi-modal classification of polyp malignancy using CNN features with balanced class augmentation

Colorectal polyps are an indicator of colorectal cancer (CRC). Classification of polyps during colonoscopy is still a challenge for which many medical experts have come up with visual models albeit with limited success. In this paper, a classification approach is proposed to differentiate between polyp malignancy, using features extracted from the Global Average Pooling (GAP) layer of a Convolutional Neural Network (CNNs). Two recent endoscopic modalities are used to improve the algorithm prediction: Blue Laser Imaging (BLI) and Linked Color Imaging (LCI). Furthermore, a new strategy of per-class data augmentation is adopted to tackle an unbalanced class distribution and to improve the decision of the classifiers. As a result, we increase the performance compared to state-of-the-art methods (0.97 vs 0.90 AUC). Our method for automatic polyp malignancy classification facilitates future advances towards patient safety and may avoid time-consuming and costly histopathological assessment.