Over the past few decades, primarily developed countries witnessed an increased incidence of esophageal adenocarcinoma (EAC). Screening and surveillance of Barrett's esophagus (BE), which is known to augment the probability of developing EAC, can significantly improve survival rates. This is because early-stage dysplasia in BE can be treated effectively, while each subsequent stage complicates successful treatment and seriously reduces survival rates. This study proposes a convolutional neural network-based algorithm, which classifies images of BE visualized with White Light Endoscopy (WLE) as either dysplastic or non-dysplastic. To this end, we use merely pixels surrounding the dysplastic region, while excluding the pixels covering the dysplastic region itself. The phenomenon where the diagnosis of a patient can be determined from tissue other than the clearly observable diseased area, is termed the field effect. With its potential to identify missed lesions, it may prove to be a helpful innovation in the screening and surveillance process of BE. A statistical significant difference test indicates the presence of the field effect in WLE, when comparing the distribution of the algorithm classifications of unseen data and the distribution obtained by a random classification.