A Deep Learning Framework for Detection and Segmentation of Multiple Artefacts in Endoscopic Images

Abstract

Endoscopy is a standard procedure for disease surveillance, monitoring inflammations, detect cancer and tumor. During the procedure the organs are visualized. Artefacts, an artificial effect is found to be present in the resultant images. They play a dominant role in increasing procedure time by more than an hour. Hence an efficient algorithm to detect, segment and restore could assist clinician. The artefacts present in an endoscopic image include saturation, specular reflections, blur, bubbles, contrast, blood, instruments and miscellaneous artefacts. The presence of these artefacts acts as a barrier when investigating the underlying tissue for identifying clinical abnormalities. It also affect post processing steps where most of the images captured are discarded due to the presence of artefacts which in turn affects information storage and extracting useful frame for report generation. Endoscopic artefact detection dataset is the only available public dataset holding endoscopic images with annotations for multiple artefacts. Hence, a custom dataset is annotated using the same annotation protocol of endoscopic artefact detection dataset to maintain homogeneity. The algorithms are trained and tested with images from both public and custom dataset for artefact detection. State of the art object detection algorithms such as YOLOv3, YOLOv4 and faster R-CNN are used for detecting artefacts in endoscopic images. The detection algorithm focusses on three important performance parameters namely mean average precision, intersection over union and inference time. The ensemble model outperformed well across all the performance parameters compared with literature. The inference time is reduced by 8.63%, whereas the mAP and IoU are increased by 61.67% and 63.47% respectively. newlineSegmentation algorithms like U-Net with EfficientNetB3 backbone, Link-Net with EfficientNetB3 backbone and U-Net with SE-ResNeXt101 backbone are used to segment the artefacts. The results are assessed with performance parameters like F2 score and Jacca