An efficient explainable attention based image captioning system

Abstract

Recent advances in deep learning have brought significant attention to the integration of vision computing and natural language processing. Captioning is a method that enables machines to understand an image and provide a natural language explanation for it. The meaningful captions generated by the image have the ability of analyzing the state, the attributes and the relationship among these objects rather merely identifying the objects in the image. newlineCurrently, the Encoder-Decoder architecture is the most effective way to implement Image Captioning. In order to effectively predict objects, sceneries, and patterns within an image and provide captions, this research work presents an innovative Automatic Image Captioning System. newlineThe encoder and decoder architecture proposed in this research makes use of a novel Augmentation and Ranking (A-R model) mechanism. A rich featured image dataset is produced by augmentation, and a ranking system aids in choosing the top k priority terms. The Ranking LSTM assists in identifying the meaningful captions through ranks. The Image Captioning system performs more effectively due to this blending technique. Greedy and beam search are used to investigate the proposed A-R model under maximum and average pooling. newline