Generalizing Cross domain Retrieval Algorithms

Abstract

Cross-domain retrieval is an important research topic due to its wide range of applications in e-commerce, forensics etc. It addresses the data retrieval problem from a search set, when the query belongs to one domain, and the search database contains samples from some other domain. Several algorithms have been proposed for the same in recent literature to address this task. In this thesis, we address some of the challenges in cross-domain retrieval, specifically for the application of sketch-based image retrieval. Traditionally, cross-domain algorithms assume that both the training and test data belong to the same set of seen-classes, which is quite restrictive. Thus, such models can only be used to retrieve data from the two specific domains on which they have been trained on, and cannot generalize to new domains or new classes, during retrieval. But in real world, new object classes are continuously being discovered over time, thus it is necessary to design algorithms that can generalize to previously unseen classes. In addition, for a practically useful retrieval model, it will be good if the model can perform retrieval between any two different data domains, whether or not those domains are used for training. In our work, we observe a significant decrease in the performance of existing approaches in these generalized retrieval scenarios, when such simplified assumptions are removed. In this thesis, we aim to address these and related challenges, so as to make the cross-domain retrieval models better suited for real-life applications. We first consider a class-wise generalized protocol, where the query data during retrieval may belong to any unseen classes. Following the nomenclature in the classification problems, we refer to this as zero-shot cross-modal retrieval and propose an add-on ranking module to improve the performance of the existing cross-modal methods in literature. This work is applicable to different modalities (eg. text-image), in addition to different domains (eg. image and RGBD data)...