Swap retrieval: Retrieving images of cats when the query shows a dog
Swap retrieval: Retrieving images of cats when the query shows a dog
Ghodrati, Amir and Jia, Xu and Pedersoli, Marco and Tuytelaars, Tinne
ICMR 2015 – Proceedings of the 2015 ACM International Conference on Multimedia Retrieval 2015
Abstract : Query-by-example remains popular in image retrieval because it can exploit contextual information encoded in the image, that is difficult to express in a traditional textual query. Textual queries, on the other hand, give more flexibility in that it’s easy to reformulate and refine a text query based on initial results. In this work we make a first step towards getting the best of both worlds: we use an image to specify the context, but let the user specify a related category as main search criterion. For instance, starting from an image of a dog in a certain situation/context, the goal is to find images of cats with a similar situation/context. We present an evaluation scheme for this new and challenging task, which we call swap retrieval, and use it to compare various methods. Results show that standard query-by-example techniques do not adapt well to the new task. Instead, techniques based on semantic knowledge extracted from textual descriptions available at training time perform reasonably well, although they are still far from the performance needed for practical use.
