StorytellingSerious

International Conference on Emerging Techniques in Computational Intelligence (ICETCI) ·

Multimodal Image Retrieval Through Image-Text Query Fusion Using CLIP

Kushagra Pathak, Alok Kumar Kamal, Appana Saketh Krishna Rao, BHVSP Subrahmanyam, Neeraj Choudhary

Published · Conference · Multimodal retrieval

A reference photo and a written change guide a CLIP model to the target fashion image.

Abstract

Multimodal image retrieval involves retrieving a target image with a reference image and a text modification. This task requires proper fusion of image and textual cues for successful performance. In this paper, we propose a two-staged CLIP-based pipeline where we first fine-tune the image/text encoders according to our retrieval goals. Then, we design a composition network to fuse the image and text representations and perform target image retrieval. Our proposed model is evaluated on the FashionIQ dataset via Recall@K evaluation metric. We obtain average Recall@10 = 30.12 and Recall@50 = 55.06 for our composed model. These values outperform baseline methods such as summation (Recall@10 = 21.17 / Recall@50 = 45.62) and weighted summation (Recall@10 = 27.12 / Recall@50 = 51.94). Moreover, we achieve improvements in mean recall by replacing the ResNet50 backbone with RN50x4 (31.64 → 33.36). These results show that a lightweight CLIP-based composine strategy can enhance retrieval performance while maintaining simplicity.

Findings

  • Reports average Recall@10 of 30.12 and Recall@50 of 55.06 on FashionIQ.
  • Outperforms simple and weighted-summation fusion baselines in the reported evaluation.

Sources and citation

DOI 10.1109/ICETCI71772.2026.11710514

View BibTeX citation
@inproceedings{clip-retrieval-2026,
  title = {{Multimodal Image Retrieval Through Image-Text Query Fusion Using CLIP}},
  author = {Kushagra Pathak and Alok Kumar Kamal and Appana Saketh Krishna Rao and BHVSP Subrahmanyam and Neeraj Choudhary},
  booktitle = {International Conference on Emerging Techniques in Computational Intelligence (ICETCI)},
  year = {2026},
  pages = {1--6},
  publisher = {IEEE},
  doi = {10.1109/ICETCI71772.2026.11710514},
  url = {https://ieeexplore.ieee.org/abstract/document/11710514/}
}

What are you in the mood for?