International Conference on Emerging Techniques in Computational Intelligence (ICETCI) ·
Multimodal Image Retrieval Through Image-Text Query Fusion Using CLIP
Published · Conference · Multimodal retrieval
A reference photo and a written change guide a CLIP model to the target fashion image.
Abstract
Multimodal image retrieval involves retrieving a target image with a reference image and a text modification. This task requires proper fusion of image and textual cues for successful performance. In this paper, we propose a two-staged CLIP-based pipeline where we first fine-tune the image/text encoders according to our retrieval goals. Then, we design a composition network to fuse the image and text representations and perform target image retrieval. Our proposed model is evaluated on the FashionIQ dataset via Recall@K evaluation metric. We obtain average Recall@10 = 30.12 and Recall@50 = 55.06 for our composed model. These values outperform baseline methods such as summation (Recall@10 = 21.17 / Recall@50 = 45.62) and weighted summation (Recall@10 = 27.12 / Recall@50 = 51.94). Moreover, we achieve improvements in mean recall by replacing the ResNet50 backbone with RN50x4 (31.64 → 33.36). These results show that a lightweight CLIP-based composine strategy can enhance retrieval performance while maintaining simplicity.
Findings
- Reports average Recall@10 of 30.12 and Recall@50 of 55.06 on FashionIQ.
- Outperforms simple and weighted-summation fusion baselines in the reported evaluation.
Sources and citation
DOI 10.1109/ICETCI71772.2026.11710514
View BibTeX citation
@inproceedings{clip-retrieval-2026,
title = {{Multimodal Image Retrieval Through Image-Text Query Fusion Using CLIP}},
author = {Kushagra Pathak and Alok Kumar Kamal and Appana Saketh Krishna Rao and BHVSP Subrahmanyam and Neeraj Choudhary},
booktitle = {International Conference on Emerging Techniques in Computational Intelligence (ICETCI)},
year = {2026},
pages = {1--6},
publisher = {IEEE},
doi = {10.1109/ICETCI71772.2026.11710514},
url = {https://ieeexplore.ieee.org/abstract/document/11710514/}
}