AAAI Publications, Twenty-Ninth AAAI Conference on Artificial Intelligence

Font Size: 
VELDA: Relating an Image Tweet’s Text and Images
Tao Chen, Hany M. SalahEldeen, Xiangnan He, Min-Yen Kan, Dongyuan Lu

Last modified: 2015-02-09


Image tweets are becoming a prevalent form of socialmedia, but little is known about their content — textualand visual — and the relationship between the two mediums.Our analysis of image tweets shows that while visualelements certainly play a large role in image-text relationships, other factors such as emotional elements, also factor into the relationship. We develop Visual-Emotional LDA (VELDA), a novel topic model to capturethe image-text correlation from multiple perspectives (namely, visual and emotional). Experiments on real-world image tweets in both Englishand Chinese and other user generated content, show that VELDA significantly outperforms existingmethods on cross-modality image retrieval. Even in other domains where emotion does not factor in imagechoice directly, our VELDA model demonstrates good generalization ability, achieving higher fidelity modeling of such multimedia documents.


image tweets; microblog; image and text; topic model

Full Text: PDF