Profile-level positivity assessment using multimodal feature aggregation from Social Media Content

Miniatura
Archivos
gorraiz_profile_2026.pdf (1.72 MB)
Fecha
2026-02-27
Autores
Título de la revista
ISSN de la revista
Título del volumen
Editor
Institute of Electrical and Electronics Engineers Inc.
google-scholar
Resumen
Accurately evaluating positivity in social media profiles has important implications for advertising, mental health monitoring, and responsible AI applications. However, the multimodal nature of Instagram, where images, captions, hashtags, and emojis co-occur, challenges conventional unimodal sentiment analysis tools. We present a multimodal system that integrates computer vision and natural language processing to assess Instagram profiles. The architecture comprises two blocks: (i) feature extraction via facial-emotion analysis, object detection, chromatic cues, and linguistic processing; and (ii) profile-level inference, which fuses aggregated features to produce an overall positivity score together with semantically meaningful profile tags. Experiments on public Instagram profiles (images, captions, and comments), including an expert-annotated subset of comments, compare unimodal baselines with transformer-based embeddings and OpenAI large language models. The multimodal pipeline improves over unimodal variants, and the OpenAI approach achieves higher concordance with expert judgements than RoBERTa/Bi-LSTM baselines. Beyond technical contributions, we discuss privacy, transparency, and misuse risks associated with affective computing on social media. The proposed framework contributes to the advancement of multimodal sentiment analysis in social media and highlights its applicability to areas such as marketing analytics, well-being monitoring, and ethically aligned AI systems.
Palabras clave
Computer vision
Deep learning
Ethical AI
Instagram
Multimodal sentiment analysis
Natural language processing
Social media analytics
Topic modeling
Descripción
Materias
Cita
Gorraiz-Bengoechea, M., Gonzalez-Santocildes, A., Pastor, I., Estévez, A., & Aonso-Diego, G. (2026). Profile-level positivity assessment using multimodal feature aggregation from Social Media Content. IEEE Access, 14, 37373-37383. https://doi.org/10.1109/ACCESS.2026.3668915
URI
https://hdl.handle.net/20.500.14454/5628
Colecciones
Artículos