Attention-based Cross-Modal Unification of Visualized Text and Image Features: Understanding the influence of interface and user idiosyncrasies on unification for free-viewing

Sandeep Vidyapu, Vijaya Saradhi Vedula, Michael Burch, Samit Bhattacharya

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

The attentional analysis on graphical user interfaces (GUIs) is shifting from Areas-of-Interest (AOIs) to Data-of-Interest (DOI). However, the heterogeneity data modalities on GUIs hinder the DOI-based analyses. To overcome this limitation, we present a Canonical Correlation Analysis (CCA) based approach to unify the heterogeneous modalities (text and images) concerning user attention. Especially, the influence of interface and user idiosyncrasies in establishing the cross-modal correlation is studied. The performance of the proposed approach is analyzed for free-viewing eye-tracking experiments conducted on bi-modal webpages. The results reveal: (i) Cross-modal text and image visual features are correlated when the interface idiosyncrasies, alone or along with user idiosyncrasies, are constrained. (ii) The font-families of text are comparable to color histogram visual features of images in drawing the users’ attention. (iii) Text and image visual features can delineate the attention of each other. Our approach finds applications in user-oriented webpage rendering and computational attention modeling.
Original languageEnglish
Title of host publicationETRA '20 Adjunct: ACM Symposium on Eye Tracking Research and Applications
EditorsStephen N. Spencer
Place of PublicationNew York
PublisherAssociation for Computing Machinery, Inc
Pages29:1-29:9
ISBN (Print)978-1-4503-7135-3
DOIs
Publication statusPublished - 2 Jun 2020
EventETRA 2020 ACM Symposium on Eye Tracking Research and Applications - Stuttgart, Germany
Duration: 2 Jun 20205 Jun 2020
https://etra.acm.org/2020/#:~:text=About%20ETRA%202020%3A,under%20the%20motto%20Bridging%20Communities.

Conference

ConferenceETRA 2020 ACM Symposium on Eye Tracking Research and Applications
Abbreviated titleETRA 2020
CountryGermany
CityStuttgart
Period2/06/205/06/20
Internet address

Keywords

  • Canonical correlation analysis (CCA)
  • Data-of-interest (DOI)
  • Eye-tracking
  • Visual attention
  • Webpages

Fingerprint Dive into the research topics of 'Attention-based Cross-Modal Unification of Visualized Text and Image Features: Understanding the influence of interface and user idiosyncrasies on unification for free-viewing'. Together they form a unique fingerprint.

Cite this