Mind the Context! Questionnaire Design Can Affect the Attribution of Gender to Robots

The field of Human-Robot Interaction (HRI) has seen growing interest in the topic of gendering robots. However, as this topic has gained momentum rapidly, the research methodologies used to study it have not yet undergone critical analysis and refinement. This study investigates whether multi-dimensional questionnaires are susceptible to context effects and how respondents' views on gender may amplify these effects. We conducted an online study using LimeSurvey, employing a mixed-model design with questionnaire design as a between-subjects variable (four conditions: three items together, three items with distractors, three items separately, and four items) and robot gender ambiguity as a within-subjects variable (gender ambiguous vs. gender non-ambiguous robots). A total of 160 participants were recruited via Prolific, with 40 assigned to each condition. Participants rated the perceived gender of 18 robots (nine gender ambiguous and nine gender non-ambiguous) using the different questionnaire designs. Results show that questionnaire design can alter the direction and strength of the relationships between masculinity, femininity, and gender neutrality. Moreover, they reveal that the questionnaire used to measure a robot's perceived gender does not influence the ratings of gender at an individual level but it does so at a group level. Finally, they disclose that benevolent sexism sensitizes participants to the attribution of masculinity, whereas non-binary gender makes participants more prone to attribute gender neutrality.