A new experiment on the use of images to answer web survey question
Date:
Description: Images may provide richer and more objective information than text-based answers to open-ended survey questions. Thus, recent research started to explore the use of images to answer web survey questions. However, very little is known yet about the use of images to answer web survey questions and its impact on break-off, item nonresponse, completion time, and question evaluation. This study addresses three research questions: What is the effect of answering web survey questions with images instead of text on these four aspects (RQ1)? What is the effect of including a motivational message on these four aspects (RQ2)? How PCs and smartphones differ on these four aspects (RQ3)? We conducted a web survey experiment (N = 3,169) in Germany using an opt-in access online panel. Our target population was the general German population aged between 18 and 70 years living in Germany. Half of the sample was required to answer with smartphones and the other half with PCs. Within each device group, respondents were randomly assigned to 1) a control group answering open-ended questions with text, 2) a first treatment group answering open-ended questions with images, and 3) a second treatment group answering with images but prompted with a motivational message. PC respondents were only asked two questions requiring to upload already saved images (with text for control group), whereas smartphone respondents were asked two additional questions requiring to take an image in-the-moment (idem). Overall, results show higher break-off and item nonresponse rates, as well as lower question evaluation for participants answering with images. Motivational messages slightly reduce item nonresponse. Finally, participants completing the survey with a PC present lower break-off rates but higher item nonresponse. To our knowledge, this is the first study that experimentally investigates the impact on break-off, item nonresponse, completion time, and question evaluation of asking respondents to answer open-ended questions with images instead of text. We also go one step further by exploring 1) how motivational messages may improve respondent’s engagement with the survey and 2) the effect of the device used to answer on these four aspects.