04.03.2024New paper: Illustrating the need of more diverse populations for cognitively-inspired NLP

In our new paper accepted to LREC 2024, we make a case for using more diverse populations in cognitively-inspired NLP research.

Reading Does Not Equal Reading: Comparing, Simulating and Exploiting Reading Behavior Across Populations, accepted to LREC 2024
David R. Reich, Shuwen Deng, Marina Björnsdóttir, Lena A. Jäger, Nora Hollenstein

Abstract

Eye-tracking-while-reading corpora play a crucial role in the study of human language processing, and, more recently, have been leveraged for cognitively enhancing neural language models. A critical limitation of existing corpora is that they often lack diversity, comprising primarily native speakers. In this study, we expand the eye-tracking-while-reading dataset CopCo, which initially included only Danish L1 readers with and without dyslexia, by incorporating a new dataset of L2 readers with diverse L1 backgrounds. Thus, the extended CopCo corpus constitutes the first eye-tracking-while-reading dataset encompassing neurotypical L1 and L1 readers with dyslexia as well as L2 readers. We first provide extensive descriptive statistics of the extended CopCo corpus. Second, we investigate how different degrees of diversity of the training data affect a state-of-the-art generative model of eye movements in reading. Finally, we use the scanpath generation models for gaze-augmented language modeling and investigate the impact of diversity in the training data on the model’s performance on a range of NLP downstream tasks.