Digital survey tools have all but replaced paper and pen in the psychological sciences, and consequently new forms of potentially useful research paradata are now routinely gathered. A particularly common byproduct of research is questionnaire timestamps, which some have suggested can be used as a measure of cognitive function. Here, we conducted a comprehensive validation of this measure, which we call the "digital questionnaire response time," or "DQRT." Using data from N = 2,977 users of a smartphone app, we first ran a data-driven bootstrapping approach to examine how best to quantify DQRT. DQRT was slower in older adults (r = 0.26) and in those with lower educational attainment and socioeconomic status. Testing the association between DQRT and working memory (range r = 0.11-0.14), model-based planning (range r = 0.03-0.06), and processing speed (range r = 0.29-0.39) across cross-sectional and longitudinal subsamples, we found support for a cognitive characterization of DQRT as a measure of cognitive processing speed. DQRT was more strongly correlated with nine out of 13 lifestyle and health factors, and four out of nine mental health factors than a task-based measure of processing speed. DQRT showed good test-retest reliability, and associations between DQRT and task-based processing speed were higher within individuals (r = 0.35) than between individuals (r = 0.25). Finally, we highlight substantial, but addressable, potential confounds inherent in the measure. We conclude that DQRT has important limitations, but overall can serve as a valid and reliable index of cognitive processing speed that can be gathered at unprecedented scale, unobtrusively, and repeatedly, during a variety of real-world digital behaviors.
Keywords: Ambulatory assessment; Cognition; Ecological momentary assessment; Paradata; Processing speed; Smartphone-based assessment; Survey response times.
© 2025. The Author(s).