Home alone: Complex problem solving performance benefits from individual online assessment

Abstract

Computer-based assessments of complex problem solving performance often take place in group settings like classrooms and computer laboratories. Such computer-based procedures provide an excellent opportunity to examine setting effects that might occur while participants are tested in a non-group session online at a time and place of their own choosing. For this purpose, N = 273 teacher students were randomly assigned to one of two settings: the individual online condition (n = 216) or the computer laboratory group condition (n = 57). Strong factorial measurement invariance was evidenced. Participants performed significantly better in the individual online condition than in the group condition (knowledge acquisition: d = 0.38; knowledge application: d = 0.39). The worse performance in the group setting compared to the individual setting could neither be explained by exploration time, nor by time on task. The internal experimental design validity strengthens the conclusion that setting-related differences in cognitive ability testing are not negligible but noteworthy.