POS tag perplexity as a measure of syntactic complexity

Abstract

Comparing languages of the world with respect to their complexity is a long-standing open question in linguistics. We here focus on syntactic complexity, aconcept that has been particularly hard to address due to the lack of readily avail-able syntactically annotated corpora and the intricacies of syntactic theories. Wepropose to use a simple information-theoretic measure, perplexity, on the POS tagsequence of texts. Perplexity captures how predictable POS tags are on averagegiven their recent co-texts. Calculating perplexity based on POS tag sequenceshelps us to abstract away from morphological or lexical features of the language,in order to get at the predictability of word order. In this paper, we compare POStag perplexity to other recently proposed measures of syntactic complexity, andevaluate measures by correlating them with expert-proposed scores of syntacticflexibility (Bakker 1998).