Experiments in Unsupervised Language Learning

Abstract

There are a number of debates in linguistic, psycholinguistic and neurolinguistic circles which have relevance to research on machine learning of natural language. Some of these concern where language lies on the spectrum between innate and learnt; how much can be learnt in the absence of semantics; how much can be achieved by neural self-organization without multi-layer back-propogation; and how important negative information is to language learning. The computational research presented in this paper places a point of reference on each of these spectra, and indeed suggests that they are not independent. We present some computational experiments and results, and propose ideas towards a theory of language learning. More importantly we pose some traditional questions in a new light and suggest new avenues of research for the traditional cognitive science disciplines as well as modern computational linguistics. The concrete experiments presented use statistical techniques for lexical learning and were inspired by earlier experiments using statistical and neural techniques for syntactic learning and speech recognition. The interrelationships and significance of all of these experiments are discussed.