Diagnostic evaluation in linguistic word recognition

Abstract

This report is concerned with a new method of evaluation for the Linguistic Word Recognition component of the Verbmobil-Project: Architektur. A two stage model of diagnostic evaluation is presented consisting of logical and empirical evaluation steps. Logical evaluation is carried out according to a data model which acts as optimal input in order that each component participating in the evaluation process can be tested for soundness and completeness. Inconsistencies can thus be remedied before empirical evaluation of the model is undertaken using real data. The diagnostic evaluation method has been operationalised within the Bielefeld Extended Evaluation Toolkit for Lattices of Events (BEETLE).