Please use this identifier to cite or link to this item:
Volltext verfügbar? / Dokumentlieferung
doi:10.22028/D291-29404
Title: | Multimodal Dialogue System Evaluation : A Case Study Applying Usability Standards |
Author(s): | Malchanau, Andrei Petukhova, Volha Bunt, Harry |
Editor(s): | D'Haro, Luis Fernando Banchs, Rafael E. Li, Haizhou |
Language: | English |
Title: | Proceedings Ninth International Workshop on Spoken Dialogue Systems Technology |
Startpage: | 145 |
Endpage: | 159 |
Publisher/Platform: | Springer |
Year of Publication: | 2019 |
Place of publication: | Singapore |
Title of the Conference: | IWSDS 2019 |
Place of the conference: | Siracusa, Italy |
Publikation type: | Conference Paper |
Abstract: | This paper presents an approach to the evaluation of multimodal dialogue systems, applying usability metrics defined in ISO standards. Users’ perceptions of effectiveness, efficiency and satisfaction were correlated with various performance metrics derived from system logfiles and reference annotations. Usability experts rated questions from a preliminary 110-items questionnaire, and an assessment of their agreement on usability concepts has led to a selection of eight main factors: task completion and quality, robustness, learnability, flexibility, likeability, ease of use and usefulness (value) of an application. Based on these factors, an internally consistent and reliable questionnaire with 32 items (Cronbach’s alpha of 0.87) was produced. This questionnaire was used to evaluate the Virtual Negotiation Coaching system for metacognitive skills training in a multi-issue bargaining setting. The observed correlations between usability perception and derived performance metrics suggest that the overall system usability is determined by the quality of agreements reached, by the robustness and flexibility of the interaction, and by the quality of system responses. |
DOI of the first publication: | 10.1007/978-981-13-9443-0_13 |
URL of the first publication: | https://link.springer.com/chapter/10.1007/978-981-13-9443-0_13 |
Link to this record: | hdl:20.500.11880/28382 http://dx.doi.org/10.22028/D291-29404 |
ISBN: | 978-981-13-9442-3 978-981-13-9443-0 |
Date of registration: | 27-Nov-2019 |
Faculty: | P - Philosophische Fakultät |
Department: | P - Sprachwissenschaft und Sprachtechnologie |
Professorship: | P - Prof. Dr. Dietrich Klakow |
Collections: | SciDok - Der Wissenschaftsserver der Universität des Saarlandes |
Files for this record:
There are no files associated with this item.
Items in SciDok are protected by copyright, with all rights reserved, unless otherwise indicated.