Please use this identifier to cite or link to this item: doi:10.22028/D291-30987
Volltext verfügbar? / Dokumentlieferung
Title: Crowdsourcing discourse interpretations: On the influence of context and the reliability of a connective insertion task
Author(s): Scholman, Merel Cleo Johanna
Demberg, Vera
Editor(s): Schneider, Nathan
Xue, Nianwen
Language: English
Title: 11th Linguistic Annotation Workshop - proceedings of the workshop
Startpage: 24
Endpage: 33
Publisher/Platform: ACL
Year of Publication: 2017
Place of publication: Stroudsburg, PA
Title of the Conference: LAW XI 2017
Place of the conference: Valencia, Spain
Publikation type: Conference Paper
Abstract: Traditional discourse annotation tasks are considered costly and time-consuming, and the reliability and validity of these tasks is in question. In this paper, we investigate whether crowdsourcing can be used to obtain reliable discourse relation annotations. We also examine the influence of context on the reliability of the data. The results of a crowdsourced connective insertion task showed that the method can be used to obtain reliable annotations: The majority of the inserted connectives converged with the original label. Further, the method is sensitive to the fact that multiple senses can often be inferred for a single relation. Regarding the presence of context, the results show no significant difference in distributions of insertions between conditions overall. However, a by-item comparison revealed several characteristics of segments that determine whether the presence of context makes a difference in annotations. The findings discussed in this paper can be taken as evidence that crowdsourcing can be used as a valuable method to obtain insights into the sense(s) of relations.
DOI of the first publication: 10.18653/v1/W17-0803
URL of the first publication: https://www.aclweb.org/anthology/W17-0803/
Link to this record: hdl:20.500.11880/29698
http://dx.doi.org/10.22028/D291-30987
ISBN: 978-1-945626-39-5
Date of registration: 22-Sep-2020
Faculty: MI - Fakultät für Mathematik und Informatik
Department: MI - Informatik
Professorship: MI - Prof. Dr. Vera Demberg
Collections:Die Universitätsbibliographie

Files for this record:
There are no files associated with this item.


Items in SciDok are protected by copyright, with all rights reserved, unless otherwise indicated.