Bitte benutzen Sie diese Referenz, um auf diese Ressource zu verweisen: doi:10.22028/D291-42325
Volltext verfügbar? / Dokumentlieferung
Titel: Entity Tracking in Language Models
VerfasserIn: Kim, Najoung
Schuster, Sebastian
HerausgeberIn: Rogers, Anna
Sprache: Englisch
Titel: The 61st Conference of the the Association for Computational Linguistics : July 9-14, 2023 : ACL 2023 : Volume 1: Long papers
Seiten: 3835-3855
Verlag/Plattform: ACL
Erscheinungsjahr: 2023
Erscheinungsort: Stroudsburg, PA
Konferenzort: Toronto, Canada
DDC-Sachgruppe: 004 Informatik
400 Sprache, Linguistik
Dokumenttyp: Konferenzbeitrag (in einem Konferenzband / InProceedings erschienener Beitrag)
Abstract: Keeping track of how states of entities change as a text or dialog unfolds is a key prerequisite to discourse understanding. Yet, there have been few systematic investigations into the ability of large language models (LLMs) to track discourse entities. In this work, we present a task probing to what extent a language model can infer the final state of an entity given an English description of the initial state and a series of state-changing operations. We use this task to first investigate whether Flan-T5, GPT-3 and GPT-3.5 can track the state of entities, and find that only GPT-3.5 models, which have been pretrained on large amounts of code, exhibit this ability. We then investigate whether smaller models pretrained primarily on text can learn to track entities, through finetuning T5 on several training/evaluation splits. While performance degrades for more complex splits, we find that even when evaluated on a different set of entities from training or longer operation sequences, a finetuned model can perform non-trivial entity tracking. Taken together, these results suggest that language models can learn to track entities but pretraining on text corpora alone does not make this capacity surface.
Link zu diesem Datensatz: urn:nbn:de:bsz:291--ds-423252
hdl:20.500.11880/37991
http://dx.doi.org/10.22028/D291-42325
ISBN: 978-1-959429-72-2
Datum des Eintrags: 3-Jul-2024
Fakultät: MI - Fakultät für Mathematik und Informatik
Fachrichtung: MI - Informatik
Professur: MI - Prof. Dr. Vera Demberg
Sammlung:SciDok - Der Wissenschaftsserver der Universität des Saarlandes

Dateien zu diesem Datensatz:
Es gibt keine Dateien zu dieser Ressource.


Alle Ressourcen in diesem Repository sind urheberrechtlich geschützt.