| title | Transformers to Learn Hierarchical Contexts in Multiparty Dialogue for Span-based Question Answering | ||
|---|---|---|---|
| authors |
|
||
| venue | Annual Conference of the Association for Computational Linguistics (ACL) | ||
| year | 2020 | ||
| published | 2020-07-06 | ||
| publicationType | conference | ||
| venueUrl | https://acl2020.org/ | ||
| paperUrl | https://aclanthology.org/2020.acl-main.505/ | ||
| abstract | We introduce a novel approach to transformers that learns hierarchical representations in multiparty dialogue. First, three language modeling tasks are used to pre-train the transformers, tokenand utterance-level language modeling and utterance order prediction, that learn both token and utterance embeddings for better understanding in dialogue contexts. Then, multi-task learning between the utterance prediction and the token span prediction is applied to fine-tune for span-based question answering (QA). Our approach is evaluated on the FriendsQA dataset and shows improvements of 3.8% and 1.4% over the two state-of-the-art transformer models, BERT and RoBERTa, respectively. |