-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathindex.html
249 lines (224 loc) · 9.39 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
<!DOCTYPE html>
<html lang="en-us">
<head>
<meta charset="UTF-8">
<title>SemEval 2024 Task: Emotion Discovery and Reasoning its Flip in Conversation (EDiReF)</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="stylesheet" type="text/css" href="stylesheets/normalize.css" media="screen">
<link href='https://fonts.googleapis.com/css?family=Open+Sans:400,700' rel='stylesheet' type='text/css'>
<link rel="stylesheet" type="text/css" href="stylesheets/stylesheet.css" media="screen">
<link rel="stylesheet" type="text/css" href="stylesheets/github-light.css" media="screen">
<script src="https://kit.fontawesome.com/f85e105653.js" crossorigin="anonymous"></script>
<style>
.button {
border: none;
color: white;
padding: 15px 32px;
text-align: center;
text-decoration: none;
display: inline-block;
font-size: 16px;
margin: 4px 2px;
cursor: pointer;
}
.button1 {background-color: #4CAF50;} /* Green */
.button2 {background-color: #008CBA;} /* Blue */
</style>
</head>
<body>
<section class="page-header">
<h1 class="project-name">SemEval 2024 Task 10: Emotion Discovery and Reasoning its Flip in Conversation (EDiReF)</h1>
<h2 class="project-tagline">Task Home Page</h2>
</section>
<section class="main-content">
<h1 style="display: inline; color: red"><a href = "#data"><i class="fa fa-bullhorn"></i> Submit your papers till 19th February!</a></h1><br/>
<h3 style="display: inline; color: red">IMPORTANT: No data from task 2 or task 3 can be used to train/evaluate task 1.</h2><br/>
<h3 style="display: inline">Registration Form: </h3> <a href="https://forms.gle/yrpbJPqxCuXB7sN76" target="_blank">https://forms.gle/yrpbJPqxCuXB7sN76</a>.<br/>
<h3 style="display: inline">Mailing List: </h3> <a href="https://groups.google.com/g/ediref2024_group" target="_blank">https://groups.google.com/g/ediref2024_group</a>.<br/>
<h3 style="display: inline">CodaLab Link: </h3> <a href="https://codalab.lisn.upsaclay.fr/competitions/16769" target="_blank">https://codalab.lisn.upsaclay.fr/competitions/16769</a>.
<p>The <strong>EDiReF</strong> shared task at <a href="https://semeval.github.io/SemEval2024/">SemEval 2024</a> is an amalgamation of three subtasks tasks- (i) Emotion Recognition in Conversation <strong>(ERC)</strong> in Hindi-English code-mixed conversations, (ii) Emotion Flip Reasoning <strong>(EFR)</strong> in Hindi-English code-mixed conversations, and (iii) <strong>EFR</strong> in English conversations. The definition for the tasks of ERC and EFR are as illustrated below:</p>
<ul>
<li><strong>ERC</strong>: Given a dialogue, ERC aims to assign an emotion to each utterance from a predefined set of possible emotions.</li>
<li><strong>EFR</strong>: Given a dialogue, EFR aims to identify the trigger utterance(s) for an emotion-flip in a multi-party conversation dialogue.</li>
</ul>
<!-- <h2>
<a id="task-example" class="anchor" href="#task-example" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Task Example</h2> -->
<p>For example, given a sample dialogue below:</p>
<table>
<thead>
<tr>
<th>Speaker</th>
<th>Utterance</th>
</tr>
</thead>
<tbody>
<tr>
<td>Sp<sub>1</sub></td>
<td>Aaj to bhot awful day tha! <i style="color:blue;">(I had an awful day today!)</i></td>
</tr>
<tr>
<td>Sp<sub>2</sub></td>
<td>Oh no! Kya hua? <i style="color:blue;">(Oh no! What happened?)</i></td>
</tr>
<tr>
<td>Sp<sub>1</sub></td>
<td>Kisi ne mera sandwich kha liya! <i style="color:blue;">(Somebody ate my sandwich!)</i></td>
</tr>
<tr>
<td>Sp<sub>2</sub></td>
<td>Me abhi tumhare liye new bana deti hun! <i style="color:blue;">(I can make you a new one right now!)</i></td>
</tr>
<tr>
<td>Sp<sub>1</sub></td>
<td>Wo great hoga! Thanks! <i style="color:blue;">(That would be great! Thanks!)</i></td>
</tr>
</tbody>
</table>
<p>ERC aims to assign emotions to each utterance</p>
<table>
<thead>
<tr>
<th>Speaker</th>
<th>Utterance</th>
<th>Emotion</th>
</tr>
</thead>
<tbody>
<tr>
<td>Sp<sub>1</sub></td>
<td>Aaj to bhot awful day tha! <i style="color:blue;">(I had an awful day today!)</td>
<td>Sad</td>
</tr>
<tr>
<td>Sp<sub>2</sub></td>
<td>Oh no! Kya hua? <i style="color:blue;">(Oh no! What happened?)</td>
<td>Sad</td>
</tr>
<tr>
<td>Sp<sub>1</sub></td>
<td>Kisi ne mera sandwich kha liya! <i style="color:blue;">(Somebody ate my sandwich!)</td>
<td>Sad</td>
</tr>
<tr>
<td>Sp<sub>2</sub></td>
<td>Me abhi tumhare liye new bana deti hun! <i style="color:blue;">(I can make you a new one right now!)</td>
<td>Joy</td>
</tr>
<tr>
<td>Sp<sub>1</sub></td>
<td>Wo great hoga! Thanks! <i style="color:blue;">(That would be great! Thanks!)</td>
<td>Joy</td>
</tr>
</tbody>
</table>
<p>The example dialogue has two emotion flips. Sp<sub>1</sub>'s emotion changed from Sad to Joy while Sp<sub>2</sub>'s emotion also shifted from Sad to joy. EFR aims to justify such emotion flips using triggers.</p>
<table>
<thead>
<tr>
<th>Speaker</th>
<th>Utterance</th>
<th>Emotion</th>
<th>Trigger</th>
</tr>
</thead>
<tbody>
<tr>
<td>Sp<sub>1</sub></td>
<td>Aaj to bhot awful day tha! <i style="color:blue;">(I had an awful day today!)</td>
<td>Sad</td>
<td>0</td>
</tr>
<tr>
<td>Sp<sub>2</sub></td>
<td>Oh no! Kya hua? <i style="color:blue;">(Oh no! What happened?)</td>
<td>Sad</td>
<td>0</td>
</tr>
<tr>
<td>Sp<sub>1</sub></td>
<td>Kisi ne mera sandwich kha liya! <i style="color:blue;">(Somebody ate my sandwich!)</td>
<td>Sad</td>
<td>0</td>
</tr>
<tr>
<td>Sp<sub>2</sub></td>
<td>Me abhi tumhare liye new bana deti hun! <i style="color:blue;">(I can make you a new one right now!)</td>
<td>Joy</td>
<td>1</td>
</tr>
<tr>
<td>Sp<sub>1</sub></td>
<td>Wo great hoga! Thanks! <i style="color:blue;">(That would be great! Thanks!)</td>
<td>Joy</td>
<td>0</td>
</tr>
</tbody>
</table>
<h2>
<a id="important-dates" class="anchor" href="#important-dates" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Important Dates</h2>
<table>
<tbody>
<tr>
<td><s>Tasks announced (with sample data available)</s></td>
<td><s>17 July 2023</s></td>
</tr>
<tr>
<td><s>Training data ready</s></td>
<td><s>4 September 2023</s></td>
</tr>
<tr>
<td><s>Evaluation start</s></td>
<td><s>10 January 2024</s></td>
</tr>
<tr>
<td><s>Evaluation end</s></td>
<td><s>3 February 2024</s></td>
</tr>
<tr>
<td>Paper submission due</td>
<td>19 February 2024</td>
</tr>
<tr>
<td>Notification to authors</td>
<td>1 April 2024</td>
</tr>
<tr>
<td>Camera ready due</td>
<td>22 April 2024</td>
</tr>
<tr>
<td>SemEval workshop</td>
<td>TBD, 2024 (co-located with a Major NLP conference)</td>
</tr>
</tbody>
</table>
<h2>
<a id="data" class="anchor" href="#data" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Data</h2>
<a href="https://drive.google.com/drive/folders/16mFdhXGECdcyDjLs_alI43W4iNQK_47B?usp=sharing" target="_blank"><button type="button" class="button button2"><i class='fas fa-folder'></i> Training set</button></a>
<a href="https://drive.google.com/drive/folders/1kcSmz-6GVdzAdfcMNwHAWPKBI-1xVOFm?usp=sharing" target="_blank"><button type="button" class="button button2"><i class='fas fa-folder'></i> Validation set</button></a>
<a href="https://drive.google.com/drive/folders/16S3HTJ6_jq7of6JD2HwtBr4aCwuaoV1g?usp=sharing" target="_blank"><button type="button" class="button button2"><i class='fas fa-folder'></i> Testing set</button></a><br/>
Kindly register to the <a href="https://codalab.lisn.upsaclay.fr/competitions/16769" target="_blank">Codalab competition</a> to get started with model submission
<h2>
<a id="codalab" class="anchor" href="#codalab" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>CodaLab</h2>
<p>CodaLab link for the shared task: <a href="https://codalab.lisn.upsaclay.fr/competitions/16769" target="_blank">https://codalab.lisn.upsaclay.fr/competitions/16769</a>. Interested participants are encouraged to <a href="https://forms.gle/yrpbJPqxCuXB7sN76" target="_blank">register now</a> and join the <a href="https://groups.google.com/g/ediref2024_group" target="_blank">mailing list</a> to stay updated.</p>
<h2>
<a id="organization" class="anchor" href="#organization" aria-hidden="true"><span aria-hidden="true" class="octicon octicon-link"></span></a>Organization</h2>
<!-- <p>Please subscribe to <a href="https://groups.google.com/group/dimsum16">https://groups.google.com/group/dimsum16</a> for announcements about the task.</p>
<p>See the <a href="http://alt.qcri.org/semeval2016/task10/index.php?id=important-dates">schedule</a> for planned data releases and deadlines.</p> -->
<p>The organizers are:</p>
<ul>
<li>
<a href="https://sites.google.com/iiitd.ac.in/shivani-kumar/home">Shivani Kumar</a>, IIIT, Delhi</li>
<li>
<a href="https://faculty.iiitd.ac.in/~shad.akhtar//">Md Shad Akhtar</a>, IIIT, Delhi</li>
<li>
<a href="https://www.tanmoychak.com/">Tanmoy Chakraborty</a>, IIT, Delhi</li>
<li>
<a href="https://dr.ntu.edu.sg/cris/rp/rp00927">Erik Cambria</a>, NTU, Singapore</li>
</ul>
<footer class="site-footer">
<span class="site-footer-credits">This page was generated by <a href="https://pages.github.com">GitHub Pages</a> using the <a href="https://github.com/jasonlong/cayman-theme">Cayman theme</a> by <a href="https://twitter.com/jasonlong">Jason Long</a>.</span>
</footer>
</section>
</body>
</html>