-
Notifications
You must be signed in to change notification settings - Fork 0
/
index.html
273 lines (245 loc) · 12.2 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">
<meta name="description" content="">
<meta name="author" content="">
<title>CFEVER Homepage</title>
<!-- Bootstrap core CSS -->
<link href="vendor/bootstrap/css/bootstrap.min.css" rel="stylesheet">
<!-- Custom fonts for this template -->
<link href="vendor/font-awesome/css/font-awesome.min.css" rel="stylesheet" type="text/css">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.3.1/css/all.css"
integrity="sha384-mzrmE5qonljUremFsqc01SB46JvROS7bZs3IO2EmfFsd15uHvIt+Y8vEf7N7fWAU" crossorigin="anonymous">
<link href='https://fonts.googleapis.com/css?family=Lora:400,700,400italic,700italic' rel='stylesheet'
type='text/css'>
<link
href='https://fonts.googleapis.com/css?family=Open+Sans:300italic,400italic,600italic,700italic,800italic,400,300,600,700,800'
rel='stylesheet' type='text/css'>
<!-- Custom styles for this template -->
<link href="css/clean-blog.min.css" rel="stylesheet">
<link href="css/CFEVER.css" rel="stylesheet">
</head>
<body>
<nav class="navbar navbar-expand-lg navbar-light fixed-top" id="mainNav">
<div class="container">
<a class="navbar-brand" href="index.html">CFEVER</a>
<button class="navbar-toggler navbar-toggler-right" type="button" data-toggle="collapse"
data-target="#navbarResponsive" aria-controls="navbarResponsive" aria-expanded="false"
aria-label="Toggle navigation">
Menu
<i class="fa fa-bars"></i>
</button>
</div>
</nav>
<!-- Page Header -->
<header class="masthead" style="background-image: url('img/background.png')">
<div class="overlay"></div>
<div class="container">
<div class="row">
<div class="col-lg-8 col-md-10 mx-auto">
<div class="site-heading">
<h1>CFEVER</h1>
<span class="subheading">
A Fact Extraction and VERification dataset in Traditional Chinese
</span>
</div>
</div>
</div>
</div>
</header>
<!-- Main Content -->
<div class="container" id="main">
<div class="row">
<div class="col-lg-5">
<div class="list-group">
<div class="list-group-item">
<h4>About</h4>
<p></p>
<p>
CFEVER is a Chinese Fact Extraction and VERification dataset.
Inspired by the FEVER dataset <a href="https://aclanthology.org/N18-1074/">(Thorne et al., 2018)</a>,
we provide class labels (Supports, Refutes, or Not Enough Information) and evidence for each claim in the
CFEVER dataset.
CFEVER is currently the largest Chinese fact-checking dataset, and it can serve as a benchmark for Chinese
fact verification and retrieval-augmented generation (RAG).
</p>
<p>For more details about CFEVER, please refer to our AAAI-24 paper:</p>
<p class="text-center">
<a href="https://arxiv.org/abs/2402.13025"><button type="button" class="btn btn-info">CFEVER Paper
(arXiv)</button></a>
</p>
</div>
<div class="list-group-item">
<h4>Dataset</h4>
<!-- <p>CFEVER ... </p> -->
<p>Please visit our GitHub repository to download the dataset:</p>
<p class="text-center">
<a href="https://github.com/IKMLab/CFEVER-data/blob/main/data"><button type="button"
class="btn btn-success">Download CFEVER Dataset</button></a>
</p>
</div>
<div class="list-group-item">
<h4>Submission</h4>
<p>We do not provide the ground-truths for the test set. To submit your model, please follow the
instructions in our GitHub repository.</p>
</div>
<div class="list-group-item">
<h4>Citation</h4>
<p>If you use CFEVER in your research, please cite our paper by:</p>
<pre>
@inproceedings{lin2023CFEVER,
title={CFEVER: A Chinese Fact Extraction and VERification Dataset},
author={Lin, Ying-Jia and Lin, Chun-Yi and Yeh, Chia-Jen and Li, Yi-Ting and Hu, Yun-Yu and Hsu, Chih-Hao and Lee, Mei-Feng and Kao, Hung-Yu},
journal={Proceedings of the AAAI Conference on Artificial Intelligence},
pages={},
year={2024},
month={Feb.}
}</pre>
</div>
</div>
</div>
<div class="col-lg-7">
<div class="card card-outline-secondary">
<div class="card-header">
Leaderboard
</div>
<div class="card-body">
<table class="table table-responsive">
<thead class="thead-light">
<tr>
<th scope="col" rowspan=1></th>
<th scope="col" rowspan=1 class="align-middle text-center">Model</th>
<th scope="col" rowspan=1 class="align-middle text-center">Code</th>
<!-- <th scope="col" rowspan=1 class="align-middle text-center">Size</th> -->
<th scope="col" colspan=1 class='text-center'>Accuracy (%)</th>
<th scope="col" colspan=1 class='text-center'>FEVER Score (%)</th>
</tr>
</thead>
<tbody>
<tr style="background:#f4f4f4">
<td scope="row" class='align-middle text-center'>1<br>
<span class="badge badge-secondary">Feb 20, 2024</span>
</td>
<td class='align-middle text-center'>BEVERS<br />
<span class="affiliation">University of Southern California & University of
Nebraska-Lincoln</span><br /><a href="https://aclanthology.org/2023.fever-1.6/"
class="paperlink">(DeHaven and Scott, 2023)</a>
<br>
<a href="https://arxiv.org/abs/2402.13025" class="paperlink">(Implemented in Lin et al., 2023)</a>
</td>
<td class='align-middle text-center'><a href="https://github.com/mitchelldehaven/bevers"><i
class="far fa-file-alt"></i></a></td>
<td class='align-middle text-center'>69.73</td>
<td class='align-middle text-center'>64.80</td>
</tr>
<tr style="background:#f4f4f4">
<td scope="row" class='align-middle text-center'>2<br>
<span class="badge badge-secondary">Feb 20, 2024</span>
</td>
<td class='align-middle text-center'>Our baseline<br />
<span class="affiliation">National Cheng Kung University</span><br /><a
href="https://arxiv.org/abs/2402.13025" class="paperlink">(Lin et al., 2023)</a>
</td>
<td class='align-middle text-center'><a href="https://github.com/IKMLab/CFEVER-baselines"><i
class="far fa-file-alt"></i></a></td>
<!-- <td class='align-middle text-center'></td> -->
<td class='align-middle text-center'>61.17</td>
<td class='align-middle text-center'>52.47</td>
</tr>
<tr style="background:#f4f4f4">
<td scope="row" class='align-middle text-center'>3<br>
<span class="badge badge-secondary">Apr 24, 2024</span>
</td>
<td class='align-middle text-center'>Kernel Graph Attention Network (KGAT)<br />
<span class="affiliation">Beijing University of Posts and Telecommunications</span><br />
</td>
<td class='align-middle text-center'></td>
<!-- <td class='align-middle text-center'></td> -->
<td class='align-middle text-center'>52.17</td>
<td class='align-middle text-center'>45.83</td>
</tr>
<tr style="background:#f4f4f4">
<td scope="row" class='align-middle text-center'>4<br>
<span class="badge badge-secondary">Feb 20, 2024</span>
</td>
<td class='align-middle text-center'>GPT-4 <i>(3-shot)</i><br />
<span class="affiliation">Microsoft & OpenAI</span><br /><a href="https://arxiv.org/abs/2402.13025"
class="paperlink">(Lin et al., 2023)</a>
</td>
<td class='align-middle text-center'></td>
<!-- <td class='align-middle text-center'></td> -->
<td class='align-middle text-center'>48.40</td>
<td class='align-middle text-center'>NA</td>
</tr>
<tr style="background:#f4f4f4">
<td scope="row" class='align-middle text-center'>5<br>
<span class="badge badge-secondary">Feb 20, 2024</span>
</td>
<td class='align-middle text-center'>GPT-4 <i>(zero-shot)</i><br />
<span class="affiliation">Microsoft & OpenAI</span><br /><a href="https://arxiv.org/abs/2402.13025"
class="paperlink">(Lin et al., 2023)</a>
</td>
<td class='align-middle text-center'></td>
<!-- <td class='align-middle text-center'></td> -->
<td class='align-middle text-center'>47.23</td>
<td class='align-middle text-center'>NA</td>
</tr>
<tr style="background:#f4f4f4">
<td scope="row" class='align-middle text-center'>6<br>
<span class="badge badge-secondary">Feb 20, 2024</span>
</td>
<td class='align-middle text-center'>GPT-3.5 <i>(3-shot)</i><br />
<span class="affiliation">Microsoft & OpenAI</span><br /><a href="https://arxiv.org/abs/2402.13025"
class="paperlink">(Lin et al., 2023)</a>
</td>
<td class='align-middle text-center'></td>
<!-- <td class='align-middle text-center'></td> -->
<td class='align-middle text-center'>44.20</td>
<td class='align-middle text-center'>NA</td>
</tr>
<tr style="background:#f4f4f4">
<td scope="row" class='align-middle text-center'>7<br>
<span class="badge badge-secondary">Feb 20, 2024</span>
</td>
<td class='align-middle text-center'>GPT-3.5 <i>(zero-shot)</i><br />
<span class="affiliation">Microsoft & OpenAI</span><br /><a href="https://arxiv.org/abs/2402.13025"
class="paperlink">(Lin et al., 2023)</a>
</td>
<td class='align-middle text-center'></td>
<!-- <td class='align-middle text-center'></td> -->
<td class='align-middle text-center'>43.17</td>
<td class='align-middle text-center'>NA</td>
</tr>
</tbody>
</table>
</div>
</div>
</div>
</div>
<hr>
<!-- Footer -->
<footer>
<div class="container">
<div class="row">
<div class="col-lg-8 col-md-10 mx-auto">
<p class="copyright text-muted">This work was led by <a href="https://ikmlab.csie.ncku.edu.tw/advisor.html">Prof. Hung-Yu Kao</a>.</p>
<p class="copyright text-muted">Theme adapted from Start Bootstrap's <a
href="https://startbootstrap.com/template-overviews/clean-blog/">Clean Blog</a> template.</p>
<p class="copyright text-muted">Leaderboard maintained by
<a href="mailto: yingjia.lin.public@gmail.com">Ying-Jia Lin</a> and
<a href="mailto: fzq1046@gmail.com">Zhi-Quan Feng</a>.
Last update: 2024-05-25.
</p>
</div>
</div>
</div>
</footer>
<!-- Bootstrap core JavaScript -->
<script src="vendor/jquery/jquery.min.js"></script>
<script src="vendor/bootstrap/js/bootstrap.bundle.min.js"></script>
<!-- Custom scripts for this template -->
<script src="js/clean-blog.min.js"></script>
</body>
</html>