data.json

{
    "https://aclanthology.org/P17-2067.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague opposition",
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "in this past election cycle for the 45th president of the united states, the world has witnessed a growing epidemic of fake news. the plague of fake news not only poses serious threats to the integrity of journalism, but has also created turmoils in the political world. the worst real-world impact is that fake news seems to create real-life fears: last year, a man carried an ar-15 rifle and walked in a washington dc pizzeria, because he recently read online that \u201cthis pizzeria was harboring young children as sex slaves as part of a childabuse ring led by hillary clinton\u201d"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "anecdote"
                    ]
                },
                {
                    "Quotes (what)": [
                        "vlachos and riedel (2014) are the first to release a public fake news detection and fact-checking dataset, but it only includes 221 statements, which does not permit machine learning based assessments. to address these issues, we introduce the liar dataset, which includes 12,836 short statements labeled for truthfulness, subject, context/venue, speaker, state, party, and prior history"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Citation support": [
                        "previous work"
                    ]
                }
            ]
        },
        "year": 2017,
        "annotator_2": {
            "narratives": [
                {
                    "type": "vague opposition",
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "in this past election cycle for the 45th president of the united states, the world has witnessed a growing epidemic of fake news. the plague of fake news not only poses serious threats to the integrity of journalism, but has also created tur- moils in the political world. the worst real-world impact is that fake news seems to create real-life fears: last year, a man carried an ar-15 rifle and walked in a washington dc pizzeria, because he recently read online that \u201cthis pizzeria was harboring young children as sex slaves as part of a child- abuse ring led by hillary clinton\u201d1. the man was later arrested by police, and he was charged for firing an assault rifle in the restaurant (kang and goldman, 2016)."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "anecdotal"
                    ]
                },
                {
                    "Quotes (what)": [
                        "the problem of fake news detection is more challenging than detecting deceptive reviews, since the political language on tv interviews, posts on facebook and twitters are mostly short statements. however, the lack of manually labeled fake news dataset is still a bottleneck for advancing computational-intensive, broad- coverage models in this direction. vlachos and riedel (2014) are the first to release a public fake news detection and fact-checking dataset, but it only includes 221 statements, which does not per- mit machine learning based assessments. to address these issues, we introduce the liar dataset, which includes 12,836 short statements labeled for truthfulness, subject, context/venue, speaker, state, party, and prior history."
                    ],
                    "Citation support": [
                        "previous research article"
                    ]
                },
                {
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        }
    },
    "https://www.ijcai.org/Proceedings/16/Papers/537.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "automated external fact-checking",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Citation Support for Narratives": [
                        "sense of threat"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "false rumors are damaging as they cause public panic and social unrest. for example, on august 25th of 2015, a rumor about \u201cshootouts and kidnappings by drug gangs happening near schools in veracruz\u201d spread through twitter and facebook1. this caused severe chaos in the city involving 26 car crashes, because people left their cars in the middle of a street and rushed to pick up their children from school."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "anecdote"
                    ]
                },
                {
                    "Quotes (what)": [
                        "this incident of a false rumor highlights that automatically predicting the veracity of information on social media is of high practical value."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                },
                {
                    "Quotes (what)": [
                        "debunking rumors at an early stage of diffusion is particularly crucial to minimizing their harmful effects. to distin- guish rumors from factual events, individuals and organiza- tions often have relied on common sense and investigative journalism. rumor reporting websites like snopes.com and factcheck.org are such collaborative efforts. however, be- cause manual verification steps are involved in such efforts, these websites are not comprehensive in their topical cover- age and also can have long debunking delay."
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ]
        },
        "year": 2016,
        "annotator_2": {
            "narratives": [
                {
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation Support for Narratives": [
                        "sense of threat"
                    ],
                    "type": "automated external fact-checking"
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "false rumors are damaging as they cause public panic and social unrest.for example, on august 25th of 2015, a rumor about \u201cshootouts and kid- nappings by drug gangs happening near schools in veracruz\u201d spread through twitter and facebook1. this caused severe chaos in the city involving 26 car crashes, because people left their cars in the middle of a street and rushed to pick up their children from school. this incident of a false rumor high- lights that automatically predicting the veracity of informa- tion on social media is of high practical value."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "anecdotal"
                    ]
                },
                {},
                {
                    "Quotes (what)": [
                        "this incident of a false rumor highlights that automatically predicting the veracity of information on social media is of high practical value."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "debunking rumors at an early stage of diffusion is particularly crucial to minimizing their harmful effects. to distinguish rumors from factual events, individuals and organizations often have relied on common sense and investigative journalism. rumor reporting websites like snopes.com and factcheck.org are such collaborative efforts. however, because manual verification steps are involved in such efforts, these websites are not comprehensive in their topical coverage and also can have long debunking delay."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/N18-1074.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague opposition",
                    "Data Subjects": [
                        "technical writers",
                        "product reviewers"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the ever-increasing amounts of textual information available combined with the ease in sharing it through the web has increased the demand for verification, also referred to as fact checking. while it has received a lot of attention in the context of journalism, verification is important for other domains, e.g. information in scientific publications, product reviews, etc."
                    ],
                    "Data Subjects": [
                        "technical writers",
                        "product reviewers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper we focus on verification of textual claims against textual sources. when compared to textual entailment (te)/natural language inference (dagan et al., 2009; bowman et al., 2015), the key difference is that in these tasks the passage to verify each claim is given, and in recent years it typically consists a single sentence, while in verification systems it is retrieved from a large set of documents in order to form the evidence"
                    ],
                    "Model Means": [
                        "evidence retrieval",
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [
                {
                    "Data Subjects": [
                        "technical writers",
                        "product reviewers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "type": "vague opposition",
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the ever-increasing amounts of textual information available combined with the ease in sharing it through the web has increased the demand for verification, also referred to as fact checking. while it has received a lot of attention in the context of journalism, verification is important for other domains, e.g. information in scientific publications, product reviews, etc."
                    ],
                    "Data Subjects": [
                        "technical writers",
                        "product reviewers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper we focus on verification of textual claims against textual sources. when compared to textual entailment (te)/natural language infer- ence (dagan et al., 2009; bowman et al., 2015), the key difference is that in these tasks the passage to verify each claim is given, and in recent years it typically consists a single sentence, while in veri- fication systems it is retrieved from a large set of documents in order to form the evidence. another related task is question answering (qa), for which approaches have recently been extended to han- dle large-scale resources such as wikipedia (chen et al., 2017). however, questions typically pro- vide the information needed to identify the answer, while information missing from a claim can of- ten be crucial in retrieving refuting evidence."
                    ],
                    "Model Means": [
                        "evidence retrieval",
                        "classify/score veracity"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper we present a new dataset for claim verification, fever: fact extraction and ver- ification. it consists of 185,445 claims manually verified against the introductory sections of wikipedia pages and classified as supported, refuted or notenoughinfo"
                    ]
                },
                {
                    "Quotes (what)": [
                        "to characterize the challenges posed by fever we develop a pipeline approach which, given a claim, first identifies relevant documents, then selects sentences forming the evidence from the doc- uments and finally classifies the claim w.r.t. ev- idence."
                    ],
                    "Quotes (why)": [
                        "however, despite the rising interest in verification and fact checking among researchers, the datasets currently used for this task are limited to a few hundred claims."
                    ],
                    "Model Means": [
                        "evidence retrieval",
                        "classify/score veracity"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/C18-1287.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague opposition",
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the proliferation of misleading information in everyday access media outlets such as social me- dia feeds, news blogs, and online newspapers have made it challenging to identify trustworthy news sources, thus increasing the need for computational tools able to provide insights into the reliability of online content."
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (why)": [
                        "we conduct a set of learning experiments to build accurate fake news detectors, and show that we can achieve accuracies of up to 76%."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "fake news detection has recently attracted a growing interest from the general public and researchers as the circulation of misinformation online increases, particularly in media outlets such as social media feeds, news blogs, and online newspapers. a recent report by the jumpshot tech blog showed that facebook referrals accounted for 50% of the total traffic to fake news sites and 20% total traffic to reputablewebsites.1 since as many as 62% of u.s. adults consume news on social media(jeffreyand elisa, 2016), being able to identify fake content in online sources is a pressing need."
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "news article"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we develop computational resources and models for the task of fake news detection. we introduce two novel datasets covering seven different domains. one of the datasets is collected by combining manual and crowdsourced annotation approaches, while the second is collected directly from the web. using these datasets, we conduct several exploratory analyses to identify linguistic properties that are predominantly present in fake news content, and we build fake news detectors relying on linguistic features that achieve accuracies of up to 76%. to place our results in perspective, we compare the performance of the developed classifiers with an empirical human baseline."
                    ]
                },
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/D17-1317.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "automated external fact-checking",
                    "Data Subjects": [
                        "public figures/politicians",
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Citation Support for Narratives": [
                        "investigative"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "words in news media and political discourse have a considerable power in shaping people\u2019s beliefs and opinions. as a result, their truthfulness is often compromised to maximize impact. recently, fake news has captured worldwide interest, and the number of organized efforts dedicated solely to fact-checking has almost tripled since 2014"
                    ],
                    "Data Subjects": [
                        "professional journalists",
                        "public figures/politicians"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "increasing number of fact-checkers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "to probe the feasi- bility of automatic political fact-checking, we also present a case study based on politifact.com using their factuality judg- ments on a 6-point scale."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ]
                }
            ]
        },
        "year": 2017,
        "annotator_2": {
            "narratives": [
                {
                    "Data Subjects": [
                        "public figures/politicians",
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Citation Support for Narratives": [
                        "investigative"
                    ],
                    "type": "automated external fact-checking",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "words in news media and political discourse have a considerable power in shaping people\u2019s beliefs and opinions. as a result, their truthfulness is of- ten compromised to maximize impact. recently, fake news has captured worldwide interest, and the number of organized efforts dedicated solely to fact-checking has almost tripled since 2014.1 organizations, such as politifact.com, actively investigate and rate the veracity of comments made by public figures, journalists, and organizations."
                    ],
                    "Data Subjects": [
                        "professional journalists",
                        "public figures/politicians"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "news article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "analysis indicates that falsehoods often arise from subtle differences in phrasing rather than outright fabrication (rubin et al., 2015). compared to most prior work on deception literature that focused on binary categorization of truth and deception, political fact-checking poses a new challenge as it involves a graded notion of truthfulness."
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we present an analytic study characterizing the language of political quotes and news media written with varying intents and degrees of truth. we also investigate graded deception detection, determining the truthfulness on a 6-point scale using the political fact-checking database available at politifact."
                    ],
                    "Data Subjects": [
                        "professional journalists",
                        "public figures/politicians"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ]
                }
            ]
        }
    },
    "https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0150989": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "scientific curiosity",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Data Actors": [
                        "scientists"
                    ],
                    "Model Owners": [
                        "scientists"
                    ],
                    "Application Means": [
                        "analyse data"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "Ends": [
                        "develop knowledge of nlp/language"
                    ]
                },
                {
                    "type": "vague opposition",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "while rumours in social media are a concern, little work has been done so far to understand how they propagate. in this work we aim to help rectify this by examining in some detail rumours generated on twitter within the context of nine different newsworthy events."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Data Actors": [
                        "scientists"
                    ],
                    "Model Owners": [
                        "scientists"
                    ],
                    "Application Means": [
                        "analyse data"
                    ],
                    "Ends": [
                        "develop knowledge of nlp/language",
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (why)": [
                        "whilst one can readily see users denying rumours once they have been debunked, users appear to be less capable of distinguishing true from false rumours when their veracity remains in question. in fact, we show that the prevalent tendency for users is to support every unverified rumour. [...] our study reinforces the need for developing robust machine learning techniques that can provide assistance in real time for assessing the veracity of rumours"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ]
        },
        "year": 2016,
        "annotator_2": {
            "narratives": [
                {
                    "type": "vague opposition"
                },
                {
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "type": "scientific curiosity"
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the potential for spreading information quickly through a large community of users is one of the most valuable characteristics of social media platforms. social media, being open to everyone, enable not only news organisations and journalists to post news stories, but also ordinary citizens to report from their own perspectives and experiences. this broadens the scope and diversity of information that one can get from social media and some- times may even lead to stories breaking before they appear in mainstream media outlets [1]. while this often leads to having access to more comprehensive information, it also comes with caveats, one of which is the need to sift through the different information sources to assess their accuracy [2]."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "news article"
                    ]
                },
                {
                    "Quotes (what)": [
                        "while rumours in social media are a concern, little work has been done so far to understand how they propagate. in this work we aim to help rectify this by examining in some detail rumours generated on twitter within the context of nine different newsworthy events."
                    ],
                    "Data Actors": [
                        "scientists"
                    ],
                    "Application Means": [
                        "analyse data"
                    ]
                },
                {
                    "Quotes (what)": [
                        "our study looks at conversations around rumours in social media, exploring how social media users respond to rumours both before and after the veracity of a rumour is resolved. our study provides insight into rumour diffusion, support and denial in social media, helping both those who gather news from social media in determining accuracy of information and the development of machine learning systems that can provide assistance in real-time for assessing the veracity of rumours [6]."
                    ],
                    "Quotes (why)": [
                        "the spread of misinformation is especially important in the context of breaking news, where new pieces of information are released piecemeal, often starting off as unverified infor- mation in the form of a rumour. these rumours then spread to large numbers of users, influ- encing perception and understanding of events, despite being unverified. social media rumours that are later proven false can have harmful consequences both for individuals and for society [3]. for instance, a rumour in 2013 about the white house having been bombed, injuring barack obama, which was tweeted from ap\u2019s twitter account by hackers, spooked stock markets in the us [4]. a major event that was similarly riddled with consequential rumours was hurricane sandy, which hit the east coast of the us in 2012. part of the city of new york suffered from power outages and many people had to rely on the internet accessed through their mobile phones for information. to prevent major incidents, the us federal emergency management agency had to set up a web page specifically for rumour control [5]."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {}
            ]
        }
    },
    "https://aclanthology.org/P18-1022.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted media consumption",
                    "Data Actors": [
                        "media consumers"
                    ],
                    "Model Owners": [
                        "social media companies"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores",
                        "identify claims"
                    ],
                    "Citation Support for Narratives": [
                        "vague community"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "type": "automated content moderation",
                    "Data Actors": [
                        "algorithm"
                    ],
                    "Model Owners": [
                        "social media companies"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Citation Support for Narratives": [
                        "vague community"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the fake news hype caused a widespread disillusionment about so- cial media, and many politicians, news publishers, it companies, activists, and scientists concur that this is where to draw the line"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "vague community"
                    ]
                },
                {
                    "Quotes (what)": [
                        "many favor a two-step approach where fake news items are detected and then countermeasures are implemented to foreclose rumors and to dis- courage repetition. while some countermeasures are already tried in practice, such as displaying warnings and withholding ad revenue, fake news detection is still in its infancy."
                    ],
                    "Data Actors": [
                        "media consumers",
                        "algorithm"
                    ],
                    "Model Owners": [
                        "social media companies"
                    ],
                    "Application Means": [
                        "identify claims",
                        "provide labels/veracity scores"
                    ],
                    "Citation support": [
                        "vague community"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we show how a style analysis can distin- guish hyperpartisan news from the main- stream (f1 = 0.78), and satire from both (f1 = 0.81)"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [
                {
                    "Application Means": [
                        "provide labels/veracity scores",
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation Support for Narratives": [
                        "vague community"
                    ],
                    "type": "automated content moderation",
                    "Model Owners": [
                        "social media companies"
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the media and the public are currently discussing a new phenomenon called \u201cfake news\u201d and its potential role in swaying recent elections, how it may affect democratic societies, and what can and should be done about it."
                    ],
                    "Model Owners": [
                        "social media companies"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (why)": [
                        "although traditional yellow press has been spreading \u2018news\u2019 of varying degrees of truthfulness long before the digital revolution, the fact that modern social media amplify fake news to outperform real news gives many people pause. the fake news hype caused a widespread disillusionment about social media, and many politicians, news publish- ers, it companies, activists, and scientists concur that this is where to draw the line."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Owners": [
                        "social media companies"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "many favor a two-step approach where fake news items are detected and then countermeasures are implemented to foreclose rumors and to discourage repetition. while some countermeasures are already tried in practice, such as displaying warnings and withholding ad revenue, fake news detection is still in its infancy. at any rate, a near- real time reaction is crucial: once a fake news item begins to spread virally, the damage is done and un- doing it becomes arduous. since knowledge-based and context-based approaches to fake news detection can only be applied after publication, i.e., as news events unfold and as social interactions occur, they may not be fast enough. we have identified style-based approaches as a viable alternative, allowing for instantaneous re- actions, albeit not to fake news, but to hyperpartisanship. in this regard we contribute (1) a large news corpus annotated by experts with respect to veracity and hyperpartisanship, (2) extensive experiments on discriminating fake news, hyperpartisan news, and satire based solely on writing style, and (3) validation experiments to verify our finding that the writing style of the left and the right have more in common than any of the two have with the mainstream, applying unmasking in a novel way."
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                }
            ]
        }
    },
    "https://dl.acm.org/doi/pdf/10.1145/3219819.3219903": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the dissemination of fake news may cause large-scale negative effects, and sometimes can affect or even manipulate important public events. for example, within the final three months of the 2016 u.s. presidential election, the fake news generated to favor either of the two nominees was believed by many people and was shared by more than 37 million times on facebook [ 1, 7 ]. therefore, it is in great need of an auto- matic detector to mitigate the serious negative effects caused by the fake news"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "anecdote"
                    ]
                }
            ]
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "type": "vague identification",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the recent proliferation of social media has significantly changed the way in which people acquire information. nowadays, there are increasingly more people consuming news through social media, which can provide timely and comprehensive multimedia information on the events taking place all over the world. compared with traditional text news, the news with images and videos can provide a better storytelling and attract more attention from readers. unfortunately, this is also taken advantage by fake news which usually contain misrepresented or even forged images, to mislead the readers and get rapid dissemination."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "the dissemination of fake news may cause large-scale negative effects, and sometimes can affect or even manipulate important public events. for example, within the final three months of the 2016 u.s. presidential election, the fake news generated to favor either of the two nominees was believed by many people and was shared by more than 37 million times on facebook [1, 7]. therefore, it is in great need of an auto- matic detector to mitigate the serious negative effects caused by the fake news."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (why)": [
                        "thus far, various fake news detection approaches, including both traditional learning [6, 15, 29] and deep learning based models [21, 25], have been exploited to identify fake news. with sufficient verified posts on different events, existing deep learning models have achieved performance improvement over traditional ones due to their superior ability of feature extraction. however, they are still not able to handle the unique challenge of fake news detection, i.e., detecting fake news on newly emerged and time-critical events [27]. due to lack of the corresponding prior knowledge, the verified posts about such events can be hardly obtained in a timely manner, which leads to the unsatisfactory performance of existing models. actually, existing models tend to capture lots of event-specific features which are not shared among different events. such event-specific features, though being able to help classify the posts on verified events, would hurt the detection with regard to newly emerged events."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "for this reason, instead of capturing event-specific features, we believe that learning the shared features among all the events would help us with the detection of fake news from unverified posts. therefore, the goal of this work is to design an effective model to remove the nontransferable event-specific features and preserve the shared features among all the events for the task of identifying fake news"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "to remove event-specific features, the first step is to identify them. for posts on different events, they have their own unique or specific features that are not sharable. such features can be de- tected by measuring the difference among posts corresponding to different events. here the posts can be represented by the learned features. thus, identifying event-specific features is equivalent to measuring the difference among learned features on different events. however, it is a technically challenging problem. first, since the learned feature representations of posts are high-dimensional, simple metrics like the squared error may not be able to estimate the differences among such complicated feature representations. second, the feature representations keep changing during the train- ing stage. this requires the proposed measurement mechanism to capture the changes of feature representations and consistently pro- vide the accurate measurement. although this is very challenging, the effective estimation of dissimilarities among the learned fea- tures on different events is the premise of removing event-specific features. thus, how to effectively estimate the dissimilarities under this condition is the challenge that we have to address. in order to address the aforementioned challenges, we propose an end-to-end framework referred to as event adversarial neural networks (eann) for fake news detection based on multi-modal features. inspired by the idea of adversarial networks [10], we incor- porate the event discriminator to predict the event auxiliary labels during training stage, and the corresponding loss can be used to estimate the dissimilarities of feature representations among differ- ent events."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "the proposed model eann consists of three main compo- nents: the multi-modal feature extractor, the fake news detector, and the event discriminator. the multi-modal feature extractor cooperates with the fake news detector to carry out the major task of identifying false news. simultaneously, the multi-modal feature extractor tries to fool the event discriminator to learn the event invariant representations."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/W16-0802.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "automated content moderation",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Data Actors": [
                        "algorithm"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "automated removal"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "type": "assisted media consumption",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Data Actors": [
                        "media consumers"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "type": "assisted internal fact-checking",
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "high rates of media consumption and low trust in news institutions create an optimal environment for the \u201crapid viral spread of information that is either intentionally or unintentionally misleading or pro- vocative\u201d (howell, 2013). journalists and other content producers are incentivized towards speed and spectacle over accuracy (chen, conroy, & rubin, 2015) and content consumers often lack the literacy skills required to interpret news critically (hango, 2014). what is needed for both content producers and consumers is an automated assistive tool that can save time and cognitive effort by flagging/filtering inaccurate or false information."
                    ],
                    "Data Subjects": [
                        "professional journalists",
                        "social media users"
                    ],
                    "Data Actors": [
                        "professional journalists",
                        "media consumers",
                        "algorithm"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores",
                        "automated removal"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "social science"
                    ]
                }
            ]
        },
        "year": 2016,
        "annotator_2": {
            "narratives": [
                {
                    "Data Subjects": [
                        "social media users",
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "type": "assisted internal fact-checking",
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Data Actors": [
                        "professional journalists",
                        "media consumers"
                    ],
                    "Application Means": [
                        "automated removal"
                    ],
                    "type": "automated content moderation",
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "type": "assisted media consumption",
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "in recent years, there has been a trend towards de- creasing confidence in the mainstream media. ac- cording to gallup polls, only 40% of americans trust their mass media sources to report the news \u201cfully, accurately and fairly\u201d (riffkin, 2015) and a similar survey in the uk has shown that the most- read newspapers were also the least-trusted (reilly & nye, 2012). one effect of this trend has been to drive news readers to rely more heavily on alternative information sources, including blogs and social media, as a means to escape the perceived bias and unreliability of mainstream news (tsfati, 2010)."
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific articls"
                    ]
                },
                {
                    "Quotes (why)": [
                        "in general, humans are fairly ineffective at recognizing deception (depaulo, charlton, cooper, lindsay, & muhlenbruck, 1997; rubin & conroy, 2012; vrij, mann, & leal, 2012)."
                    ],
                    "Citation support": [
                        "polls"
                    ]
                },
                {
                    "Quotes (what)": [
                        "high rates of media consumption and low trust in news institutions create an optimal environment for the \u201crapid viral spread of information that is either intentionally or unintentionally misleading or pro- vocative\u201d (howell, 2013). journalists and other content producers are incentivized towards speed and spectacle over accuracy (chen, conroy, & rubin, 2015) and content consumers often lack the literacy skills required to interpret news critically (hango, 2014). what is needed for both content producers and consumers is an automated assistive tool that can save time and cognitive effort by flagging/filtering inaccurate or false information. in developing such a tool, we have chosen news satire as a starting point for the investigation of deliberate deception in news. unlike subtler forms of deception, satire may feature more obvious cues that reveal its disassociation from truth because the objective of satire is for at least some subset of readers to recognize the joke (pfaff & gibbs, 1997). and yet, articles from the onion and other satirical news sources are often shared and even reprinted in newspapers as if the stories were true."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                },
                {
                    "Data Actors": [
                        "media consumers",
                        "professional journalists"
                    ],
                    "Application Means": [
                        "automated removal"
                    ]
                },
                {}
            ]
        }
    },
    "https://arxiv.org/pdf/1501.03471.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague opposition",
                    "Citation Support for Narratives": [
                        "automation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "traditional fact checking by expert journalists cannot keep up with the enormous volume of information that is now generated online. computational fact checking may significantly enhance our ability to evaluate the veracity of dubious information."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "our initial focus is on computing the support of simple statements of fact using a large-scale knowledge graph obtained from wikipedia."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2015,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "type": "vague opposition",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "while attempts to partially automate the detection of various forms of misinformation are burgeoning [11, 12, 13, 14, 15], automated reasoning methods are hampered by the inherent ambiguity of language and by deliberate deception. however, under certain conditions, reliable knowledge transmission can take place online [16]."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (what)": [
                        "here we show that we can leverage any collection of factual human knowledge, such as wikipedia, for automatic fact checking [20] loosely inspired by the principle of epistemic closure [21], we computationally gauge the support for statements by mining the connectivity patterns on a knowledge graph. our initial focus is on computing the support of simple statements of fact using a large-scale knowledge graph obtained from wikipedia."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/P09-2078.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "scientific curiosity",
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "Citation Support for Narratives": [
                        "investigative"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "in this paper, we explore the applicability of computational approaches to the recognition of deceptive language. in particular, we investigate whether automatic classification techniques represent a viable approach to distinguish between truth and lies as expressed in written text."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                }
            ]
        },
        "year": 2009,
        "annotator_2": {
            "narratives": [
                {
                    "Citation Support for Narratives": [
                        "investigative"
                    ],
                    "type": "scientific curiosity",
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the discrimination between truth and falsehood has received significant attention from fields as diverse as philosophy, psychology and sociology. recent advances in computational linguistics motivate us to approach the recognition of deceptive language from a data-driven perspective, and attempt to identify the salient features of lying texts using natural language processing techniques"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we explore the applicability of computational approaches to the recognition of deceptive language in particular, we investigate whether automatic classification techniques repre- sent a viable approach to distinguish between truth and lies as expressed in written text."
                    ]
                },
                {
                    "Quotes (what)": [
                        "specifically, we try to answer the following two questions. first, are truthful and lying texts separable, and does this property hold for different datasets? to answer this question, we use three different data sets that we construct for this purpose"
                    ]
                },
                {
                    "Quotes (what)": [
                        "second, if truth and lies are separable, what are the distinctive features of deceptive texts? in answer to this second question, we attempt to identify some of the most salient features of lying texts, and analyse their occurrence in the three data sets."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/P18-1184.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "automated external fact-checking",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "type": "assisted external fact-checking",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "analysis shows that people tend to stop spread- ing a rumor if it is known as false (zubiaga et al., 2016b). however, identifying such misinforma- tion is non-trivial and needs investigative jour- nalism to fact check the suspected claim, which is labor-intensive and time-consuming. the pro- liferation of social media makes it worse due to the ever-increasing information load and dynam- ics. therefore, it is necessary to develop automatic and assistant approaches to facilitate real-time ru- mor tracking and debunking."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Application Means": [
                        "identify claims",
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "prior work"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we present a neural rumor de- tection approach based on recursive neural net- works (rvnn) to bridge the content semantics and propagation clues."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [
                {
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers",
                        "identify claims"
                    ],
                    "type": "assisted external fact-checking",
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                },
                {
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "type": "automated content moderation",
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "rumors have always been a social disease. in recent years, it has become unprecedentedly conve- nient for the \u201cevil-doers\u201d to create and disseminate rumors in massive scale with low cost thanks to the popularity of social media outlets on twitter, facebook, etc. the worst effect of false rumors could be devastating to individual and/or society."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "previous work"
                    ]
                },
                {
                    "Quotes (why)": [
                        "analysis shows that people tend to stop spread- ing a rumor if it is known as false (zubiaga et al., 2016b). however, identifying such misinformation is non-trivial and needs investigative journalism to fact check the suspected claim, which is labor-intensive and time-consuming. the proliferation of social media makes it worse due to the ever-increasing information load and dynamics. therefore, it is necessary to develop automatic and assistant approaches to facilitate real-time ru- mor tracking and debunking."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we present a neural rumor detection approach based on recursive neural net- works (rvnn) to bridge the content semantics and propagation clues."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        ""
                    ],
                    "Quotes (why)": [
                        "so, why can such neural model do better for the task? analysis has generally found that twitter could \u201cself-correct\u201d some inaccurate information as users share opinions, conjectures and evidences (zubiaga et al., 2017)."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {}
            ]
        }
    },
    "https://aclanthology.org/W14-2508.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "automated external fact-checking",
                    "Application Means": [
                        "gather and present evidence"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "type": "assisted knowledge curation",
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Owners": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Application Means": [
                        "maintain consistency with kb"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "type": "assisted media consumption",
                    "Data Actors": [
                        "media consumers"
                    ],
                    "Application Means": [
                        "vague persuasion"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "type": "assisted internal fact-checking",
                    "Data Subjects": [
                        "citizen journalists"
                    ],
                    "Data Actors": [
                        "citizen journalists"
                    ],
                    "Application Means": [
                        "vague persuasion"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "fact-checking is a time-consuming process. in assessing the first claim in figure 1 a journalist would need to consult a variety of sources to find the average \u201cfull-time earnings\u201d for criminal bar- risters. fact checking websites commonly provide the detailed analysis (not shown in the figure) per- formed to support the verdict. automating the process of fact checking has re- cently been discussed in the context of computa- tional journalism (cohen et al., 2011; flew et al., 2012)."
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "inspired by the recent progress in natural language processing, databases and information retrieval, the vision is to provide journalists with tools that would allow them to perform this task automatically, or even render the articles \u201clive\u201d by updating them with most current data."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Owners": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Application Means": [
                        "maintain consistency with kb"
                    ]
                },
                {
                    "Quotes (why)": [
                        "furthermore, ordinary citizens need to fact check the information provided to them. this need is intensified with the proliferation of social media such as twitter, since the dissemination of news and information commonly circumvents the tra- ditional news channels (petrovic, 2013)"
                    ],
                    "Data Actors": [
                        "media consumers"
                    ],
                    "Application Means": [
                        "vague persuasion"
                    ],
                    "Citation support": [
                        "scientifc article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "in addition, the rise of citizen journalism (goode, 2009) suggests that often citizens become the sources of information. since the information provided by them is not edited or curated, automated fact checking would assist in avoiding the spreading false information."
                    ],
                    "Data Subjects": [
                        "citizen journalists"
                    ],
                    "Data Actors": [
                        "citizen journalists"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Application Means": [
                        "vague persuasion"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ]
        },
        "year": 2014,
        "annotator_2": {
            "narratives": [
                {
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Application Means": [
                        "maintain consistency with kb"
                    ],
                    "type": "assisted knowledge curation",
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Data Actors": [
                        "citizen journalists"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "type": "assisted media consumption",
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Application Means": [
                        "vague persuasion"
                    ],
                    "type": "assisted internal fact-checking",
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "fact-checking is a time-consuming process. in assessing the first claim in figure 1 a journalist would need to consult a variety of sources to find the average \u201cfull-time earnings\u201d for criminal bar- risters. fact checking websites commonly provide the detailed analysis (not shown in the figure) performed to support the verdict."
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "inspired by the recent progress in natural language processing, databases and information retrieval, the vision is to provide journalists with tools that would allow them to perform this task automatically, or even render the articles \u201clive\u201d by updating them with most current data"
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Application Means": [
                        "maintain consistency with kb"
                    ]
                },
                {
                    "Quotes (why)": [
                        "furthermore, ordinary citizens need to fact check the information provided to them. this need is intensified with the proliferation of social media such as twitter, since the dissemination of news and information commonly circumvents the traditional news channels (petrovic, 2013). in addition, the rise of citizen journalism (goode, 2009) suggests that often citizens become the sources of information. since the information provided by them is not edited or curated, automated fact checking would assist in avoiding the spreading false information."
                    ],
                    "Data Actors": [
                        "citizen journalists"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper we define the task of fact-checking. we then detail the construction of a dataset using fact-checked statements available online. finally, we describe the challenges it poses and its relation to current research in natural language processing."
                    ],
                    "Application Means": [
                        "vague persuasion"
                    ]
                }
            ]
        }
    },
    "https://dl.acm.org/doi/pdf/10.1145/3292500.3330935": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "social media enables users to get exposed to a myriad of misinformation and disinformation, including fake news, i.e., news stories with intentionally false information [ 1, 40]. for example, a report es- timated that over 1 million tweets were related to the fake news story \u201cpizzagate\u201d by the end of 2016 presidential election2. such widespread of fake news has detrimental societal effects."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "therefore, it has become critically important to be able to curtail the spread of fake news on social media, promoting trust in the entire news ecosystem. however, detecting fake news on social media presents unique challenges."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ]
        },
        "year": 2019,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "type": "vague identification",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "social media platforms provide convenient conduit for users to create, access, and share diverse information. due to the increased usage and convenience of social media, more people seek out and receive timely news information online. for example, the pew research center announced that approximately 68% of us adults get news from social media in 2018, while only 49% reported seeing news on social media in 20121. however, at the same time, social media enables users to get exposed to a myriad of misinformation and disinformation, including fake news, i.e., news stories with intentionally false information [1, 40]. for example, a report es- timated that over 1 million tweets were related to the fake news story \u201cpizzagate\u201d by the end of 2016 presidential election."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "news articles"
                    ]
                },
                {
                    "Quotes (why)": [
                        "such widespread of fake news has detrimental societal effects. first, it significantly weakens the public trust in governments and journalism. for example, the reach of fake news during the 2016 u.s. presidential election campaign for top-20 fake news pieces was, ironically, larger than the top-20 most-discussed true stories3. second, fake news may change the way people respond to legitimate news. a study has shown that people\u2019s trust in mass media has dramatically degraded across different age groups and political parties4. third, rampant \u201conline\u201d fake news can lead to \u201coffline\u201d societal events. for example, fake news claiming that barack obama was injured in an explosion wiped out $130 billion in stock value5. therefore, it has become critically important to be able to curtail the spread of fake news on social media, promoting trust in the entire news ecosystem."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "despite the success of existing deep learning based fake news detection methods, however, the majority of these methods focus on detecting fake news effectively with latent features but cannot explain \u201cwhy\u201d a piece of news was detected as fake news. being able to explain why news was determined as fake is much desirable because: (1) the derived explanation can provide new insights and knowledge originally hidden to practitioners; and (2) extracting explainable features from noisy auxiliary information can further help improve fake news detection performance. however, to our best knowledge, there has been no prior attempt to computationally detect fake news with proper explanation on social media. in particular, we propose to derive explanation from the perspectives of news contents and user comments (see figure 1)."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "therefore, in this paper, we study the problem of fake news detection by jointly exploring explainable information from news contents and user comments. first, news contents may contain information that is verifiably false. for example, journalists manually check the claims in news articles on fact-checking websites such as politifact, which is usually laborintensive and time-consuming. researchers also attempt to use external sources to fact-check the claims in news articles to decide and explain whether a news piece is fake or not [6], which may not be able to check newly emerging events (that has not been fact- checked). second, user comments have rich information from the crowd on social media, including opinions, stances, and sentiment, that are useful to detect fake news. for example, researchers propose to use social features to select important comments to predict fake news pieces [13]. moreover, news contents and user comments inherently are related each other and can provide important cues to explain why a given news article is fake or not."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        }
    },
    "https://arxiv.org/pdf/1902.06673.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the extensive spread of fake news has recently become a global problem and threat to modern democracies. the extensive spread of fake news before the united states 2016 presidential elections [3] and the brexit vote in united kingdom has become the centerpiece of the controversy surrounding these political events and allegations of public opinion manipulation. due the very high societal and economic cost of the phenomenon [ 11], in the past year, fake news detection in social media has attracted enormous attention in the academic and industrial realms [16]. automatically detecting fake news poses challenges that defy existing content-based analysis ap- proaches."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                }
            ]
        },
        "year": 2019,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "type": "vague identification",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the extensive spread of fake news has recently become a global problem and threat to modern democracies. the extensive spread of fake news before the united states 2016 presidential elections [3] and the brexit vote in united kingdom has become the centerpiece of the controversy surrounding these political events and allegations of public opinion manipulation."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "automatically detecting fake news poses challenges that defy existing content-based analysis approaches. one of the main reasons is that often the interpretation of the news is highly nuanced and requires the knowledge of political or social context, or \u201ccommon sense\u201d, which even the currently most advanced natural language processing algorithms are still missing."
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we propose learning fake news specific propagation patterns by exploiting geometric deep learning, a novel class of deep learning methods designed to work on graph-structured data [4]. geometric deep learning naturally deals with heterogeneous data (such as user demographic and activity, social network structure, news propagation and content)"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                }
            ]
        }
    },
    "https://dl.acm.org/doi/pdf/10.1145/3123266.3123454": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "according to the check of snopes.com, as many as 529 different rumors pertaining donald trump and hillary clinton were spreading on social media during the election [ 12], which could have impacts on the election. to increase the credibility of information on mircoblogs and prevent the spreading of fake contents, it is crucial to detect rumors automatically on microblogs."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "anecdote"
                    ]
                }
            ]
        },
        "year": 2017,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "type": "vague identification"
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "microblogs, including twitter and chinese weibo, have become important news media and public opinion field in various events recently. for example, during the 2016 u.s. presidential election, candidates and their supporters were actively involved on twitter to do campaigns and express their opin- ions. the convenience and openness of microblogs have also fostered various false rumors and fake news, which have become a serious public concern recently.  according to the check of snopes.com, as many as 529 different rumors pertaining donald trump and hillary clinton were spreading on social media during the election [12], which could have impacts on the election. to increase the credibility of information on mircoblogs and prevent the spreading of fake contents, it is crucial to detect rumors automatically on microblogs."
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "previous work"
                    ]
                },
                {
                    "Quotes (what)": [
                        "considering these limitations and our motivation to leverage multimodal contents, we propose an end-to-end rnn with attention mechanism to fuse features from text, image and social context for the rumor detection task. deep neutral networks are proved to be effective in learning accurate tex- tual or visual representations [26, 29]. in the proposed model (figure 2), we use an rnn to learn the joint representations of text and social context. image visual features, represented with a pre-trained deep cnn, are then fused with them. we employ the attention mechanism in the model to capture the relations between visual features and joint textual/social features."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Citation support": [
                        "none"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/P17-2102.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted media consumption",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Owners": [
                        "social media companies"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "recently, there has been an increased number of disturbing incidents of fabricated stories proliferated through social media having a serious impact on real-world events (perrott, 2016; connolly et al., 2016)"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "several tools have been recently developed to verify and reestablish trusted sources of informa- tion online e.g., google fact checking (gindras, 2016) and facebook repost verification (mosseri, 2016). these projects, among others, teach news literacy2 and contribute to fact-checking online.3 we believe our models and novel findings on lin- guistic differences between suspicious and ver- ified news will contribute to these fact-checking systems, as well as help readers to judge the accu- racy of information they consume in social media."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Owners": [
                        "social media companies"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we first investigate several features and neural network architectures for automatically classifying verified and suspicious news posts, and four sub-types of suspicious news. we find that incorporating linguistic and network features via a \u201clate fusion\u201d technique boosts performance"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2017,
        "annotator_2": {
            "narratives": [
                {
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Owners": [
                        "social media companies"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "type": "assisted media consumption"
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "popular social media platforms such as twitter and facebook have proven to be effective channels for disseminating falsified information, unverified claims, and fabricated attention-grabbing stories due to their wide reach and the speed at which this information can be shared. recently, there has been an increased number of disturbing incidents of fabricated stories proliferated through social media having a serious impact on real-world events (perrott, 2016; connolly et al., 2016)"
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "our novel contributions in this paper are twofold. we first investigate several features and neural network architectures for automatically classifying verified and suspicious news posts, and four sub-types of suspicious news. we find that incorporating linguistic and network features via a \u201clate fusion\u201d technique boosts performance. we then investigate differences between verified and suspicious news tweets by conducting a statistical analysis of linguistic features in both types of account."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                },
                {
                    "Quotes (what)": [
                        "several tools have been recently developed to verify and reestablish trusted sources of informa- tion online e.g., google fact checking (gindras, 2016) and facebook repost verification (mosseri, 2016). these projects, among others, teach news literacy2 and contribute to fact-checking online.3 we believe our models and novel findings on lin- guistic differences between suspicious and ver- ified news will contribute to these fact-checking systems, as well as help readers to judge the accu- racy of information they consume in social media."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Owners": [
                        "social media companies"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/S17-2006.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague debunking",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score stance",
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "rumours are rife on the web. false claims affect people\u2019s perceptions of events and their behaviour, sometimes in harmful ways. with the increasing reliance on the web \u2013 social media, in particular \u2013 as a source of information and news updates by in- dividuals, news professionals, and automated sys- tems, the potential disruptive impact of rumours is further accentuated."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "while breaking news unfold, gathering opinions and evidence from as many sources as possible as communities react becomes crucial to determine the veracity of rumours and consequently reduce the impact of the spread of misinformation."
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ]
                },
                {
                    "Quotes (what)": [
                        "within this scenario where one needs to listen to, and assess the testimony of, different sources to make a final decision with respect to a rumour\u2019s veracity, we ran a task in semeval consisting of two subtasks: (a) stance classification towards ru- mours, and (b) veracity classification."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "classify/score stance"
                    ]
                }
            ]
        },
        "year": 2017,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score stance",
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "type": "vague debunking",
                    "Data Subjects": [
                        "social media users"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "rumours are rife on the web. false claims affect people\u2019s perceptions of events and their behaviour, sometimes in harmful ways. with the increasing reliance on the web \u2013 social media, in particular \u2013 as a source of information and news updates by individuals, news professionals, and automated systems, the potential disruptive impact of rumours is further accentuated."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "while breaking news unfold, gathering opinions and evidence from as many sources as possible as communities react becomes crucial to determine the veracity of rumours and consequently reduce the impact of the spread of misinformation"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "within this scenario where one needs to listen to, and assess the testimony of, different sources to make a final decision with respect to a rumour\u2019s veracity, we ran a task in semeval consisting of two subtasks: (a) stance classification towards rumours, and (b) veracity classification."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "classify/score stance"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ]
        }
    },
    "https://dl.acm.org/doi/pdf/10.1145/3308558.3313552": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the change to neoteric methods of news consumption in recent times has brought the issue of fake news and misinformation to the forefront of discussion. as thousands of new news articles proliferate social media networks every day, with each having nei- ther a credibility nor a validation check, an ecosystem fuelled by misinformation (the inadvertent sharing of false information) and disinformation (the deliberate creation and sharing of information known to be false) has been established."
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (why)": [
                        "this makes the task of detecting fake news a crucial one."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                }
            ]
        },
        "year": 2019,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "type": "vague identification",
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the change to neoteric methods of news consumption in recent times has brought the issue of fake news and misinformation to the forefront of discussion. as thousands of new news articles proliferate social media networks every day, with each having nei- ther a credibility nor a validation check, an ecosystem fuelled by misinformation (the inadvertent sharing of false information) and disinformation (the deliberate creation and sharing of information known to be false) has been established. social media networks have enabled news articles to evolve from traditional text-only news to news with images and videos which can provide a better story- telling experience and has the ability to engage more readers. recent fake news articles leverage this very change to visual context aided news. fake news articles can now contain misrepresented, irrele- vant and forged images to mislead readers. peer-to-peer networks (primarily social media networks) allow fake news (propaganda) to be targeted at users who are more likely to accept and share a par- ticular message. fake news has been in the limelight in recent years for having an extensive negative effect on public events. a major turning point of realization was the 2016 u.s. presidential elections. it was believed that within the final three months leading up to the election, fake news favoring either of the two nominees was accepted and shared by more than 37 million times on facebook [1]."
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (why)": [
                        "this makes the task of detecting fake news a crucial one."
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "the target of our study is detection of news content that is fabricated and can be verified to be false."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                },
                {
                    "Quotes (what)": [
                        "kingma et al. [12] proposed that latent variable models act as a powerful approach to generatively model complicated distributions and brought forth the ideas of variational autoencoder (vae). vaes can learn probabilis- tic latent variable models by optimizing a bound on the marginal likelihood of the observed data. overcoming the limitations of the current models, we propose a multimodal variational autoencoder capable of learning shared (visual + textual) representations, trained to discover correlations across modalities in tweets."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                }
            ]
        }
    },
    "http://eegilbert.org/papers/icwsm15.credbank.mitra.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "scientific curiosity",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Data Actors": [
                        "scientists"
                    ],
                    "Model Owners": [
                        "scientists"
                    ],
                    "Model Means": [
                        "corpora analysis"
                    ],
                    "Application Means": [
                        "analyse data"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "Ends": [
                        "develop knowledge of nlp/language"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "when the ebola virus arrived in the united states, a satirical website claimed that two ebola victims in africa had risen from the dead1. this led to widespread panic about a potential \u201cebola zombie apocalypse\u201d, eventually flooding social media streams with inaccurate information about the disease. there is a popular narrative that social media is full of inaccurate information like this. but how much? do these rumors have structure\u2014temporal, linguistic or otherwise? in this paper, we introduce a new corpus called credbank designed to help answer questions like these"
                    ],
                    "Model Means": [
                        "corpora analysis"
                    ],
                    "Ends": [
                        "develop knowledge of nlp/language"
                    ],
                    "Citation support": [
                        "anecdote",
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "credbank enables a set of new research questions. for example, social scientists might explore what role does the mainstream media plays in online rumors; a data mining researcher might explore how the temporal pattens of rumors differ from highly credible information or study the interplay between highly disputed event and other less disputed ones occurring in the same time span; a health researcher could investigate how folk theo- ries of a new disease (the emergence of ebola is captured in credbank) diffuse through a population."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Data Actors": [
                        "scientists"
                    ],
                    "Model Owners": [
                        "scientists"
                    ],
                    "Model Means": [
                        "corpora analysis"
                    ],
                    "Application Means": [
                        "analyse data"
                    ],
                    "Citation support": [
                        "anecdote"
                    ]
                }
            ]
        },
        "year": 2015,
        "annotator_2": {
            "narratives": [
                {
                    "Data Actors": [
                        "scientists"
                    ],
                    "Model Means": [
                        "corpora analysis"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "type": "scientific curiosity",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "when the ebola virus arrived in the united states, a satirical website claimed that two ebola victims in africa had risen from the dead1. this led to widespread panic about a potential \u201cebola zombie apocalypse\u201d, eventually flooding social media streams with inaccurate information about the disease. there is a popular narrative that social media is full of inaccurate information like this. but how much? do these rumors have structure\u2014temporal, linguistic or otherwise?"
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "anecdote"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we introduce a new corpus called credbank designed to help answer questions like these. credbank systematically weaves together machine computation with the judgements of human annotators to produce credibility annotations of the global tweet stream."
                    ]
                },
                {
                    "Quotes (what)": [
                        "the primary contribution of credbank is a unique dataset compiled to link social media event streams with human credibility judgements in a systematic and comprehensive way."
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "this is the first attempt we are aware of to do so at such a scale and in a naturalistic setting, that is in an environment which closely resembles the way a person would search twitter for event information.. credbank enables a set of new research questions. for example, social scientists might explore what role does the mainstream media plays in online rumors; a data mining researcher might explore how the temporal pattens of rumors differ from highly credible information or study the interplay between highly disputed event and other less disputed ones occurring in the same time span; a health researcher could investigate how folk theo- ries of a new disease (the emergence of ebola is captured in credbank) diffuse through a population."
                    ],
                    "Data Actors": [
                        "scientists"
                    ],
                    "Model Means": [
                        "corpora analysis"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/D18-1003.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "automated external fact-checking",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "justification/explanation production"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "modern media (e.g., news feeds, mi- croblogs, etc.) exhibit an increasing fraction of misleading and manipulative content, from ques- tionable claims and \u201calternative facts\u201d to com- pletely faked news. the media landscape is be- coming a twilight zone and battleground. this so- cietal challenge has led to the rise of fact-checking and debunking websites, such as snopes.com and politifact.com, where people research claims, manually assess their credibility, and present their verdict along with evidence (e.g., background ar- ticles, quotations, etc.). however, this manual ver- ification is time-consuming. to keep up with the scale and speed at which misinformation spreads, we need tools to automate this debunking process."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "de- clare goes far beyond in terms of considering ex- ternal evidence and joint interactions between sev- eral factors, and also in its ability to generate user- interpretable explanations in addition to highly accurate assessments."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "justification/explanation production"
                    ]
                }
            ]
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ],
                    "type": "vague debunking",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "modern media (e.g., news feeds, microblogs, etc.) exhibit an increasing fraction of misleading and manipulative content, from questionable claims and \u201calternative facts\u201d to completely faked news. the media landscape is be- coming a twilight zone and battleground. this societal challenge has led to the rise of fact-checking and debunking websites, such as snopes.com and politifact.com, where people research claims, manually assess their credibility, and present their verdict along with evidence (e.g., background ar- ticles, quotations, etc.). however, this manual verification is time-consuming. to keep up with the scale and speed at which misinformation spreads, we need tools to automate this debunking process."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "to overcome the limitations of the prior works, we present declare, an end-to-end neural network model for assessing and explaining the credibility of arbitrary claims in natural-language text form. our approach combines the best of both families of prior methods. similar to popat et al. (2017), declare incorporates external evidence or counter- evidence from the web as well as signals from the language style and the trustworthiness of the un- derlying sources. however, our method does not require any feature engineering, lexicons, or other manual intervention. rashkin et al. (2017); wang (2017) also develop an end-to-end model, but de- clare goes far beyond in terms of considering ex- ternal evidence and joint interactions between sev- eral factors, and also in its ability to generate user- interpretable explanations in addition to highly accurate assessments."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ]
                }
            ]
        }
    },
    "https://arxiv.org/pdf/1705.01613.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague opposition",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "measuring accuracy and credibility in text are well-studied topics in disciplines from psychology to journalism[1], [2], [3]. the proliferation of large-scale social media data and its increasing use as a primary news source [4], however, is forcing a re-examination of these issues. past approaches that relied on journalistically trained \u201cgatekeepers\u201d to filter out low-quality content are no longer applicable as social media\u2019s volume has quickly overwhelmed our ability to control quality manually."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (why)": [
                        "computational methods have proven useful in similar contexts where data volumes overwhelm human analysis capabilities. furthermore, regularities in bot behavior [11] and financially motivated sensationalists [12] suggest machine learning-based approaches could help address these quality issues."
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "this method uses a classification model to predict whether a thread of twitter conversation will be labeled as accurate or inaccurate using features inspired by existing work on credibility of twitter stories [13], [6]."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                }
            ]
        },
        "year": 2017,
        "annotator_2": {
            "narratives": [
                {
                    "Data Subjects": [
                        "social media users",
                        "professional journalists"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "type": "vague identification",
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "measuring accuracy and credibility in text are well-studied topics in disciplines from psychology to journalism[1], [2], [3]. the proliferation of large-scale social media data and its increasing use as a primary news source [4], however, is forcing a re-examination of these issues. past approaches that relied on journalistically trained \u201cgatekeepers\u201d to filter out low-quality content are no longer applicable as social media\u2019s volume has quickly overwhelmed our ability to control quality manually."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "instead, platforms like twitter and facebook have allowed questionable and inaccurate \u201cnews\u201d content to reach wide audiences without review. social media users\u2019s bias toward believing what their friends share and what they read regardless of accuracy allows these fake stories to propagate widely through and across multiple platforms[5]."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ]
                },
                {
                    "Quotes (why)": [
                        "despite research into rumor propagation on twitter [6], [7], [8], fake image sharing in disaster aftermath [9], and politically moti- vated \u201castroturfing\u201d [10], rumor and \u201cfake news\u201d are becoming increasingly problematic. computational methods have proven useful in similar contexts where data volumes overwhelm human analysis capabilities."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we present a method for automating \u201cfake news\u201d detection in twitter, one of the most popular online social media platforms. this method uses a classification model to predict whether a thread of twitter conversation will be labeled as accurate or inaccurate using features inspired by existing work on credibility of twitter stories [13], [6]."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                },
                {
                    "Quotes (what)": [
                        "this work makes the following contributions: \u2022 an automated mechanism for classifying popular twitter threads into true and fake news stories, \u2022 an analysis of the different features used by journalists and crowdsourced workers/non-experts in assessing ac- curacy in social media stories, and \u2022 an aligned collection of three datasets that capture accu- racy judgements across true and false stories."
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                }
            ]
        }
    },
    "https://arxiv.org/pdf/2001.06362.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "due to a large number of users and easy access to social media, rumors can spread widely and quickly on social media, bringing huge harm to society and causing a lot of economic losses. therefore, re- garding to the potential panic and threat caused by rumors, it is urgent to come up with a method to identify rumors on social media efficiently and as early as possible."
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "conventional detection methods mainly adopt hand- crafted features such as user characteristics, text contents and propagation patterns to train supervised classifiers"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "type": "vague identification",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "with the rapid development of the internet, social media has become a convenient online platform for users to obtain information, express opinions and communicate with each other. as more and more people are keen to participate in discussions about hot topics and exchange their opinions on social media, many rumors appear. due to a large number of users and easy access to social media, rumors can spread widely and quickly on social media, bringing huge harm to society and causing a lot of economic losses. therefore, regarding to the potential panic and threat caused by rumors, it is urgent to come up with a method to identify rumors on social media efficiently and as early as possible."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "to deal with both propagation and dispersion of rumors, in this paper, we propose a novel bi-directional gcn (bi- gcn), which operates on both top-down and bottom-up propagation of rumors. the proposed method obtains the features of propagation and dispersion via two parts, the top-down graph convolutional networks (td-gcn) and bottom-up graph convolutional networks (bu-gcn), re- spectively."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                }
            ]
        }
    },
    "https://arxiv.org/pdf/1811.07039.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague opposition",
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the explosion of online textual content with unknown in- tegrity and verification raises an important concern about misinformation such as fake news, socio-political decep- tion, and online rumors. this problem of misinformation could potentially produce uncontrollable and harmful social impacts, thus stimulating recent research efforts on lever- aging modern machine learning techniques for automatic fact checking."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this work, we propose a joint system consisting of three connected homogeneous networks for the 3-stage fever task of document retrieval"
                    ],
                    "Model Means": [
                        "evidence retrieval",
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "type": "vague identification",
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Model Means": [
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ],
                    "type": "vague debunking",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the explosion of online textual content with unknown integrity and verification raises an important concern about misinformation such as fake news, socio-political deception, and online rumors. this problem of misinformation could potentially produce uncontrollable and harmful social impacts, thus stimulating recent research efforts on leveraging modern machine learning techniques for automatic fact checking"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this work, we propose a joint system consisting of three connected homogeneous networks for the 3-stage fever task of document retrieval, sentence selection, and claim verification and frame them as a similar semantic matching problem."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                },
                {
                    "Model Means": [
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/2020.acl-main.48.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "justification/explanation production"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the conve- nient and low-cost essence of social networking brings collective intelligence, but at the same time leads to a negative by-product, the propagation of misinformation such as fake news. fake news is a kind of news story possess- ing intentionally false information on social me- dia (rashkin et al., 2017; allcott and gentzkow, 2017). the widespread of fake news can mislead the public, and produce unjust political, economic, or psychological profit for some parties (horne and adali, 2017; allcott and gentzkow, 2017)."
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific paper"
                    ]
                },
                {
                    "Quotes (what)": [
                        "this paper deals with fake news detection un- der a more realistic scenario on social media. we predict whether a source tweet story is fake, given only its short text content and its retweet sequence of users, along with user profiles."
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we require the fake news detection model to be capable of explainability, i.e., highlighting the evidence when determining a story is fake. the model is expected to point out the suspicious retweeters who support the spreading of fake news, and highlight the words they especially pay attention to from the source tweet"
                    ],
                    "Model Means": [
                        "justification/explanation production"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity",
                        "justification/explanation production"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "type": "vague identification",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "social media is indispensable in people\u2019s daily life, where users can express themselves, access news, and interact with each other. information can further spread through the social network. opinions and sentiments on source stories can be reflected by user participation and interaction. the convenient and low-cost essence of social networking brings collective intelligence, but at the same time leads to a negative by-product, the propagation of misinformation such as fake news."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (why)": [
                        "the widespread of fake news can mislead the public, and produce unjust political, economic, or psychological profit for some parties (horne and adali, 2017; allcott and gentzkow, 2017)."
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (what)": [
                        "this paper deals with fake news detection under a more realistic scenario on social media. we predict whether a source tweet story is fake, given only its short text content and its retweet sequence of users, along with user profiles."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                },
                {
                    "Quotes (what)": [
                        "moreover, we require the fake news detection model to be capable of explainability, i.e., highlighting the evidence when determining a story is fake. the model is expected to point out the suspicious retweeters who support the spreading of fake news, and highlight the words they especially pay attention to from the source tweet."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Model Means": [
                        "justification/explanation production"
                    ]
                }
            ]
        }
    },
    "https://resources.mpi-inf.mpg.de/impact/web_credibility_analysis/cikm2016-popat.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "automated external fact-checking",
                    "Data Subjects": [
                        "product reviewers",
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "with the explosive growth of the web, online news, and social media, there is also a large amount of false claims. this issue is present in many domains, ranging from fake reviews on product websites, erroneous stock prices, manipulative statements about companies, celebrities, and politicians, all the way to disseminating false news [4, 7]. determining the credibility of a claim is a challenging task. as reported in [5], even humans sometimes cannot easily distinguish hoax articles in wikipedia from authentic ones."
                    ],
                    "Data Subjects": [
                        "product reviewers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific paper"
                    ]
                },
                {
                    "Quotes (what)": [
                        "with the increasing number of hoaxes and rumors, factchecking websites like snopes.com, politifact.com, truthorfiction.com and others have become popular. these websites compile articles written by experts who manually investigate contentious claims by determining their provenance and authenticity from various sources; and provide a verdict (true or fake) with supporting evidence. the work in this paper aims to replace this manual verification/falsification with an automated system."
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "example: consider the following claim1 from the fake news website thenochill.com: \u201c15 year old killed trespass- ing while playing pokemon go\u201d. our objective is to assess the credibility of this statement as true or fake. for instance, our model classifies this claim as fake."
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2016,
        "annotator_2": {
            "narratives": [
                {
                    "Data Subjects": [
                        "product reviewers",
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores",
                        "supplant human fact-checkers"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "type": "automated external fact-checking",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "with the explosive growth of the web, online news, and social media, there is also a large amount of false claims. this issue is present in many domains, ranging from fake reviews on product websites, erroneous stock prices, manipulative statements about companies, celebrities, and politicians, all the way to disseminating false news [4, 7]. determining the credibility of a claim is a challenging task. as reported in [5], even humans sometimes cannot easily distinguish hoax articles in wikipedia from authentic ones, and quite a few people have mistaken satirical articles (e.g., from theonion.com) as truthful news."
                    ],
                    "Data Subjects": [
                        "product reviewers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (what)": [
                        "with the increasing number of hoaxes and rumors, fact- checking websites like snopes.com, politifact.com, truthorfiction.com and others have become popular. these websites compile articles written by experts who manually investigate contentious claims by determining their provenance and authenticity from various sources; and provide a verdict (true or fake) with supporting evidence. the work in this paper aims to replace this manual verification/falsification with an automated system."
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we present a novel approach to identify fake textual claims, in an open-domain setting, where we do not assume any community-specific characteristics or structure in the input data.  given a claim in the form of a sentence or paragraph, we first use a search engine to identify documents from multiple web-sources, which refer to the claim. we refer to these documents as reporting articles in this paper. then, we analyze the interplay between the language (e.g., bias, subjectivity, etc.) of the retrieved articles, and the reliability of the web-sources where the articles appeared. finally, we propose a distant supervision based classifier which uses these factors to assess the credibility of the claim reported by multiple sources (cf. section 2, 3)."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                }
            ]
        }
    },
    "https://dl.acm.org/doi/pdf/10.1145/3377478": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "humans have been proven to be not proficient in differentiating between truth and falsehood when overloaded with deceptive information. studies in social psychology and communications have demon- strated that human ability to detect deception is only slightly better than chance: typical accuracy rates are in the range of 55%\u201358%, with a mean accuracy of 54% over 1k participants in over 100 experiments [45]. many expert-based (e.g., politifact 2 and snopes 3 ) and crowd-sourced (e.g., fiskkit 4 and textthresher [62]) manual fact-checking websites, tools, and platforms thus have emerged to serve the public on this matter. 5 nevertheless, manual fact-checking does not scale well with the volume of newly created information, especially on social media [60]. hence, automatic fake news detection has been developed in recent years"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific paper"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we propose a model that enables fake news early detection. by solely relying on news content, the model allows to conduct detection when fake news has been published on a news outlet while it has not been disseminated on social media."
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "type": "vague identification",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "meanwhile, humans have been proven to be not proficient in differentiating between truth and falsehood when overloaded with deceptive information. studies in social psychology and communications have demon- strated that human ability to detect deception is only slightly better than chance: typical accuracy rates are in the range of 55%\u201358%, with a mean accuracy of 54% over 1k participants in over 100 experiments [45]. many expert-based (e.g., politifact2 and snopes3) and crowd-sourced (e.g., fiskkit4 and textthresher [62]) manual fact-checking websites, tools, and platforms thus have emerged to serve the public on this matter.5 nevertheless, manual fact-checking does not scale well with the volume of newly created information, especially on social media [60]. hence, automatic fake news detection has been developed in recent years, where current methods can be generally grouped into content-based and propagation-based methods."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article",
                        "news article",
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (why)": [
                        "to detect fake news at an early stage, i.e., when it is published on a news outlet but has not yet spread on social media sites, to take early actions for fake news intervention (i.e., fake new early detection) motivates us to deeply mine news content. such early detection is particularly crucial for fake news as more individuals become exposed to some fake news, the more likely they may trust it [4]. meanwhile, it has been demonstrated that it is difficult to correct one\u2019s cognition after fake news has gained their trust (i.e., semmelweis reflex [3], confirmation bias [34], and anchoring bias [54])."
                    ]
                },
                {
                    "Quotes (what)": [
                        "we propose a model that enables fake news early detection.by solely relying on news content, the model allows to conduct detection when fake news has been published on a news outlet while it has not been disseminated on social media. experimental results on real-world datasets validate the model effectiveness when limited news content information is available."
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we explore the relationships among fake news, types of deception, and clickbaits. by empirically studying their characteristics in, e.g., content quality, sentiment, quantity, and readability, some news patterns unique to fake news or shared with deception or clickbaits are revealed."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        }
    },
    "https://openreview.net/pdf?id=rkeJRhNYDH": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "scientific curiosity",
                    "Application Means": [
                        "analyse data"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "Ends": [
                        "develop knowledge of nlp/language"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the problem of verifying whether a textual hypothesis holds based on the given evidence, also known as fact verification, plays an important role in the study of natural language understanding and semantic representation [...]  (katz & fodor, 1963; van benthem et al., 2008)"
                    ],
                    "Application Means": [
                        "analyse data"
                    ],
                    "Ends": [
                        "develop knowledge of nlp/language"
                    ],
                    "Citation support": [
                        "scientific paper"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Ends": [
                        "develop knowledge of nlp/language"
                    ],
                    "type": "scientific curiosity"
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "since such structured evidence (graphs, tables, or databases) are also ubiquitous in real-world applications like database systems, dialog systems, commercial management systems, social networks, etc, we argue that the fact verification under structured evidence forms is an equivalently important yet under- explored problem."
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (what)": [
                        "therefore, in this paper, we are specifically interested in studying fact verification with semi-structured wikipedia tables (bhagavatula et al., 2013)2 as evidence owing to its structured and ubiquitous nature (jauhar et al., 2016; zhong et al., 2017; pasupat & liang, 2015). to this end, we introduce a large-scale dataset called tabfact, which consists of 118k manually annotated statements with regard to 16k wikipedia tables, their relations are classified as entailed and refuted3. the entailed and refuted statements are both annotated by human workers."
                    ],
                    "Ends": [
                        "develop knowledge of nlp/language"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/S19-2147.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Data Actors": [
                        "professional journalists",
                        "media consumers"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "since the first rumoureval shared task in 2017 (derczynski et al., 2017), interest in automated verification of rumours has deepened, as research has demonstrated the potential impact of false claims on important political outcomes (allcott and gentzkow, 2017)."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific paper"
                    ]
                },
                {
                    "Quotes (what)": [
                        "as a result the importance of educating young people about crit- ical thinking is increasingly emphasised. more- over the european commission\u2019s high level ex- pert group on fake news provides tools to em- power users and journalists to tackle disinforma- tion as one of the five pillars of their recommended approach."
                    ],
                    "Data Actors": [
                        "media consumers",
                        "professional journalists"
                    ]
                },
                {
                    "Quotes (what)": [
                        "within nlp research the tasks of stance clas- sification of news articles and social media posts and the creation of systems to automatically iden- tify false content are gaining momentum. work in credibility assessment has been around since 2011 (castillo et al., 2011)."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                }
            ]
        },
        "year": 2019,
        "annotator_2": {
            "narratives": [
                {
                    "Data Actors": [
                        "professional journalists",
                        "media consumers"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "type": "vague identification",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "since the first rumoureval shared task in 2017 (derczynski et al., 2017), interest in automated verification of rumours has deepened, as research has demonstrated the potential impact of false claims on important political outcomes (allcott and gentzkow, 2017). living in a \u201cpost-truth world\u201d, in which perceived truth can matter more than actual truth (dale, 2017), the dangers posed by unchecked market forces and cheap platforms, as well as poor ability by many readers to discern credible information, are evident."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific articles"
                    ]
                },
                {
                    "Quotes (what)": [
                        "as a result the importance of educating young people about critical thinking is increasingly emphasised . moreover the european commission\u2019s high level expert group on fake news provides tools to empower users and journalists to tackle disinforma- tion as one of the five pillars of their recommended approach."
                    ],
                    "Data Actors": [
                        "professional journalists",
                        "media consumers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "within nlp research the tasks of stance classification of news articles and social media posts and the creation of systems to automatically iden- tify false content are gaining momentum. work in credibility assessment has been around since 2011 (castillo et al., 2011), making use initially of local features. fact checking is a broad com-lex task, challenging the resourcefulness of even a human expert. claims such as \u201dwe send the eu 350 million a week\u201d which is partially true would need to be decomposed into statements to be checked against knowledge bases and multiple sources."
                    ],
                    "Data Actors": [
                        "professional journalists",
                        "media consumers"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                }
            ]
        }
    },
    "https://idir.uta.edu/~naeemul/file/factchecking-cikm15-hassan-cameraready.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "automated external fact-checking",
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "type": "assisted external fact-checking",
                    "Data Subjects": [
                        "public figures/politicians"
                    ],
                    "Data Actors": [
                        "citizen journalists",
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "human in the loop"
                    ],
                    "Application Means": [
                        "triage claims"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "with tech- nology and modern day media helping spread information to mass audiences through all types of channels, there is a press- ing need for checking the veracity of factual claims important to the public."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (why)": [
                        "the process of fact checking requires many challenging steps|extracting natural language sentences from speeches, interviews, press releases, campaign brochures and social media; separating factual claims from opinions, beliefs, hy- perboles, questions, and so on; detecting topics of factual claims and discerning which are \"check-worthy\"; assessing the veracity of such claims, which itself requires collecting information and data, interviewing experts, and presenting evidence and explanations.2 part of the goal of computational journalism [3, 4] is use computing to automate fact checking [11, 9]."
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "a fully au- tomatic fact checking system is not yet within our reach. it calls for breakthroughs in several fronts related to the aforementioned fact checking steps. this paper's focus is on detecting check-worthy factual claims from natural language sentences, specifically transcripts of presidential debates"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "triage claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we envision, during presidential debates of u.s. election 2016, for every sentence spoken by the two candidates and extracted into transcripts, our model will immediately pre- dict whether the sentence has a factual claim and whether checking its truthfulness is important to the public. fur- thermore, factual claims will be ranked by their signi cance, which will help professional and citizen journalists focus on the right target."
                    ],
                    "Data Subjects": [
                        "public figures/politicians"
                    ],
                    "Data Actors": [
                        "professional journalists",
                        "citizen journalists"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Application Means": [
                        "triage claims"
                    ]
                }
            ]
        },
        "year": 2015,
        "annotator_2": {
            "narratives": [
                {
                    "Data Subjects": [
                        "public figures/politicians"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "type": "assisted external fact-checking",
                    "Data Actors": [
                        "citizen journalists",
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Application Means": [
                        "triage claims"
                    ]
                },
                {
                    "Data Subjects": [
                        "public figures/politicians"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "triage claims"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "type": "automated external fact-checking",
                    "Data Actors": [
                        "citizen journalists",
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "public figures such as politicians make claims about \u201cfacts\u201d all the time. oftentimes there are false, exaggerated and mis- leading claims on important topics, due to careless mistakes and even deliberate manipulation of information. with tech- nology and modern day media helping spread information to mass audiences through all types of channels, there is a press- ing need for checking the veracity of factual claims important to the public. journalists and citizens spend good amount of time doing that. more and more dedicated platforms and institutes are being created for fact checking."
                    ],
                    "Data Actors": [
                        "professional journalists",
                        "citizen journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (why)": [
                        "according to a census from the duke university reporters\u2019 lab,1 the number of fact checking platforms such as politifact.com and factcheckeu.org has increased from 59 (may 2014) to 89 (january 2015), a 50.8% increase in eight months. this genre of investigative reporting has become a basic feature of political coverage, especially during elections, and plays an important role in improving political discourse and increasing democratic accountability [8, 5]."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "the process of fact checking requires many challenging steps\u2014extracting natural language sentences from speeches, interviews, press releases, campaign brochures and social media; separating factual claims from opinions, beliefs, hyperboles, questions, and so on; detecting topics of factual claims and discerning which are \u201ccheck-worthy\u201d; assessing the veracity of such claims, which itself requires collecting information and data, interviewing experts, and presenting evidence and explanations."
                    ],
                    "Application Means": [
                        "triage claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "part of the goal of computational journalism [3, 4] is use computing to automate fact checking [11, 9]. a fully automatic fact checking system is not yet within our reach. it calls for breakthroughs in several fronts related to the aforementioned fact checking steps. this paper\u2019s focus is on detecting check-worthy factual claims from natural language sentences, specifically transcripts of presidential debates. we model this problem as a classification task and we follow a supervised learning approach to tackle it. we constructed a labeled dataset of spoken sentences by presidential candidates during 2004, 2008 and 2012 presidential debates. (data collection for earlier debates is still in progress.) each sentence is given one of three possible labels\u2014it is not a factual claim; it is an unimportant factual claim; it is an important factual claim."
                    ],
                    "Data Subjects": [
                        "public figures/politicians"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we envision, during presidential debates of u.s. election 2016, for every sentence spoken by the two candidates and extracted into transcripts, our model will immediately pre- dict whether the sentence has a factual claim and whether checking its truthfulness is important to the public."
                    ],
                    "Data Subjects": [
                        "public figures/politicians"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "triage claims"
                    ],
                    "Citation support": [
                        "news article"
                    ]
                }
            ]
        }
    },
    "https://dl.acm.org/doi/pdf/10.1145/3269206.3271709": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "automated content moderation",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Data Actors": [
                        "algorithm"
                    ],
                    "Model Owners": [
                        "social media companies"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "automated removal"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "type": "automated external fact-checking",
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "social media has become a major source for news. it, however, also fostered rapid spread of rumors carrying unverified or even fake information could cause panic or other serious consequences. for instance, during the 2016 u.s. presidential election, twitter became a social medium for candidates and voters to share news and opin- ions."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "anecdote"
                    ]
                },
                {
                    "Quotes (why)": [
                        "recently, rumors on social media have become a major concern. several organizations, such as snopes1 and the community man- agement center of weibo2, aim to debunk rumors emerging on the social media. their judgment are mainly made by editor staffs. however, manually collecting and investigating rumors are quite time-consuming and automatic rumor detection are very useful when dealing with massive real-time rumor posts."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Data Actors": [
                        "algorithm"
                    ],
                    "Model Owners": [
                        "social media companies"
                    ],
                    "Application Means": [
                        "automated removal",
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "automatic rumor detection can be regarded as a binary classifica- tion problem"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [
                {
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Data Actors": [
                        "algorithm"
                    ],
                    "Model Owners": [
                        "social media companies"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "automated removal",
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "type": "automated content moderation"
                },
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "type": "automated external fact-checking",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "social media has become a major source for news. it, however, also fostered rapid spread of rumors carrying unverified or even fake information could cause panic or other serious consequences. for instance, during the 2016 u.s. presidential election, twitter became a social medium for candidates and voters to share news and opin- ions. meantime, it also created a fertile soil for the emergence and propagation of rumors due to its openness. according to a recent work [13], 529 rumors about donald trump and hillary clinton were released during the election, which defamed the candidates and misled the voters with fake stories or malicious comments."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "recently, rumors on social media have become a major concern. several organizations, such as snopes and the community management center of weibo, aim to debunk rumors emerging on the social media. their judgment are mainly made by editor staffs. however, manually collecting and investigating rumors are quite time-consuming and automatic rumor detection are very useful when dealing with massive real-time rumor posts."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Data Actors": [
                        "algorithm"
                    ],
                    "Model Owners": [
                        "social media companies"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "automated removal",
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "this paper proposes a novel hierarchical network with social attention for rumor detection(hsa-blstm). we first model the rumor event as hierarchical time-series containing different semantic levels of information. more specifically, an event is divided into several sub-events containing several posts. each post is further segmented into several words. a structured event is then fed into our hierarchical bi-lstm network. social features are utilized as another clue to identify the prominent part of rumor. we implement these features via the attention mechanism in bi-lstm to obtain an accurate representation for rumor detection."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                },
                {}
            ]
        }
    },
    "https://aclanthology.org/W18-5516.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "automated external fact-checking",
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "in the past years, the amount of false or misleading content on the internet has significantly increased. as a result, information evaluation in terms of fact-checking has become increasingly important as it allows to verify controversial claims stated on the web. however, due to the large number of fake news and hyperpartisan articles published on- line every day, manual fact-checking is no longer feasible. thus, researchers as well as corporations are exploring different techniques to automate the fact-checking process."
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (why)": [
                        "in order to advance research in this direction, the fact extraction and verification (fever) shared task2 was launched."
                    ],
                    "Citation support": [
                        "past work"
                    ]
                },
                {
                    "Quotes (what)": [
                        "the shared task organizers provide a large-scale dataset for the consecutive steps involved in claim verification, in particular, document retrieval, fact extraction, and claim classification."
                    ],
                    "Model Means": [
                        "evidence retrieval",
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores",
                        "gather and present evidence"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "type": "automated external fact-checking",
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "in the past years, the amount of false or misleading content on the internet has significantly increased. as a result, information evaluation in terms of fact-checking has become increasingly important as it allows to verify controversial claims stated on the web. however, due to the large number of fake news and hyperpartisan articles published on- line every day, manual fact-checking is no longer feasible."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (why)": [
                        "thus, researchers as well as corporations are exploring different techniques to automate the fact-checking process. in order to advance research in this direction, the fact extraction and verification (fever) shared task2 was launched."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we describe the pipeline system that we developed to address the fever task. for document retrieval, we implemented an entity linking approach based on constituency parsing and handcrafted rules. for sentence selection, we developed a sentence ranking model based on the enhanced sequential inference model (esim) (chen et al., 2016). we furthermore extended the esim for recognizing textual entailment between multiple input sentences and the claim using an attention mechanism."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores",
                        "gather and present evidence"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/2020.emnlp-main.609.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted media consumption",
                    "Data Actors": [
                        "scientists",
                        "media consumers"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "vague persuasion"
                    ],
                    "Citation Support for Narratives": [
                        "sense of threat"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "due to rapid growth in the scientific literature, it is difficult for researchers \u2013 and the general pub- lic even more so \u2013 to stay up to date on the latest findings. this challenge is especially acute during public health crises like the current covid-19 pan- demic, due to the extremely fast rate at which new findings are reported and the risks associated with making decisions based on outdated or incomplete information. as a result, there is a need for auto- mated tools to assist researchers and the public in evaluating the veracity of scientific claims."
                    ],
                    "Data Actors": [
                        "scientists",
                        "media consumers"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Application Means": [
                        "vague persuasion"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we construct scifact, an expert-annotated dataset of 1,409 scientific claims accompanied by abstracts that support or refute each claim, and annotated with rationales (lei et al., 2016) justifying each supports / refutes deci- sion."
                    ],
                    "Model Means": [
                        "evidence retrieval",
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Data Actors": [
                        "scientists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "type": "assisted media consumption",
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "due to rapid growth in the scientific literature, it is difficult for researchers \u2013 and the general public even more so \u2013 to stay up to date on the latest findings. this challenge is especially acute during public health crises like the current covid-19 pandemic, due to the extremely fast rate at which new findings are reported and the risks associated with making decisions based on outdated or incomplete information. as a result, there is a need for automated tools to assist researchers and the public in evaluating the veracity of scientific claims."
                    ],
                    "Data Actors": [
                        "scientists",
                        "media consumers"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Application Means": [
                        "vague persuasion"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "to facilitate research on this task, we construct scifact, an expert-annotated dataset of 1,409 scientific claims accompanied by abstracts that support or refute each claim, and annotated with rationales (lei et al., 2016) justifying each supports / refutes decision. to create the dataset, we develop a novel annotation protocol in which annotators reformulate naturally occurring claims in the scientific literature \u2013 citation sentences \u2013 into atomic scientific claims."
                    ],
                    "Data Actors": [
                        "scientists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/W18-5501v3.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted knowledge curation",
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "filter system outputs"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "information extraction is a well studied domain and the outputs of such systems enable many nat- ural language technologies such as question an- swering and text summarization. however, since information sources can contain errors, there ex- ists an additional need to verify whether the infor- mation is correct."
                    ],
                    "Application Means": [
                        "filter system outputs"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ]
                },
                {
                    "Quotes (what)": [
                        "his shared task required partic- ipants to develop systems to predict the veracity of human-generated textual claims against textual evidence to be retrieved from wikipedia."
                    ],
                    "Model Means": [
                        "evidence retrieval",
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores",
                        "filter system outputs"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ],
                    "type": "assisted knowledge curation"
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "information extraction is a well studied domain and the outputs of such systems enable many natural language technologies such as question answering and text summarization. however, since information sources can contain errors, there exists an additional need to verify whether the information is correct. for this purpose, we hosted the first fact extraction and verification (fever) shared task to raise interest in and awareness of the task of automatic information verification - a research domain that is orthogonal to information extraction."
                    ],
                    "Application Means": [
                        "filter system outputs"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ],
                    "Citation support": [
                        "none but it's a shared task"
                    ]
                },
                {
                    "Quotes (what)": [
                        "this shared task required participants to develop systems to predict the veracity of human-generated textual claims against textual evidence to be retrieved from wikipedia."
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we constructed a purpose-built dataset for this task (thorne et al., 2018) that contains 185,445 human-generated claims, manually veri- fied against the introductory sections of wikipedia pages and labeled as supported, refuted or notenoughinfo. the claims were generated by paraphrasing facts from wikipedia and mutat- ing them in a variety of ways, some of which were meaning-altering. for each claim, and without the knowledge of where the claim was generated from, annotators selected evidence in the form of sentences from wikipedia to justify the labeling of the claim. for each claim, and without the knowledge of where the claim was generated from, annotators selected evidence in the form of sentences from wikipedia to justify the labeling of the claim."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "filter system outputs"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ]
                }
            ]
        }
    },
    "https://arxiv.org/pdf/1510.05911.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague opposition",
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "misinformation in media and communication creates a situation in which opposing assertions of fact com- pete for attention. this problem is exacerbated in modern, digital society, where people increasingly rely on the aggregate ratings from their social circles for news and information"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "based on the principles underlying link prediction, similarity search and network closure, we computationally gauge the truthfulness of an assertion by mining connectivity patterns within a network of factual statements."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2016,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "type": "vague opposition",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "misinformation in media and communication creates a situation in which opposing assertions of fact compete for attention. this problem is exacerbated in modern, digital society, where people increasingly rely on the aggregate ratings from their social circles for news and information. although much of the information presented on the web is a good resource, its accuracy certainly cannot be guaranteed. in order to avoid being fooled by false assertions, it is necessary to separate fact from fiction and to assess the credibility of an information source."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (why)": [
                        "given a knowledge base that is extracted from a large repository of statements, like wikipedia or the web at large, the resulting knowledge graph represents some of the factual relationships among the entities mentioned in the statements. if there existed an ultimate knowledge graph which knew everything, then fact checking would be as easy as checking for the presence of an edge in the knowledge graph."
                    ],
                    "Citation support": [
                        "none"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in the present work, we present a discriminative path-based method for fact checking in knowledge graphs that incorporates connectivity, type information, and predicate interactions. given a statement s of the form (subject, predicate, object), for example, (chicago, capitalof, illinois), our approach mines discriminative paths that alternatively define the generalized statement (u.s. city, predicate, u.s. state) and uses the mined rules to evaluate the veracity of statement s."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in the present work, we show how to understand a statement by inspecting the related discriminative paths retrieved from the knowledge graph."
                    ],
                    "Quotes (why)": [
                        "unlike existing models, the proposed method simulates how experienced human fact-checkers examine a statement: fact-checkers will first attempt to understand the generalized notion of the statement using prior knowledge, and then validate the specific statement by applying their knowledge. the statement usually can be generalized by replacing the specific entities by their type-labels."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        }
    },
    "https://arxiv.org/pdf/2006.11343.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Owners": [
                        "social media companies",
                        "law enforcement"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the gov- ernment of various countries and mediators of social media platforms have been consistently trying to stop the publish- ing of fake news but the process has been ineffective due to numerous hurdles (tidy 2020)"
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Owners": [
                        "law enforcement",
                        "social media companies"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (why)": [
                        "one of the key issues with detecting misinformation related to covid-19 is that there is a lack of corpus to test methods for fake news detection."
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "a machine learning-based classifier is built to detect the misinformation at the time of the pandemic"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "type": "vague identification",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Model Owners": [
                        "law enforcement"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "people from all over the world are using keywords such as covid19, coronavirus etc. for dis- cussion. so, perhaps for the first time in history, we see that humanity has been exposed to substantial interaction on a single subject in dozens of different languages and on many platforms. in parallel, the \u201dinfodemic\u201d of rumours and mis- information related to the virus came to surface."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (why)": [
                        "the government of various countries and mediators of social media platforms have been consistently trying to stop the publishing of fake news but the process has been ineffective due to numerous hurdles (tidy 2020). one of the key issues with detecting misinformation related to covid-19 is that there is a lack of corpus to test methods for fake news detection."
                    ],
                    "Model Owners": [
                        "law enforcement"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we have presented a human-annotated multilingual cross-domain fact-checked set for the covid-19."
                    ]
                },
                {
                    "Quotes (what)": [
                        "the main contribution of this work is to prepare an open- source data set for detection of misinformation at the time of the pandemic. a machine learning-based classifier is built to detect the misinformation at the time of the pandemic."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/P19-1085.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted knowledge curation",
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "filter system outputs"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "due to the rapid development of information ex- traction (ie), huge volumes of data have been extracted. how to automatically verify the data becomes a vital problem for various data- driven applications, e.g., knowledge graph com- pletion (wang et al., 2017) and open domain question answering (chen et al., 2017a)"
                    ],
                    "Application Means": [
                        "filter system outputs"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ]
                },
                {
                    "Quotes (what)": [
                        "given a claim, an fv system is asked to label it as \u201csupported\u201d, \u201crefuted\u201d, or \u201cnot enough info\u201d, which indicate that the evidence can support, refute, or is not suffi- cient for the claim"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2019,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ],
                    "type": "assisted knowledge curation",
                    "Application Means": [
                        "filter system outputs"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "due to the rapid development of information extraction (ie), huge volumes of data have been extracted. how to automatically verify the data becomes a vital problem for various data- driven applications, e.g., knowledge graph completion (wang et al., 2017) and open domain question answering (chen et al., 2017a)."
                    ],
                    "Application Means": [
                        "filter system outputs"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (what)": [
                        "to integrate and reason over information from multiple pieces of evidence, we propose a graph-based evidence aggregating and reasoning (gear) framework. specifically, we first build a fully-connected evidence graph and encourage information propagation among the evidence. then, we aggregate the pieces of evidence and adopt a classifier to decide whether the evidence can support, refute, or is not sufficient for the claim."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/2020.acl-main.655.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague opposition",
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "online contents with false information, such as fake news, political deception, and online rumors, have been growing significantly and spread widely over the past several years. how to automatically \u201cfact check\u201d the integrity of textual contents, to pre- vent the spread of fake news, and to avoid the un- desired social influences of maliciously fabricated statements, is urgently needed for our society"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "recent research formulates this problem as the fact verification task, which targets to automatically verify the integrity of statements using trustworthy corpora, e.g., wikipedia (thorne et al., 2018a)."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "type": "vague opposition",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "online contents with false information, such as fake news, political deception, and online rumors, have been growing significantly and spread widely over the past several years. how to automatically \u201cfact check\u201d the integrity of textual contents, to pre- vent the spread of fake news, and to avoid the un- desired social influences of maliciously fabricated statements, is urgently needed for our society."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "this paper presents a new neural structural reasoning model, kernel graph attention network (kgat), that provides more fine-grained evidence selection and reasoning capability for fact verification using neural matching kernels (xiong et al., 2017; dai et al., 2018)."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/N18-2004.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "automated external fact-checking",
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval",
                        "justification/explanation production"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "fact checking has recently emerged as an im- portant research topic due to the unprecedented amount of fake news and rumors that are flood- ing the internet in order to manipulate people\u2019s opinions (darwish et al., 2017a; mihaylov et al., 2015a,b; mihaylov and nakov, 2016) or to influ- ence the outcome of major events such as politi- cal elections (lazer et al., 2018; vosoughi et al., 2018). while the number of organizations per- forming fact checking is growing, these efforts cannot keep up with the pace at which false claims are being produced, including also click- bait (karadzhov et al., 2017a), hoaxes (rashkin et al., 2017), and satire (hardalov et al., 2016). hence, there is need for automatic fact checking."
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "a reasonable approach for fact checking a claim involves retrieving potentially relevant documents from different sources (e.g., news websites, social media, etc.), determining the stance of each document with respect to the claim, and finally making a prediction about the claim\u2019s factuality by aggregating the strength of the stances, while taking the relia- bility of the source into account. moreover, a fact checking system should be able to explain its decision by providing relevant extracts (ra- tionales) from the documents"
                    ],
                    "Model Means": [
                        "evidence retrieval",
                        "classify/score veracity",
                        "justification/explanation production"
                    ]
                }
            ]
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [
                {
                    "Application Means": [
                        "gather and present evidence"
                    ],
                    "type": "assisted external fact-checking",
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "type": "automated external fact-checking",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "fact checking has recently emerged as an important research topic due to the unprecedented amount of fake news and rumors that are flooding the internet in order to manipulate people\u2019s opinions (darwish et al., 2017a; mihaylov et al., 2015a,b; mihaylov and nakov, 2016) or to influence the outcome of major events such as political elections (lazer et al., 2018; vosoughi et al., 2018)."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "while the number of organizations performing fact-checking is growing, these efforts cannot keep up with the pace at which false claims are being produced, including also clickbait (karadzhov et al., 2017a), hoaxes (rashkin et al., 2017), and satire (hardalov et al., 2016)."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "hence, there is need for automatic fact checking. while most previous research has focused on english, here we target arabic. moreover, we pro- pose some guidelines, which we believe should be taken into account when designing fact-checking corpora, irrespective of the target language."
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "despite the interdependency between fact checking and stance detection, research on these two problems has not been previously supported by an integrated corpus. this is a gap we aim to bridge by retrieving documents for each claim and annotating them for stance, thus ensuring a natural distribution of the stance labels."
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ]
                },
                {
                    "Quotes (what)": [
                        "a reasonable approach for fact checking a claim involves retrieving potentially relevant documents from different sources (e.g., news websites, social media, etc.), determining the stance of each document with respect to the claim, and finally making a prediction about the claim\u2019s factuality by aggregating the strength of the stances, while taking the reliability of the source into account. moreover, a fact-checking system should be able to explain its decision by providing relevant extracts (rationales) from the documents."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/S17-2083.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Model Means": [
                        "classify/score stance"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (what)": [
                        "being able to detect stance automatically is very useful in the context of events provoking public resonance and associ- ated rumours, as a first step towards verification of early reports (zhao et al., 2015). for instance, it has been shown that rumours that are later proven to be false tend to spark significantly larger num- bers of denying tweets than rumours that are later confirmed to be true (mendoza et al., 2010; proc- ter et al., 2013; derczynski et al., 2014; zubiaga et al., 2016b)."
                    ],
                    "Model Means": [
                        "classify/score stance"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                }
            ]
        },
        "year": 2017,
        "annotator_2": {
            "narratives": [
                {
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "type": "vague identification",
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "stance classification is considered to be an important step towards rumour verification, therefore performing well in this task is expected to be useful in debunking false rumours."
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "none"
                    ]
                },
                {
                    "Quotes (what)": [
                        "here we focus on stance classification of tweets towards the truthfulness of rumours circulating in twitter conversations in the context of breaking news."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                },
                {}
            ]
        }
    },
    "https://arxiv.org/pdf/1708.07239.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Model Means": [
                        "classify/score veracity",
                        "justification/explanation production"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "misinformation, unverified rumors, hoaxes, and lies have\r become rampant on the internet nowadays, primarily due to\r the ability to quickly disseminate information at a large scale\r through the web and social media. this phenomenon has led to\r many ill effects and, according to experts, poses a severe threat\r to society at large [1]. to address these problems, numerous\r approaches have been designed to study and mitigate the\r effects of misinformation spread (see zubiaga et al. [2])"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers",
                        "past work"
                    ]
                },
                {
                    "Quotes (what)": [
                        "most strategies rely on contextual indicators of rumors (e.g., number of inquiring tweets, reporting dynamics during breaking news, temporal patterns, or source credibility) for their detection and veracity assessment."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "our approach not only delivers performance comparable\r to state-of-the-art methods like predpath; it also produces\r more meaningful explanations of its predictions."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "justification/explanation production"
                    ]
                }
            ]
        },
        "year": 2017,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "justification/explanation production"
                    ],
                    "type": "vague identification",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "misinformation, unverified rumors, hoaxes, and lies have become rampant on the internet nowadays, primarily due to the ability to quickly disseminate information at a large scale through the web and social media. this phenomenon has led to many ill effects and, according to experts, poses a severe threat to society at large [1]."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "most strategies rely on contextual indicators of rumors (e.g., number of inquiring tweets, reporting dynamics during breaking news, temporal patterns, or source credibility) for their detection and veracity assessment. to go beyond contextual approaches one would need to assess the truthfulness of claims by reasoning about their content and related facts. moreover, a fact-checking system would ideally need to operate in near real-time, to match the rate at which false or misleading claims are made."
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper we propose knowledge stream (ks), an unsupervised approach for fact-checking triples based on the idea of treating a kg as a flow network."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "our approach not only delivers performance comparable to state-of-the-art methods like predpath; it also produces more meaningful explanations of its predictions."
                    ],
                    "Model Means": [
                        "justification/explanation production"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/C08-1006.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "truth-telling for law enforcement",
                    "Data Actors": [
                        "law enforcement"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "Ends": [
                        "detect falsehood for law enforcement"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the ability to detect deceptive statements in text\r and speech has broad applications in law enforcement and intelligence gathering"
                    ],
                    "Data Actors": [
                        "law enforcement"
                    ],
                    "Ends": [
                        "detect falsehood for law enforcement"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (why)": [
                        "reviews by shuy (1998), vrij (2000), and depaulo et al. (2003) indicate that many types of deception can be identified because the liar\u2019s verbal and non-verbal behavior varies considerably from that of the truth teller\u2019s. even so, the literature reports that human lie detectors rarely perform at a level above chance. vrij (2000) gives a summary of 39 studies of human ability to detect lies. the majority of the studies report accuracy rates between 45-60%, with the mean accuracy rate at 56.6%. the goal of our research is to develop and\r implement a system for automatically identifying\r deceptive and truthful statements in narratives\r and transcribed interviews"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we focus exclusively\r on verbal cues to deception for this initial\r experiment, ignoring at present potential\r prosodic cues (but see hirschberg et al.)."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2008,
        "annotator_2": {
            "narratives": [
                {
                    "Data Subjects": [
                        "suspects"
                    ],
                    "Data Actors": [
                        "law enforcement"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "Ends": [
                        "detect falsehood for law enforcement"
                    ],
                    "type": "truth-telling for law enforcement"
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the ability to detect deceptive statements in text and speech has broad applications in law enforcement and intelligence gathering. the scientific study of deception in language dates at least from undeutsch (1954, 1989), who hypothesized that it is \u201cnot the veracity of the reporting person but the truthfulness of the statement that matters and there are certain relatively exact, definable, descriptive criteria that form a key tool for the determination of the truthfulness of statements\u201d. reviews by shuy (1998), vrij (2000), and depaulo et al. (2003) indicate that many types of deception can be identified because the liar\u2019s verbal and non-verbal behavior varies considerably from that of the truth teller\u2019s. even so, the literature reports that human lie detectors rarely perform at a level above chance. vrij (2000) gives a summary of 39 studies of human ability to detect lies. the majority of the studies report accuracy rates between 45-60%, with the mean accuracy rate at 56.6%."
                    ],
                    "Ends": [
                        "detect falsehood for law enforcement"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (what)": [
                        "the goal of our research is to develop and implement a system for automatically identifying deceptive and truthful statements in narratives and transcribed interviews. we focus exclusively on verbal cues to deception for this initial experiment, ignoring at present potential prosodic cues (but see hirschberg et al.)."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "Ends": [
                        "detect falsehood for law enforcement"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we describe a language-based analysis of deception that we have constructed and tested using \u201creal world\u201d sources\u2014criminal narratives, police interrogations and legal testimony."
                    ],
                    "Data Subjects": [
                        "suspects"
                    ],
                    "Data Actors": [
                        "law enforcement"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "Ends": [
                        "detect falsehood for law enforcement"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/W18-5515.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "automated external fact-checking",
                    "Data Actors": [
                        "professional journalists",
                        "media consumers"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Citation Support for Narratives": [
                        "sense of threat"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "we often hear the word \u201cfake news\u201d these days. recently, russian meddling, for example, has been blamed for the prevalence of inaccurate news stories on social media,1 but even the reporting on this topic often turns out to be fake news (uberti, 2016). an abundance of incorrect information can plant wrong beliefs in individual citizens and lead to a misinformed public, undermining the democratic process"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "anecdote"
                    ]
                },
                {
                    "Quotes (why)": [
                        "in this context, technology to automate fact-checking and source verification (vlachos and riedel, 2014) is of great interest to both media consumers and publishers."
                    ],
                    "Data Actors": [
                        "media consumers",
                        "professional journalists"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Citation support": [
                        "past work"
                    ]
                },
                {
                    "Quotes (what)": [
                        "systems are evaluated on the proportion of claims\r for which both the predicted label is correct and a complete set of relevant evidence sentences has\r been identified."
                    ],
                    "Model Means": [
                        "evidence retrieval",
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "type": "assisted external fact-checking"
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "we often hear the word \u201cfake news\u201d these days. recently, russian meddling, for example, has been blamed for the prevalence of inaccurate news stories on social media, but even the reporting on this topic often turns out to be fake news (uberti, 2016). an abundance of incorrect information can plant wrong beliefs in individual citizens and lead to a misinformed public, undermining the democratic process. in this context, technology to automate fact-checking and source verification (vlachos and riedel, 2014) is of great interest to both media consumers and publishers."
                    ],
                    "Data Actors": [
                        "media consumers",
                        "publishers"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "anecdote (news article)"
                    ]
                },
                {
                    "Quotes (what)": [
                        "the original dataset description paper (thorne et al., 2018) evaluates a simple baseline system that achieves a score of \u223c33% on this metric, using tf-idf based retrieval to find the relevant evidence and a natural language inference (nli) model to classify the relation between the returned evidence and the claim. our system attempts to improve on this baseline by addressing two major weaknesses."
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (what)": [
                        "our system is a four-stage model consisting of document retrieval, sentence retrieval, natural language inference and aggregation."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/2020.acl-main.549.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague debunking",
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "Ends": [
                        "limit ai-generated misinformation",
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "malicious people spread false news, which may have significant influence on public opinions, stock prices, even presidential elections (faris et al., 2017). vosoughi et al. (2018) show that false news reaches more people than the truth"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (why)": [
                        "the situation is more urgent as advanced pre-trained language models (radford et al., 2019) can produce remarkably coherent and fluent texts, which lowers the barrier for the abuse of creating deceptive content."
                    ],
                    "Ends": [
                        "limit ai-generated misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (why)": [
                        "in this paper, we study fact checking with the goal of automatically assessing the truthfulness of a textual claim by looking for textual evidence."
                    ],
                    "Model Means": [
                        "evidence retrieval",
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "type": "vague debunking",
                    "Ends": [
                        "limit ai-generated misinformation",
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "internet provides an efficient way for individuals and organizations to quickly spread information to massive audiences. however, malicious people spread false news, which may have significant influence on public opinions, stock prices, even presidential elections (faris et al., 2017). vosoughi et al. (2018) show that false news reaches more people than the truth."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "the situation is more urgent as advanced pre-trained language models (radford et al., 2019) can produce remarkably coherent and fluent texts, which lowers the barrier for the abuse of creating deceptive content."
                    ],
                    "Ends": [
                        "limit ai-generated misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we study fact checking with the goal of automatically assessing the truthfulness of a textual claim by looking for textual evidence."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ]
                },
                {
                    "Quotes (what)": [
                        "with a given claim, we represent the retrieved evidence sentences as a graph, and then use the graph structure to guide the reasoning process. specifically, we apply semantic role labeling (srl) to parse each evidence sentence, and establish links between arguments to construct the graph."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/2020.lrec-1.755.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the dissemination and consequences of fake news are exacerbating due to the rise of popular social media applications and other online sources with inadequate factchecking or third-party filtering, enabling any individual to broadcast fake news easily and at a large scale (allcott and gentzkow, 2017). though steps have been taken to detect and eliminate fake news, it still poses a dire threat to society (dreyfuss and lapowsky, 2019)."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (why)": [
                        "as such, research in the area of fake news detection is of high importance for society"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "the dataset we propose: fakeddit3\r , a novel\r multimodal fake news detection dataset consisting of over\r 1 million samples with 2-way, 3-way, and 6-way classification labels, along with comment data and metadata"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "type": "vague identification",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "within our progressively digitized society, the spread of fake news and disinformation has enlarged in journalism, news reporting, social media, and other forms of online information consumption. false information from these sources, in turn, has caused many problems such as spurring irrational fears during medical outbreaks like ebola"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "news article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "the dissemination and consequences of fake news are exacerbating due to the rise of popular social media applications and other online sources with inadequate fact- checking or third-party filtering, enabling any individual to broadcast fake news easily and at a large scale (allcott and gentzkow, 2017)."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat",
                        "scientific article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "though steps have been taken to detect and eliminate fake news, it still poses a dire threat to society (dreyfuss and lapowsky, 2019). according to a pew research center report2, 50% of americans view fake news as a critical problem, placing it above violent crime. in ad- dition, the report found that 68% of americans view fake news as having a significant impact on their confidence of the government and 54% viewed it as having a large impact in their trust in one another."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article",
                        "news article"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we overcome these limitations posed by conventional datasets through the dataset we propose: fakeddit3, a novel multimodal fake news detection dataset consisting of over 1 million samples with 2-way, 3-way, and 6-way classification labels, along with comment data and metadata."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                }
            ]
        }
    },
    "https://www.acl-bg.org/proceedings/2017/RANLP%202017/pdf/RANLP037.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted external fact-checking",
                    "Data Subjects": [
                        "social media users",
                        "public figures/politicians",
                        "professional journalists"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "human in the loop"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the current coverage of the political landscape in the press and in social media has led to an unprecedented situation. like never before, a statement in an interview, a press release, a blog note, or a tweet can spread almost instantaneously and reach the public in no time. this proliferation speed has left little time for double-checking claims against the facts, which has proven critical in politics, e.g., during the 2016 presidential campaign in the usa, which was arguably impacted by fake news in social media and by false claims."
                    ],
                    "Data Subjects": [
                        "professional journalists",
                        "social media users",
                        "public figures/politicians"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat",
                        "anecdote"
                    ]
                },
                {
                    "Quotes (why)": [
                        "manual fact-checking has proven very timeconsuming, and thus automatic methods have been proposed as a way to speed-up the process"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ]
                },
                {
                    "Quotes (why)": [
                        "first, an intrinsic analysis is carried out in which check-worthy text fragments are identified. then, other documents that might support or rebut a claim in the document are retrieved from various sources. finally, by comparing a claim against the retrieved evidence, a system can determine whether the claim is likely true or likely false. for instance, ciampaglia et al. (2015) do this on the basis of a knowledge graph derived from wikipedia. the outcome could then be presented to a human expert for final judgment"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we focus on the first step: predicting\r check-worthiness of claims"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2017,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "type": "vague identification",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the current coverage of the political landscape in the press and in social media has led to an unprecedented situation. like never before, a statement in an interview, a press release, a blog note, or a tweet can spread almost instantaneously and reach the public in no time. this proliferation speed has left little time for double-checking claims against the facts, which has proven critical in politics, e.g., during the 2016 presidential campaign in the usa, which was arguably impacted by fake news in social media and by false claims."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat",
                        "anecdote"
                    ]
                },
                {
                    "Quotes (why)": [
                        "investigative journalists and volunteers have been working hard trying to get to the root of a claim and to present solid evidence in favor or against it. manual fact-checking has proven very time- consuming, and thus automatic methods have been proposed as a way to speed-up the process. for in- stance, there has been work on checking the factu- ality/credibility of a claim, of a news article, or of an information source (castillo et al., 2011; ba et al., 2016; zubiaga et al., 2016; ma et al., 2016; hardalov et al., 2016; karadzhov et al., 2017a,b; nakov et al., 2017)."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "the process starts when a document is made public. first, an intrinsic analysis is carried out in which check-worthy text fragments are identified."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we focus on the first step: predicting check-worthiness of claims. our contributions can be summarized as follows: 1. new dataset: we build a new dataset of manually-annotated claims, extracted from the 2016 us presidential and vice- presidential debates, which we gathered from nine reputable sources such as cnn, npr, and politifact, and which we release to the research community. 2. modeling the context: we develop a novel approach for automatically predicting which claims should be prioritized for fact- checking, based on a rich input representa- tion."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/S17-2082.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted external fact-checking",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "human in the loop"
                    ],
                    "Application Means": [
                        "vague persuasion"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "over the past 15 years, and in a gradual manner, social media has started to become a main source of news. however, social media has also become a ripe ground for rumours, spreading them in a matter of a few minutes."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (why)": [
                        "false rumours may greatly affect the social, economic and political stability of any society around the world, hence the need for tools to help people, especially journalists, analyze the spread of rumours and their effect on the society as well as determine their veracity"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Application Means": [
                        "vague persuasion"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "our participation was in the closed task variant, in which the prediction is made solely from the tweet itself. for subtask a, linear support vector classification was applied to a model of bag of words, and the help of a na\u00efve bayes classifier was used for semantic feature extraction. for subtask b, a similar approach was used."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2017,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores",
                        "vague persuasion"
                    ],
                    "type": "vague identification",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores",
                        "vague persuasion"
                    ],
                    "type": "vague debunking",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "over the past 15 years, and in a gradual manner, social media has started to become a main source of news. however, social media has also become a ripe ground for rumours, spreading them in a matter of a few minutes."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "this paper presents the results and conclusions of our participation in semeval-2017 task 8: determining rumour veracity and support for rumours. we have participated in 2 subtasks: sdqc (subtask a) which deals with tracking how tweets orient to the accuracy of a rumourous story, and veracity prediction (subtask b) which deals with the goal of predicting the veracity of a given rumour."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores",
                        "vague persuasion"
                    ]
                }
            ]
        }
    },
    "https://aaai.org/ocs/index.php/ICWSM/ICWSM18/paper/view/17825/17046": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores",
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "as the internet becomes an ever-increasing presence in the life of the average person, more and more obtain their news from facebook and other forms of social media (gottfried and shearer 2016). since this dissemination of news content is by and large unsupervised and often strictly user generated, quality control has become a pressing concern"
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (why)": [
                        "since shutting down the production of all outlets that produce such content would be nearly impossible, the main method being explored for the systematic squelching of this content is detection. an algorithm that would take a news article and its associated features and assign a veracity score would prove a potent weapon in combating misinformation online"
                    ],
                    "Application Means": [
                        "identify claims",
                        "provide labels/veracity scores"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ]
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [
                {
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "type": "vague identification",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "clearly, misinformation on the internet is not a new problem, as fact-checking websites such as snopes have existed since at least 1994. what is new is this meteoric rise in social media which has made it easier than ever before for organizations to produce and spread news content of questionable validity to massive audiences (chen, con- roy, and rubin 2015). the spread of intentional misinformation through online forums during the 2016 brexit vote and u.s. presidential election (howard and kollanyi 2016; howard, kollanyi, and woolley 2016) have put the spotlight on what has recently been dubbed \u201cfake news\u201d. facebook and other social media corporations have made attempts to counter this manufacture of misleading news content, but it has only become more prominent (weedon, nuland, and stamos 2017)."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific articles"
                    ]
                },
                {
                    "Quotes (why)": [
                        "since shutting down the production of all outlets that produce such content would be nearly impossible, the main method being explored for the systematic squelching of this content is detection. an algorithm that would take a news article and its associated features and assign a veracity score would prove a potent weapon in combating misinformation online."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ]
                },
                {
                    "Quotes (why)": [
                        "an additional area of concern with respect to the propagation of information on social media platforms is social bots, which are often the means by which questionable news content is spread (ferrara et al. 2016). social bots are auto-mated users of social media platforms which promote specific ideologies. it is widely thought that the misinforma- tion campaigns associated with the 2016 u.s. presidential election and the brexit vote were enacted by large num- bers of coordinated social bots (howard and kollanyi 2016; howard, kollanyi, and woolley 2016)."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "the data set that we detail here \u2014 buzzface \u2014 will be particularly interesting to those that wish to study the social bots of facebook."
                    ]
                },
                {
                    "Quotes (what)": [
                        "the initial dataset created by buzzfeed was formed using a sample of news articles posted to facebook by a select group of news outlets during a specific time period (sil- verman et al. 2016). each article was read and analyzed for veracity, and given a categorization. while it is in and of itself a useful dataset, we have identified this opportunity for its enrichment via the features that come along with social media posts. as the articles were all posted to facebook, the facebook graph api allows for the collection of data on \u201creactions\u201d, facebook comments, and various metadata. additionally, since many of these facebook posts link to outside web pages, content acquisition can be performed to obtain the article text along with images, links, and embed- ded tweets associated with each article. finally, many news outlets allow for additional commentary on the actual article web pages themselves\u2014using either or both of the facebook comments and disqus comments system plugins. this allows for up to two additional sources of commentary associated with each article."
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/2020.acl-main.656.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "automated external fact-checking",
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "justification/explanation production"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "when a potentially viral news item is rapidly or indiscriminately published by a news outlet, the responsibility of verifying the truthfulness of the item is often passed on to the audience. to alleviate this problem, independent teams of professional fact checkers manually verify the veracity and credibility of common or particularly checkworthy statements circulating the web. however, these teams have limited resources to perform manual fact checks, thus creating a need for automating the fact checking process."
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we present the first study on generating veracity explanations, showing that they can successfully describe the reasons behind a veracity prediction."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "justification/explanation production"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity",
                        "justification/explanation production"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "type": "automated external fact-checking",
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "when a potentially viral news item is rapidly or indiscriminately published by a news outlet, the responsibility of verifying the truthfulness of the item is often passed on to the audience. to alleviate this problem, independent teams of professional fact-checkers manually verify the veracity and credibility of common or particularly check-worthy statements circulating the web. however, these teams have limited resources to perform manual fact-checks, thus creating a need for automating the fact-checking process."
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (why)": [
                        "a prevalent component of existing fact-checking systems is a stance detection or textual entailment model that predicts whether a piece of evidence contradicts or supports a claim (ma et al., 2018; mohtarami et al., 2018; xu et al., 2018). existing research, however, rarely attempts to directly optimise the selection of relevant evidence, i.e., the self-sufficient explanation for predicting the veracity label (thorne et al., 2018; stammbach and neumann, 2019). on the other hand, alhindi et al. (2018) have reported a significant performance improvement of over 10% macro f1 score when the system is provided with a short human explanation of the veracity label. still, there are no attempts at automatically producing explanations, and automating the most elaborate part of the process - producing the justification for the veracity prediction - is an understudied problem"
                    ]
                },
                {
                    "Quotes (what)": [
                        "inspired by this, we research how to generate explanations for veracity prediction. we frame this as a summarisation task, where, provided with elaborate fact checking reports, later referred to as ruling comments, the model has to generate veracity explanations close to the human justifications as in the example in table 1. we then explore the bene- fits of training a joint model that learns to generate veracity explanations while also predicting the ve- racity of a claim. in summary, our contributions are as follows: 1. we present the first study on generating veracity explanations, showing that they can successfully describe the reasons behind a veracity prediction."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "justification/explanation production"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/W18-5513.pdf": {
        "annotator_1": {
            "narratives": [],
            "quotes": []
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity",
                        "justification/explanation production"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores",
                        "gather and present evidence"
                    ],
                    "type": "vague opposition"
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "until recently, the bottleneck for developing au- tomatic methods for fact-checking has been the lack of large datasets for building machine learn- ing models. thorne and vlachos (2018) provide a survey of current datasets and models for fact- checking (e.g., (wang, 2017; rashkin et al., 2017; vlachos and riedel, 2014; thorne et al., 2018; long et al., 2017; potthast et al., 2018; wang et al., 2018)). wang (2017) has introduced a large dataset (liar) of claims from politifact, the associated metadata for each claim and the ver- dict (6 class labels). most work on the liar dataset has focused on modeling the content of the claim (including hedging, sentiment and emotion analysis) and the speaker-related metadata (wang, 2017; rashkin et al., 2017; long et al., 2017). however, these approaches do not use the evidence and the justification provided by humans to predict the label. extracting evidence from (trusted) sources for fact-checking or for argument mining is a difficult task (rinott et al., 2015; thorne et al., 2018; baly et al., 2018)."
                    ],
                    "Model Means": [
                        "justification/explanation production"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (what)": [
                        "for the purpose of our paper, we rely on the fact-checking article associated with the claim. we extend the original liar dataset by automatically extract- ing the justification given by humans for labeling the claim, from the fact-checking article (section 2). we release the extended liar dataset (liar- plus) to the community1."
                    ]
                },
                {
                    "Quotes (what)": [
                        "we are primarily concerned on showing the impact of modeling the human-provided justification for predicting the veracity of a claim. in addition, our task aims to capture the varying degrees of truth that some claims might have and that are usually labeled as such by professionals (rather than binary true vs. false labels)."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                }
            ]
        }
    },
    "https://arxiv.org/pdf/2001.10667.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the spread of fake news can have far reaching and devastating effects. $130 billion in stock value was wiped out in minutes after a false associated press\u2019s tweet claimed that barack obama was injured following an explosion in 2013 (rapoza 2017). some researchers even believe that fake news has affected the outcome of the 2016 united states presidential election (gunther, beck, and nisbet 2018)"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (why)": [
                        "the\r severity of the impact of fake news warrants the need for an\r effective and automated means of detecting fake news and\r has hence spurred much research in this area."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "this area of research\r exploits the collective wisdom of the community by applying natural language processing to comments directed\r towards a claim (see figure 1). the key principle behind such works is that users on social media would share opinions, conjectures and evidences for inaccurate information"
                    ],
                    "Data Subjects": [
                        "social media users"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores",
                        "identify claims"
                    ],
                    "type": "vague identification",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the spread of fake news can have far reaching and devastating effects. $130 billion in stock value was wiped out in minutes after a false associated press\u2019s tweet claimed that barack obama was injured following an explosion in 2013 (rapoza 2017). some researchers even believe that fake news has affected the outcome of the 2016 united states presidential election (gunther, beck, and nisbet 2018). the severity of the impact of fake news warrants the need for an effective and automated means of detecting fake news and has hence spurred much research in this area."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "anecdotal",
                        "scientific article"
                    ]
                },
                {
                    "Quotes (what)": [
                        "our work focuses on the detection of fake claims using community response to such claims. this area of research exploits the collective wisdom of the community by applying natural language processing to comments directed towards a claim (see figure 1). the key principle behind such works is that users on social media would share opinions, conjectures and evidences for inaccurate information."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (why)": [
                        "a user debunking a fake news may not be directed solely at the person he is replying to - the content created could also be applicable to other tweets in the thread."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we utilize the attention weights from our model to provide both token-level and post-level explanations behind the model\u2019s prediction. to the best of our knowledge, we are the first paper that has done this."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                }
            ]
        }
    },
    "https://arxiv.org/pdf/1809.08193.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted external fact-checking",
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "human in the loop"
                    ],
                    "Application Means": [
                        "triage claims"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research",
                        "automation"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "misinformation has recently become more central in public discourse [6, 37, 46]. as a consequence, interest has increased in the scientific community to further natural language processing (nlp) approaches that can help alleviate the burdensome and time-consuming human activity of factchecking [38, 40]. factchecking is known as the task of producing an informed assessment of the veracity of a claim [13, 14]. the main mission of factcheckers is to give citizens information to make political choices, improve the quality of public political discourse and to hold politicians accountable [14]. misinformation and misperceptions can undermine this goal [9, 11]."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "the speed at which information flows online means there is less time to verify the claims made and myths spread further before being factchecked, if they are factchecked at all. automating any parts of the factchecking process could cut down the time it takes to respond to a claim. it could also protect human factchecker\u2019s time to work on the more complicated checks that need careful human judgement"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ]
                },
                {
                    "Quotes (what)": [
                        "monitoring and spotting within a factchecking organisation is a time consuming semi-manual task, which is inevitably limited by the resources available. automating the process of claim detection could mean that factcheckers can monitor a greater set of media, extract the claims made, and hopefully make smarter choices about what the most valuable items to be checking that day are"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "human in the loop"
                    ],
                    "Application Means": [
                        "triage claims"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Application Means": [
                        "triage claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "type": "assisted external fact-checking"
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "misinformation has recently become more central in public discourse [6, 37, 46]. as a consequence, interest has increased in the scientific community to further natural language processing (nlp) approaches that can help alleviate the burdensome and time-consuming human activity of factchecking [38, 40]."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "the main mission of factcheckers is to give citizens information to make political choices, improve the quality of public political discourse and to hold politicians accountable [14]. misinformation and misperceptions can undermine this goal [9, 11]."
                    ]
                },
                {
                    "Quotes (why)": [
                        "however, there is a very small number of factchecking organisations in the world, about 160,1 compared to the volume of media items produced daily. the speed at which information flows online means there is less time to verify the claims made and myths spread further before being factchecked, if they are factchecked at all. automating any parts of the factchecking process could cut down the time it takes to respond to a claim. it could also protect human factchecker\u2019s time to work on the more complicated checks that need careful human judgement."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "triage claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "automating the process of claim detection could mean that factcheckers can monitor a greater set of media, extract the claims made, and hopefully make smarter choices about what the most valuable items to be checking that day are."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims",
                        "supplant human fact-checkers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "if deployed in a live factchecking situation, it could also help separate out claims made in real-time during a ministerial speech, for example. this could help factcheckers quickly skim transcripts when time is limited."
                    ],
                    "Application Means": [
                        "triage claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ]
        }
    },
    "https://aaai.org/ocs/index.php/ICWSM/ICWSM18/paper/view/17796/17044": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted media consumption",
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Data Actors": [
                        "scientists"
                    ],
                    "Model Means": [
                        "corpora analysis"
                    ],
                    "Citation Support for Narratives": [
                        "investigative"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "type": "scientific curiosity",
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Data Actors": [
                        "scientists"
                    ],
                    "Model Means": [
                        "corpora analysis"
                    ],
                    "Application Means": [
                        "analyse data"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "Ends": [
                        "develop knowledge of nlp/language"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "as news producers and distributors can be established quickly with relatively little effort, there is limited prior data on the reliability of some of many sources, even though the information they provide can end up being widely disseminated due to algorithmic and social filtering in social media."
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (why)": [
                        "as a result, there is a great deal of early research in automatically identifying different writing styles and persuasion techniques employed by news sources (popat et al. 2016) (potthast et al. 2017) (horne and adal\u0131 2017) (chakraborty et al. 2016) (singhania, fernandez, and rao 2017)."
                    ],
                    "Model Means": [
                        "corpora analysis"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "while much of recent research has focused on automatic news characterization methods, there are many other news publishing behaviors that are not well-studied."
                    ],
                    "Application Means": [
                        "analyse data"
                    ]
                },
                {
                    "Quotes (what)": [
                        "these open research problems are the primary reasons we have created the nela2017 data set."
                    ],
                    "Data Actors": [
                        "scientists"
                    ],
                    "Ends": [
                        "develop knowledge of nlp/language"
                    ]
                }
            ]
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [
                {
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Application Means": [
                        "analyse data"
                    ],
                    "Ends": [
                        "limit misinformation",
                        "develop knowledge of nlp/language"
                    ],
                    "Citation Support for Narratives": [
                        "investigative"
                    ],
                    "type": "scientific curiosity"
                },
                {
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Data Actors": [
                        "media consumers"
                    ],
                    "Model Means": [
                        "corpora analysis"
                    ],
                    "Application Means": [
                        "analyse data"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "type": "assisted media consumption"
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the complexity and diversity of today\u2019s media landscape provides many challenges for researchers studying news producers. these producers use many different strategies to get their message believed by readers through the writing styles they employ, by repetition across different media sources with or without attribution, as well as other mechanisms that are yet to be studied deeply."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we introduce a broad news benchmark data set, called the news landscape (nela2017) data set, to facilitate the study of many problems in this domain. the data set includes articles on u.s. politics from a wide range of news sources that includes well-established news sources, satire news sources, hyper-partisan sources (from both ends of the political spectrum), as well as, sources that have been known to distribute maliciously fake information. at the time of writing, this data set contains 136k news articles from 92 sources between april 2017 and october 2017."
                    ],
                    "Data Actors": [
                        "scientists"
                    ],
                    "Ends": [
                        "develop knowledge of nlp/language"
                    ]
                },
                {
                    "Quotes (why)": [
                        "it has been argued that the traditionally slow fact-checking process and journalistically trained \u201cgatekeepers\u201dare insufficient to counteract the potentially damaging effect these sources have on the public (mele et al. 2017) (buntain and golbeck 2017). as a result, there is a great deal of early research in automatically identifying different writing styles and persuasion techniques employed by news sources (popat et al. 2016) (potthast et al. 2017) (horne and adal\u0131 2017) (chakraborty et al. 2016) (singhania, fernandez, and rao 2017). hence, a broad data set including many different types of sources is especially useful in further refining these methods."
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Data Actors": [
                        "media consumers"
                    ],
                    "Model Means": [
                        "corpora analysis"
                    ],
                    "Application Means": [
                        "analyse data"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "while misinformation in news has attracted a lot of interest lately, it is important to note that many sources mix true and false information in strategic ways to not only to distribute false information, but also to create mistrust for other sources. this mistrust and uncertainty may be accomplished by writing specific narratives and having other similar sources copy that information verbatim (lytvynenko 2017). in some cases, sources may copy information with the intention to misrepresent it and undermine its reliability. in other cases, a source may copy information to gain credibility itself. similarly, the coverage of topics in sources can be highly selective or may include well-known conspiracy theories. hence, it may be important to study a source\u2019s output over time and compare it to other sources publishing news in the same time frame. this can sometimes be challenging as sources are known to remove articles that attract unwanted attention. we have observed this behavior with many highly shared false articles during the 2016 u.s. election."
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Application Means": [
                        "analyse data"
                    ],
                    "Ends": [
                        "develop knowledge of nlp/language"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/2021.findings-emnlp.56.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague opposition",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "human in the loop"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "unfortunately, the democratic nature of social media has raised questions about the quality and the factuality of the information that is shared on these platforms. eventually, social media have become one of the main channels to spread disinformation."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "for the tweets in figure 1, it is necessary to un- derstand whether the information is correct, harm- ful, calling for action to be taken by relevant author- ities, etc. rapidly sorting these questions is crucial to help organizations channel their efforts, and to counter the spread of disinformation, which may cause panic, mistrust, and other problems"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we focused on three key aspects, which are formulated into seven questions: (i) check worthiness and veracity of the tweet (q1- 4 and q5). (ii) harmfulness to society (q6); and (iii) call for action addressing a government / policy makers (q7)."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2021,
        "annotator_2": {
            "narratives": [
                {
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "human in the loop"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "type": "vague opposition",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Application Means": [
                        "analyse data"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "unfortunately, the democratic nature of social media has raised questions about the quality and the factuality of the information that is shared on these platforms. eventually, social media have become one of the main channels to spread disinformation."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (why)": [
                        "for the tweets in figure 1, it is necessary to understand whether the information is correct, harmful, calling for action to be taken by relevant author- ities, etc. rapidly sorting these questions is crucial to help organizations channel their efforts, and to counter the spread of disinformation, which may cause panic, mistrust, and other problems."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "with this consideration in mind, we adopt a multifaceted approach, which is motivated by engaging with different stakeholders such as journalists and policy makers. we focused on three key aspects, which are formulated into seven questions: (i) check worthiness and veracity of the tweet (q1- 4 and q5). (ii) harmfulness to society (q6); and (iii) call for action addressing a government / policy makers (q7). q1\u2013q5 were motivated by conversations with journalists and professional fact-checkers, while q6-q7 were formulated in conversations with a ministry of public health."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "human in the loop"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we develop a large manually annotated dataset of 16k tweets related to the covid- 19 infodemic in four languages (arabic, bulgarian, dutch, and english), using a schema that combines the perspective of journalists, fact-checkers, social media platforms, policy-makers, and the society."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Application Means": [
                        "analyse data"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/D18-1010.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted knowledge curation",
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "filter system outputs"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "this problem has broad applications. for example, knowledge bases (kb), such as freebase (bollacker et al., 2008), yago (suchanek et al., 2007), can be augmented with a new relational statement such as \u201c(afghanistan, is source of, kushan dynasty)\u201d. this needs to be first verified by a claim verification process and supported by evidence (roth et al., 2009; chaganty et al., 2017)."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "filter system outputs"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (why)": [
                        "claim verification is a key component in any technical solution addressing recent concerns about the trustworthiness of online content (vydiswaran et al., 2011; pasternack and roth, 2013; hovy et al., 2013)."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                }
            ]
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "filter system outputs"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "type": "assisted knowledge curation"
                }
            ],
            "quotes": [
                {
                    "Quotes (what)": [
                        "specifically, given a claim and a text corpus, evidential claim verification, demonstrated in  figure 1, aims at identifying text snippets in the corpus that act as evidence that supports or refutes the claim"
                    ]
                },
                {
                    "Quotes (why)": [
                        "this problem has broad applications. for example, knowledge bases (kb), such as freebase (bol- lacker et al., 2008), yago (suchanek et al., 2007), can be augmented with a new relational statement such as \u201c(afghanistan, is source of, kushan dynasty)\u201d. this needs to be first verified by a claim verification process and supported by evidence (roth et al., 2009; chaganty et al., 2017). more broadly, claim verification is a key component in any technical solution addressing recent concerns about the trustworthiness of online content (vydiswaran et al., 2011; pasternack and roth, 2013; hovy et al., 2013). in both scenarios, we care about whether or not a claim holds, and seek reliable evidence in support of this decision."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "filter system outputs"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (why)": [
                        "the discussion above suggests that claim verification and evidence identification are tightly cou- pled. claim should influence the identification of appropriate evidence, and \u201ctrusted evidence boosts the claim\u2019s veracity\u201d (vydiswaran et al., 2011)."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ]
                },
                {
                    "Quotes (what)": [
                        "our framework, twowingos, optimizes the two subtasks jointly, so that both claim verification and evidence identification can enhance each other. twowingos is a generic framework making use of a shared representation of the claim to co-train evidence identification and claim verification."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "filter system outputs"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ]
                }
            ]
        }
    },
    "https://arxiv.org/pdf/1803.03178.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted media consumption",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Data Actors": [
                        "media consumers"
                    ],
                    "Model Owners": [
                        "social media companies"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "community question answering (cqa) forums such as stackoverflow, yahoo! answers, and quora are very popular nowadays, as they represent effective means for communities around particular topics to share information and to collectively satisfy their information needs. however, the information being shared is not always factual."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we explore a new dimension in the context of cqa: checking the veracity of answers to a given question"
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                },
                {
                    "Quotes (what)": [
                        "for instance, the user could be presented with veracity scores, where low scores would warn him/her not to completely trust the answer or to double-check it."
                    ],
                    "Data Actors": [
                        "media consumers"
                    ],
                    "Model Owners": [
                        "social media companies"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                }
            ]
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "type": "assisted media consumption",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Owners": [
                        "social media companies"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "community question answering (cqa) forums such as stackoverflow, yahoo! answers, and quora are very popular nowadays, as they represent effective means for com- munities around particular topics to share information and to collectively satisfy their information needs. however, the information being shared is not always factual. there are multiple factors explaining the presence of incorrect answers in cqa forums, e.g., misunderstanding, ignorance, or mali- ciousness of the responder. this is exacerbated by the fact that most cqa forums are barely moderated and lack systematic quality control. moreover, in our dynamic world of today, truth is often time-sensitive: what was true yesterday may become false today."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Owners": [
                        "social media companies"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "none"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we explore a new dimension in the context of cqa: checking the veracity of answers to a given question. this aspect has been ignored so far, e.g., in recent cqa tasks at ntcir and semeval (ishikawa, sakai, and kando 2010; nakovetal.2015; nakovetal.2016; nakovetal.2017a), where an answer is considered as good if it tries to address the question, irrespective of its veracity. yet, veracity is an important aspect, as high-quality automatic fact check- ing can offer better user experience for cqa systems. for instance, the user could be presented with veracity scores, where low scores would warn him/her not to completely trust the answer or to double-check it."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Owners": [
                        "social media companies"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "provide labels/ veracityscores"
                    ]
                },
                {
                    "Quotes (what)": [
                        "here, we approach it as a supervised classification task, and we propose a novel model based on multi-faceted modeling of the facts, which integrates knowledge from several complementary sources, such as the answer content (what is said and how), the author profile (who says it), the rest of the community forum (where it is said), and external authoritative sources of information (external support)."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/C18-1284.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "social media are rife with rumours, which are fast-spreading, unverified pieces of information (zubiaga et al., 2018). with fake news and misinformation now widely recognised as a major problem for journalists, media, online platforms, and citizens, automatic rumour detection and analysis has become a hot research topic too."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                }
            ]
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "type": "vague identification",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "social media are rife with rumours, which are fast-spreading, unverified pieces of information (zubiaga et al., 2018). with fake news and misinformation now widely recognised as a major problem for journalists, media, online platforms, and citizens, automatic rumour detection and analysis has become a hot research topic too. rumour analysis research has focused on twitter in particular, as it has established itself as the go-to social platform for real-time news (hu et al., 2012). twitter\u2019s unmoderated nature is also the perfect ground for spreading rumours (qazvinian et al., 2011)."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific articles"
                    ]
                },
                {
                    "Quotes (what)": [
                        "a key focus of prior work on rumour analysis has been rumour stance classification, where the stance of each tweet on a given rumour is classified as supporting, denying, questioning or commenting on the rumour (procter et al., 2013). our work builds on the hypothesis that, as rumours evolve over time, so does the stance expressed by the public towards those rumours. in the early stages of a rumour, its actual veracity tends to be unknown. however, as new evidence emerges over time, twitter users take more pronounced and continuously evolving stance towards the information asserted in the rumour."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score stance"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "this paper investigates whether and to what extent rumour veracity classification can be predicted on the basis of crowd stance."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/2020.emnlp-main.623.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague opposition",
                    "Model Means": [
                        "classify/score veracity",
                        "justification/explanation production"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "recent events, including the covid-19 pandemic, demonstrate the significant potential harm of misinformation in the public health setting, and the importance in accurately fact-checking claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we introduce a framework for generating explanations and veracity prediction specific to\r public health fact-checking."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "justification/explanation production"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "type": "vague opposition",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "a great amount of progress has been made in the area of automated fact-checking. this includes more accurate machine learning models for veracity prediction and datasets of both naturally occurring (wang, 2017; augenstein et al., 2019; hanselowski et al., 2019) and human-crafted (thorne et al., 2018) fact-checking claims, against which the models can be evaluated. however, a few blind spots exist in the state-of-the-art."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this work we address specifically two shortcomings: the nar- row focus on political claims, and the paucity of explainable systems."
                    ]
                },
                {
                    "Quotes (why)": [
                        "one subject area which we believe could benefit from expertise-based fact-checking is public health \u2013 including the study of epidemiology, disease pre- vention in a population, and the formulation of public policies (turnock, 2012). recent events, including the covid-19 pandemic, demonstrate the significant potential harm of misinformation in the public health setting, and the importance in accurately fact-checking claims. unlike political and general misinformation, specific expertise is required in order to fact-check claims in this domain. oftentimes this expertise may be limited, and thus claims which surround public health may be inaccessible (e.g., because of the use of jargon and biomedical terminology) in a way political claims are not. nonetheless, like political misinformation, the public health variety is also potentially very dangerous, because it can put people in imminent danger and risk lives."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (why)": [
                        "fact-checking in domains where specific subject expertise is required presents an interesting challenge because general purpose fact-checking systems will not necessarily adapt well to these domains."
                    ]
                },
                {
                    "Quotes (why)": [
                        "explainable models can also aid the end users\u2019 understanding as they further elucidate claims and their context."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "justification/explanation production"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this study we explore the novel case of explainable automated fact-checking for claims for which specialised expertise or in-domain knowledge is essential. for our case study we examine the the public health (biomedical) context."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "justification/explanation production"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we present a novel dataset for explain- able fact-checking with gold standard fact- checking explanations by journalists. to the best of our knowledge, this is the first dataset specifically for fact-checking in the public health setting."
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/P19-1244.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "automated external fact-checking",
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research",
                        "automation"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the increasing popularity of social media has drastically changed how our daily news are produced, disseminated and consumed.1 without systematic moderation, a large volume of information based on false or unverified claims (e.g., fake news, rumours, propagandas, etc.) can proliferate online. such misinformation poses unprecedented challenges to information credibility"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (why)": [
                        "the effectiveness and efficiency of human fact-checking is handicapped by the volume and fast pace the noteworthy claims being produced on daily basis. therefore, it is an urgent need to automate the process and ease the human burden in assessing the veracity of claims (thorne and vlachos, 2018)."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ]
                }
            ]
        },
        "year": 2019,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "type": "automated external fact-checking",
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the increasing popularity of social media has drastically changed how our daily news are produced, disseminated and consumed.1 without systematic moderation, a large volume of information based on false or unverified claims (e.g., fake news, rumours, propagandas, etc.) can proliferate online. such misinformation poses unprecedented challenges to information credibility, which traditionally relies on fact-checkers to manually assess whether specific claims are true or not."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "news article/polls"
                    ]
                },
                {
                    "Quotes (why)": [
                        "therefore, it is an urgent need to automate the process and ease the human burden in assessing the veracity of claims (thorne and vlachos, 2018)."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "inspired by the fever task (thorne et al., 2018a) and declare (popat et al., 2018), we propose our approach to claim verification by using representation learning to embed sentence-level evidences based on coherence modeling and natural language inference (nli)."
                    ]
                },
                {
                    "Quotes (what)": [
                        "we propose a novel claim verification framework based on hierarchical attention neural networks to learn sentence-level evidence embeddings to obtain claim-specific representation."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        }
    },
    "https://www.acl-bg.org/proceedings/2017/RANLP%202017/pdf/RANLP005.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague debunking",
                    "Model Means": [
                        "classify/score stance"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (what)": [
                        "knowing the stance that authors hold in response to claims, e.g. in online commentary, gives useful insights. it can reveal rumours and fake news claims as the discourse around them is monitored (procter et al., 2013)"
                    ],
                    "Model Means": [
                        "classify/score stance"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                }
            ]
        },
        "year": 2017,
        "annotator_2": {
            "narratives": [
                {
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score stance"
                    ],
                    "Citation Support for Narratives": [
                        "investigative"
                    ],
                    "type": "assisted media consumption",
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Data Subjects": [
                        "social media users"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (what)": [
                        "open stance classification is often applied in rumour resolution. since attitudes in discourse around a claim are indicative not only of the controversiality of the claim, but also can act as a proxy for its veracity, it is reasonable to consider the application of open stance detection for rumour analysis. indeed, many approaches to rumour and fake news analysis rely on this signal (derczynski et al., 2017)1."
                    ],
                    "Model Means": [
                        "classify/score stance"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article + news article"
                    ]
                },
                {
                    "Quotes (what)": [
                        "this paper addresses the general-purpose, or open stance classification task. this is distinct from target-specific stance classification, as in augenstein et al. (2016) and mohammad et al. (2016), which focus on stances towards known, pre-determined targets. in the latter task, the target has already been extracted, from e.g. conversational cues. target-specific stance classification is suited to situations where the target is already known, such as analyses of a specific product or political actor. in contrast, the open stance classification task is appropriate in emerging news or novel contexts, such as working with online media or streaming news analysis."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score stance"
                    ]
                },
                {
                    "Quotes (what)": [
                        "our simple approach to open stance classification implements common features used in stance classification reported by related work (e.g. bag-of-words, named entities, user activity infor- mation, url presence). we extend this with problem-specific features (which we refer to as the af features) designed to capture how users react to tweets and express confidence in them."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score stance"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/S19-2149.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted external fact-checking",
                    "Data Subjects": [
                        "social media users",
                        "public figures/politicians",
                        "professional journalists"
                    ],
                    "Data Actors": [
                        "citizen journalists",
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval",
                        "human in the loop"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research",
                        "automation"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "like never before, a statement in an interview, a press release, a blog note, or a tweet can spread almost instantaneously. the speed of proliferation leaves little time for double-checking claims against the facts, which has proven critical in politics, e.g., during the 2016 presidential campaign in the usa, which was dominated by fake news in social media and by false claims."
                    ],
                    "Data Subjects": [
                        "professional journalists",
                        "social media users",
                        "public figures/politicians"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "anecdote"
                    ]
                },
                {
                    "Quotes (why)": [
                        "investigative journalists and volunteers have been working hard to get to the root of a claim and to present solid evidence in favor or against it manual fact-checking is very time-consuming, and thus automatic methods have been proposed to speed-up the process, e.g., there has been work on checking the factuality/credibility of a claim, of a news article, or of an information source (ba et al., 2016; zubiaga et al., 2016; ma et al., 2016; castillo et al., 2011; baly et al., 2018)."
                    ],
                    "Data Actors": [
                        "professional journalists",
                        "citizen journalists"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "Citation support": [
                        "past work"
                    ]
                },
                {
                    "Quotes (what)": [
                        "by comparing a claim against the retrieved evidence, a system can determine whether the claim is likely true or likely false (or unsure, if no strong enough evidence either way could be found)"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ]
                },
                {
                    "Quotes (what)": [
                        "the outcome could then be presented to a human expert for final judgement"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ]
                }
            ]
        },
        "year": 2019,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "type": "assisted external fact-checking",
                    "Data Subjects": [
                        "social media users",
                        "public figures/politicians",
                        "professional journalists"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "like never before, a statement in an interview, a press release, a blog note, or a tweet can spread almost instantaneously. the speed of proliferation leaves little time for double-checking claims against the facts, which has proven critical in politics, e.g., during the 2016 presidential campaign in the usa, which was dominated by fake news in social media and by false claims."
                    ],
                    "Data Subjects": [
                        "professional journalists",
                        "social media users",
                        "public figures/politicians"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat/ anecdote"
                    ]
                },
                {
                    "Quotes (why)": [
                        "investigative journalists and volunteers have been working hard to get to the root of a claim and to present solid evidence in favor or against it. manual fact-checking is very time-consuming, and thus automatic methods have been proposed to speed-up the process, e.g., there has been work on checking the factuality/credibility of a claim, of a news article, or of an information source (ba et al., 2016; zubiaga et al., 2016; ma et al., 2016; castillo et al., 2011; baly et al., 2018)."
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (why)": [
                        "also, as a result of our dynamic world, the truth is time-sensitive: something that was true yesterday may be false today. moreover, forums are often barely moderated and thus lack systematic quality control."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "here we focus on checking the factuality of questions and answers in cqa forums. this aspect was ignored in recent cqa tasks (ishikawa et al., 2010; nakov et al., 2015, 2016a, 2017a), where an answer is considered good if it addresses the question, irrespective of its veracity, accuracy, etc."
                    ]
                },
                {
                    "Quotes (what)": [
                        "moreover, high-quality automatic fact-checking would offer better experience to users of cqa systems, e.g., the user could be presented with veracity scores, where low scores would warn the user not to completely trust the answer or to double-check it."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ]
        }
    },
    "https://www.ijcai.org/proceedings/2020/0197.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "large amounts of fake or unverified information have emerged and spread to affect online\r social network users, which also leads to tremendous effects\r on the offline society. for example, the wide spread of fake\r news on social media has influenced the 2016 us presidential election [allcott and gentzkow, 2017]. thus, identifying\r rumors automatically is beneficial for providing early precautions to minimize its negative influence."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "type": "vague identification",
                    "Data Subjects": [
                        "social media users"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "with the rapid growth of social media such as twitter and weibo, information campaigns are frequently carried out by misinformation producers with various commercial and political purposes. consequently, large amounts of fake or unverified information have emerged and spread to affect online social network users, which also leads to tremendous effects on the offline society. for example, the wide spread of fake news on social media has influenced the 2016 us presidential election [allcott and gentzkow, 2017]. thus, identifying rumors automatically is beneficial for providing early precautions to minimize its negative influence."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "despite the success of deep graph neural networks, the lack of interpretability and robustness would make it risky for rumor detection. a fundamental assumption of these models is that a piece of information connecting with high-credit users would also have high credit, but the credit can be manipulated by rumor producers. a rumor could try to disguising itself by posting with hijacked or traded accounts from honest users. for example, in 2013, associated press\u2019s twitter account was hijacked and the rumor of explosion at white house was posted, causing the stock market to briefly plunge. other cheaper and more common camouflage strategies include connecting the posting user with high-credit users, removing opposing comments, and creating fake supporting comments."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we investigate: (i) how to simulate various camouflage methods on social networks to evade the rumor detector (ii) how to make the detector reveal camouflages as much as possible. to address these two issues, we rigorously define the possible camouflage, i.e., attack types in consider tion of domain constraints. we then propose a novel graph ad- versarial learning framework that enables such attacks to fool the detector by dynamically adding intentional perturbations. meanwhile, the detector would be enhanced to learn more distinctive structural features to resist such perturbations. the attacker and detector would strengthen each other iteratively, making our model produce robust detection results."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ]
        }
    },
    "https://public.ukp.informatik.tu-darmstadt.de/UKP_Webpage/publications/2019/2019_SIGNLL_AnH_RichlyAnnotatedCorpus.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "automated external fact-checking",
                    "Model Means": [
                        "classify/score stance",
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research",
                        "automation"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "false information can be shared through this channel reaching a much wider audience than traditional means of disinformation (howell et al., 2013)."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific paper"
                    ]
                },
                {
                    "Quotes (why)": [
                        "while human fact-checking still remains the primary method to counter this issue, the amount and the speed at which new information is spread makes manual validation challenging and costly. this motivates the development of automated factchecking pipelines (thorne et al., 2018a; popat et al., 2017; hanselowski and gurevych, 2017)"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Citation support": [
                        "past work"
                    ]
                },
                {
                    "Quotes (what)": [
                        "given a controversial claim, document retrieval is applied to identify documents that contain important information for the validation of the claim. evidence extraction aims at retrieving text snippets or sentences from the identified documents that are related to the claim. this evidence can be further processed via stance detection to infer whether it supports or refutes the claim. finally, claim validation assesses the validity of the claim given the evidence."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "classify/score stance",
                        "evidence retrieval"
                    ]
                }
            ]
        },
        "year": 2019,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research",
                        "automation"
                    ],
                    "type": "automated external fact-checking"
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the ever-increasing role of the internet as a primary communication channel is arguably the single most important development in the media over the past decades. while it has led to unprecedented growth in information coverage and distribution speed, it comes at a cost. false information can be shared through this channel reaching a much wider audience than traditional means of disinformation (howell et al., 2013). while human fact-checking still remains the primary method to counter this issue, the amount and the speed at which new information is spread makes manual validation challenging and costly. this motivates the development of automated fact-checking pipelines (thorne et al., 2018a; popat et al., 2017; hanselowski and gurevych, 2017) consisting of several consecutive tasks."
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific articles"
                    ]
                },
                {
                    "Quotes (what)": [
                        "first, training data needs to contain a large number of instances with high-quality annotations for the different fact-checking sub-tasks. second, the training data should not be limited to a particular domain, since potentially wrong information sources can range from official statements to blog and twitter posts. we analyzed existing corpora regarding their adherence to the above criteria and identified several drawbacks"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in order to address the drawbacks of existing datasets, we introduce a new corpus based on the snopes fact-checking website. our corpus consists of 6,422 validated claims with comprehensive annotations based on the data collected by snopes fact-checkers and our crowd-workers.  the corpus covers multiple domains, including discus- sion blogs, news, and social media, which are of- ten found responsible for the creation and distribu- tion of unreliable information. in addition to vali- dated claims, the corpus comprises over 14k doc- uments annotated with evidence on two granular- ity levels and with the stance of the evidence with respect to the claims. our data allows training machine learning models for the four steps of the automated fact-checking process described above: document retrieval, evidence extraction, stance de- tection, and claim validation."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/D19-1216.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "automated external fact-checking",
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify multimodal inconsistencies",
                        "supplant human fact-checkers"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "as social media become a bigger part of our daily lives, their influence over the way people think and make decisions increases. inevitably, this has offered opportunities for fake content to arise and to spread faster than ever, e.g., recent research has shown that fake news spreads six time faster than real news (vosoughi et al., 2018)"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific paper"
                    ]
                },
                {
                    "Quotes (why)": [
                        "to deal with the problem, a number of manual fact-checking initiatives\r have been launched, but they remain insufficient\r to cope with the ever growing number of checkworthy claims. thus, automated methods have\r been proposed as a more scalable solution."
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we study a new problem: predict the factual- ity of a claim with respect to an image."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify multimodal inconsistencies"
                    ]
                }
            ]
        },
        "year": 2019,
        "annotator_2": {
            "narratives": [
                {
                    "Application Means": [
                        "identify multimodal inconsistencies",
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "type": "automated external fact-checking",
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "as social media become a bigger part of our daily lives, their influence over the way people think and make decisions increases. inevitably, this has offered opportunities for fake content to arise and to spread faster than ever, e.g., recent research has shown that fake news spreads six time faster than real news (vosoughi et al., 2018).  sometimes such content is created for pure entertainment or for financial gain from advertisement shown alongside the fake content, but more often and especially recently it has been used to spread disinformation, e.g., with the aim to influence political elections (atanasov et al., 2019). to deal with the problem, a number of manual fact-checking initiatives have been launched, but they remain insufficient to cope with the ever growing number of check- worthy claims. thus, automated methods have been proposed as a more scalable solution."
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "given the recent proliferation of fake news, and given that many of the questionable claims are about images, it would be natural to expect similar interest in the computational linguistics community, especially given the fact that visual imagery is more influential than text and naturally appears alongside fake news."
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "here we aim at bridging this gap. in particular, we create a new dataset for this problem, and we explore a variety of features modeling the claim, the image, and the relationship between the claim and the image."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify multimodal inconsistencies"
                    ]
                },
                {
                    "Quotes (what)": [
                        "the contributions of this paper can be summa- rized as follows: \u2022 we study a new problem: predict the factuality of a claim with respect to an image. \u2022 we create a new dataset for this problem, which we release to the research community in order to enable further work. \u2022 we explore a variety of features, and we demonstrate sizable improvements over the baseline."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify multimodal inconsistencies"
                    ]
                }
            ]
        }
    },
    "https://ojs.aaai.org/index.php/ICWSM/article/view/3254/3122": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "unlike bias that is per-ceived  in  opinion  columns,  the  spread  of  fake  news  sur-rounding  the  documentation  of  the  war  compromises  notonly the integrity of journalism, but can contribute to psy-chological warfare that drives the exodus and constant mo-bility of refugees, and hampers humanitarian planning fordelivering aid to distraught communities."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (why)": [
                        "an evidence-basedapproach to combating fake news necessitates that one em-bark on a data scientific approach by which the general pub-lic  can  be  assisted  in  automatically  identifying  fake  newsaround the syrian conflict with some reasonable assurance"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (why)": [
                        "our news corpus consists of804english news articles thatreport on war incidents that took place from 2011 to 2018"
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ]
                },
                {
                    "Quotes (why)": [
                        "using  the  above  technique,  our  dataset  consists  of  426true articles and 378 fake articles"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2019,
        "annotator_2": {
            "narratives": [
                {
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "type": "automated external fact-checking"
                },
                {
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "type": "vague identification"
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "unlike bias that is perceived  in  opinion  columns,  the  spread  of  fake  news  sur-rounding  the  documentation  of  the  war  compromises  notonly the integrity of journalism, but can contribute to psy-chological warfare that drives the exodus and constant mobility of refugees, and hampers humanitarian planning fordelivering aid to distraught communities. an evidence-basedapproach to combating fake news necessitates that one em-bark on a data scientific approach by which the general public  can  be  assisted  in  automatically  identifying  fake  news around the syrian conflict with some reasonable assurance.the lack of manually labeled fake news datasets around the syrian conflict is the major bottleneck for advancing au-tomatic  fake  news  detection."
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in  this  work,  we  embark  onthe  first  step  towards  this  goal,  which  necessitates  acquiring and exploring media accounts from the syrian war, andusing them to generate labeled benchmark datasets. to thisend, we develop a general-purpose distributed architectureleveraging some of the most recent technologies to handlebig data such as spark streaming, and hbase, for pullinglive-data streams and scraping for historical data from sev-eral media outlets. using this news scraping framework, weexplore a variety of media outlets representing mobilisationpress, loyalist press, and diverse print media, and generate arepresentative corpus of these various types of media outlets.our news corpus consists of804english news articles thatreport on war incidents that took place from 2011 to 2018."
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "manually labeling news articles as true or fake is not onlya difficult task; it can also be very subjective. this is par-ticularly true for the case of news articles reporting on warincidents, where fake news might be accurately reporting acertain incident, and yet distorting some of the facts such asthe number of casualties, the type of attack or the actor re-sponsible for the attack. to avoid any subjectivity and obtainas accurate labels as possible, we employ a semi-supervisedfact-checkinglabeling approach. more precisely, we tap onthe  database  of  the  syrian  violation  documentation  cen-ter  (vdc)1.  the  vdc  is  a  non-profit,  non-governmentalorganization  registered  in  switzerland  that  documents  hu-man rights violations from the syrian war."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                },
                {}
            ]
        }
    },
    "https://aclanthology.org/2020.findings-emnlp.27.pdf": {
        "annotator_1": {
            "narratives": [],
            "quotes": []
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "filter system outputs"
                    ],
                    "Citation Support for Narratives": [
                        "investigative"
                    ],
                    "type": "scientific curiosity"
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "textual entailment is well studied, but many relevant data sources are structured or semi-structured: health data both worldwide and personal, fitness trackers, stock markets, and sport statistics. while some information needs can be anticipated by hand- crafted templates, user queries are often surprising, and having models that can reason and parse that structure can have a great impact in real world ap- plications (khashabi et al., 2016; clark, 2019). a recent example is tabfact (chen et al., 2020), a dataset of statements that are either en- tailed or refuted by tables from wikipedia (figure 1). because solving these entailment problems requires sophisticated reasoning and higher-order operations like arg max, averaging, or comparing, human accuracy remains substantially (18 points) ahead of the best models (zhong et al., 2020)."
                    ]
                },
                {
                    "Quotes (what)": [
                        "this paper addresses these shortcomings using intermediate task pre- training (pruksachatkun et al., 2020), creating efficient data representations, and applying these improvements to the tabular entailment task."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "filter system outputs"
                    ]
                }
            ]
        }
    },
    "https://arxiv.org/pdf/1903.01728.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "in recent years, fake news on social media has threatened not only cyberspace security, but also the real-world order in politics [14], economy [12], society [2], etc. the most recent example is the concomitant infodemic during the covid-19 pandemic across the world [41]. thousands of news pieces with misleading content have been spreading through social media [44] and led to socio-economic disorder [6] and weakened the effect of pandemic prevention [4]. to tackle this issue, researchers have been devoted to developing automatic methods to detect fake news (i.e., designing a classifier to judge a given news piece as real or fake) by leveraging signals from text [5, 32, 34], images [20, 35], or social contexts [17, 24, 26\u2013 28, 39, 40]. 3"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we use news pieces to refer to social media news posts. a news piece\r generally contains content and its attached comments."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ]
                }
            ]
        },
        "year": 2019,
        "annotator_2": {
            "narratives": [
                {
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "type": "vague identification",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "in recent years, fake news on social media has threatened not only cyberspace security, but also the real-world order in politics [14], economy [12], society [2], etc. the most recent example is the concomitant infodemic during the covid-19 pandemic across the world [41]. thousands of news pieces with misleading content have been spreading through social media [44] and led to socio-economic disorder [6] and weakened the effect of pandemic prevention [4]. to tackle this issue, researchers have been devoted to developing automatic methods to detect fake news (i.e., designing a classifier to judge a given news piece as real or fake) by leveraging signals from text [5, 32, 34], images [20, 35], or social contexts [17, 24, 26\u2013 28, 39, 40]. 3"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific articles"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we use news pieces to refer to social media news posts. a news piece generally contains content and its attached comments."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "to describe the two emotions clearly, we define them respectively as 1) publisher emotion: the emotions conveyed by publishers of the news pieces; and 2) social emotion: the emotions aroused in the crowd facing to the news pieces. and we adopt dual emotion as a general term of these two emotions. for a news piece, dual emotion has two appearances: emotion resonances (i.e., the pub- lisher emotion is same or similar to the social emotion) and emotion dissonances (i.e., the publisher emotion is different from the social emotion). we analyze the data and find that the two appearances have a statistically significant distinction between fake and real news (see details in section 4.2)."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                }
            ]
        }
    },
    "http://ceur-ws.org/Vol-2125/invited_paper_13.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted external fact-checking",
                    "Data Subjects": [
                        "social media users",
                        "public figures/politicians",
                        "professional journalists"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "human in the loop"
                    ],
                    "Application Means": [
                        "triage claims"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the current coverage of the political landscape in both the press and in social media has led to an unprecedented situation. like never before, a statement in an interview, a press release, a blog note, or a tweet can spread almost instantaneously across the globe. this proliferation speed has left little time for double-checking claims against the facts, which has proven critical in politics, e.g., during the 2016 us presidential campaign, which was influenced by fake news in social media and by false claims. indeed, some politicians were fast to notice that when it comes to shaping public opinion, facts were secondary, and that appealing to emotions and beliefs worked better, especially in social media. it has been even proposed that this was marking the dawn of a post-truth age."
                    ],
                    "Data Subjects": [
                        "professional journalists",
                        "social media users",
                        "public figures/politicians"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat",
                        "anecdote"
                    ]
                },
                {
                    "Quotes (why)": [
                        "as the problem became evident, a number of fact-checking initiatives have started, led by organizations such as factcheck and snopes, among many others. yet, this has proved to be a very demanding manual effort, which means that only a relatively small number of claims could be fact-checked.7 this makes it important to prioritize the claims that fact-checkers should consider first. task 1 of the checkthat! lab at clef-2018 [17] aims to help in that respect"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Application Means": [
                        "triage claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "given a transcription of a political debate/speech, predict which claims should be prioritized for fact-checking."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "human in the loop"
                    ],
                    "Application Means": [
                        "triage claims"
                    ]
                }
            ]
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity",
                        "human in the loop"
                    ],
                    "Application Means": [
                        "triage claims"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "type": "assisted external fact-checking",
                    "Data Subjects": [
                        "public figures/politicians"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the current coverage of the political landscape in both the press and in social media has led to an unprecedented situation. like never before, a statement in an interview, a press release, a blog note, or a tweet can spread almost instantaneously across the globe. this proliferation speed has left little time for double-checking claims against the facts, which has proven critical in politics, e.g., during the 2016 us presidential campaign, which was influenced by fake news in social media and by false claims. indeed, some politicians were fast to notice that when it comes to shaping public opinion, facts were secondary, and that appealing to emotions and beliefs worked better, especially in social media. it has been even proposed that this was marking the dawn of a post-truth age."
                    ],
                    "Data Subjects": [
                        "public figures/politicians"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (why)": [
                        "as the problem became evident, a number of fact-checking initiatives have started, led by organizations such as factcheck and snopes, among many others. yet, this has proved to be a very demanding manual effort, which means that only a relatively small number of claims could be fact-checked.7 this makes it important to prioritize the claims that fact-checkers should consider first. task 1 of the checkthat! lab at clef-2018 [17] aims to help in that respect, asking participants to build systems that can mimic the selection strategies of a particular fact-checking organization: factcheck.org."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                },
                {
                    "Quotes (what)": [
                        "it is defined as follows: given a transcription of a political debate/speech, predict which claims should be prioritized for fact-checking. the goal is to produce a ranked list of sentences ordered by their worthiness for fact-checking."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "human in the loop"
                    ],
                    "Application Means": [
                        "triage claims"
                    ]
                }
            ]
        }
    },
    "http://ceur-ws.org/Vol-2125/invited_paper_14.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague debunking",
                    "Data Subjects": [
                        "public figures/politicians"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "human in the loop"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (what)": [
                        "this paper offers an overview of the clef 2018 checkthat lab \u201ctask 2: factuality\u201d, which focuses on tools to verify (and possibly to provide evidence to an expert about) the factuality of a claim in a political debate or a speech."
                    ],
                    "Data Subjects": [
                        "public figures/politicians"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ]
                },
                {
                    "Quotes (what)": [
                        "given a check-worthy claim in the form of a (transcribed) sentence, determine whether the claim is likely to be true, half-true, or false."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [
                {
                    "Data Subjects": [
                        "public figures/politicians"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "human in the loop"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "type": "assisted external fact-checking"
                },
                {
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "gather and present evidence",
                        "identify claims"
                    ],
                    "type": "vague debunking"
                }
            ],
            "quotes": [
                {
                    "Quotes (what)": [
                        "the checkthat! lab at clef-2018 [24] promotes the development of tools for computational journalism. it is divided into two tasks. this paper offers an overview of the clef 2018 checkthat lab \u201ctask 2: factuality\u201d, which focuses on tools to verify (and possibly to provide evidence to an expert about) the factuality of a claim in a political debate or a speech. the reader interested in \u201ctask 1: check-worthiness\u201d can refer to [1]. task 2 represents the final step in the pipeline of the full fact-checking process, displayed in figure 1. it is defined as follows: given a check-worthy claim in the form of a (transcribed) sentence, determine whether the claim is likely to be true, half-true, or false."
                    ],
                    "Data Subjects": [
                        "public figures/politicians"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "human in the loop"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "none"
                    ]
                },
                {
                    "Quotes (what)": [
                        "the most successful approaches used by the participants based their veracity predictions on evidence retrieved from the web, which they compared to the target claim. then, they used a supervised model to predict whether the claim should be considered as true, half-true, or false."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "identify claims",
                        "gather and present evidence"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/D15-1312.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted internal fact-checking",
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores",
                        "identify claims"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ]
                },
                {
                    "type": "assisted internal fact-checking",
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "statistical properties are commonly used to describe entities, e.g. population for countries, net value for companies, points scored for athletes, etc. claims about such properties are very common in news articles and social media, however they can be erroneous, either due to author error or negligence at the time of writing or because they eventually become out of date. while manual verification (also referred to as factchecking) is conducted by journalists in news organizations and dedicated websites such as www. emergent.info, the volume of the claims calls for automated approaches, which is one of the main objectives of computational journalism (cohen et al., 2011; flew et al., 2012)."
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper we develop a baseline approach to identify and verify simple claims about statistical properties against a database"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims",
                        "provide labels/veracity scores"
                    ]
                },
                {
                    "Quotes (what)": [
                        "given a sentence, we first identify whether it contains a claim about a property we are interested in (population in the example), which entity it is about and the value claimed (lesotho and 2,000,000 respectively). we then proceed to verify the value claimed in text for the property of this entity against the value known in a knowledge base such as freebase and return a score reflecting the accuracy of the claim (absolute percentage error in the example)"
                    ],
                    "Model Means": [
                        "evidence retrieval",
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                }
            ]
        },
        "year": 2015,
        "annotator_2": {
            "narratives": [
                {
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores",
                        "supplant human fact-checkers",
                        "identify claims"
                    ],
                    "type": "assisted internal fact-checking",
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "statistical properties are commonly used to describe entities, e.g. population for countries, net value for companies, points scored for athletes, etc. claims about such properties are very common in news articles and social media, however they can be erroneous, either due to author error or negligence at the time of writing or because they eventually become out of date. while manual verification (also referred to as fact-checking) is conducted by journalists in news organizations and dedicated websites such as www. emergent.info, the volume of the claims calls for automated approaches, which is one of the main objectives of computational journalism (cohen et al., 2011; flew et al., 2012)."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Citation support": [
                        "scientific articles"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper we develop a baseline approach to identify and verify simple claims about statistical properties against a database. the task is illustrated in figure 1. given a sentence, we first identify whether it contains a claim about a property we are interested in (population in the example), which entity it is about and the value claimed (lesotho and 2,000,000 respectively). we then proceed to verify the value claimed in text for the property of this entity against the value known in a knowledge base such as freebase and return a score reflecting the accuracy of the claim (absolute percentage error in the example)."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims",
                        "provide labels/veracity scores"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ]
                },
                {
                    "Quotes (what)": [
                        "claim identification is essentially an instance of information extraction. while it would be possible to develop supervised models, this would require expensive manual data annotation for each property of interest. instead, we follow the distant supervision paradigm (craven and kumlien, 1999; mintz et al., 2009) using supervision obtained by combining triples from a knowledge base and raw text."
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                }
            ]
        }
    },
    "https://dl.acm.org/doi/pdf/10.1145/3289600.3290996": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted knowledge curation",
                    "Data Actors": [
                        "engineers and curators"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval",
                        "human in the loop"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "kgs also contain doubtful if not incorrect spo triples, as they are partly built by automatic information extraction, crowd-sourcing, or methods for kg completion using rules [16] or embeddings [29, 31, 40]. this incurs the problem of validating if an spo triple is correct or not, a task that is often referred to as fact-checking or truth discovery [24]."
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ],
                    "Citation support": [
                        "scientific articles"
                    ]
                },
                {
                    "Quotes (why)": [
                        "traditionally, fact-checking has been performed manually by human reviewers but this is time-consuming."
                    ],
                    "Data Actors": [
                        "engineers and curators"
                    ]
                },
                {
                    "Quotes (what)": [
                        "methods for automatic fact-checking (e.g., [18, 24, 26, 30, 32]) proceed in two steps. first, they perform fact-spotting by searching for occurrences of a fact candidate, such as \u27e8sadiq_khan citizenof uk\u27e9, and possible alternatives, such as \u27e8sadiq_khan citizenof pakistan\u27e9, in the web sources. this is done by expanding the predicate into paraphrases (e.g., \"has nationality\", \u201chas passport\u201d) and searching for it jointly with the alias names of the s and o arguments. then, the extracted evidence (or counterevidence) is used to infer the truth value of the candidate fact."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ]
                },
                {
                    "Quotes (what)": [
                        "to better support kg curators in deciding the correctness of the candidate facts, we propose a novel framework for finding semantically related evidence in web sources and the underlying kg, and for computing human-comprehensible explanations for facts"
                    ],
                    "Data Actors": [
                        "engineers and curators"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ]
                }
            ]
        },
        "year": 2019,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity",
                        "justification/explanation production",
                        "human in the loop"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ],
                    "type": "assisted knowledge curation"
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "however, kgs also contain doubtful if not incorrect spo triples, as they are partly built by automatic information extraction, crowd-sourcing, or methods for kg completion using rules [16] or embeddings [29, 31, 40]. this incurs the problem of validating if an spo triple is correct or not, a task that is often referred to as fact-checking or truth discovery [24]."
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ],
                    "Citation support": [
                        "scientific articles"
                    ]
                },
                {
                    "Quotes (why)": [
                        "traditionally, fact-checking has been performed manually by human reviewers but this is time-consuming. therefore, with the increase of false facts on the web, the automation of fact-checking is gaining more attention."
                    ],
                    "Data Subjects": [
                        "engineers and curators"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ]
                },
                {
                    "Quotes (what)": [
                        "to better support kg curators in deciding the correctness of the candidate facts, we propose a novel framework for finding semantically related evidence in web sources and the underlying kg, and for computing human-comprehensible explanations for facts. we refer to our framework, as exfakt (explaining facts over kgs and text resources)."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "human in the loop",
                        "justification/explanation production"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/2021.naacl-main.52.pdf": {
        "annotator_1": {
            "narratives": [],
            "quotes": []
        },
        "year": 2021,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ],
                    "Citation Support for Narratives": [
                        "investigative"
                    ],
                    "type": "assisted knowledge curation",
                    "Ends": [
                        "increase veracity of published content"
                    ]
                },
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "type": "vague debunking",
                    "Ends": [
                        "increase veracity of published content"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "determining the truthfulness of factual claims by comparing them to textual sources of evidence has received intense research interest in recent years. an underlying, but often overlooked, challenge for this paradigm, however, is the dynamic nature of today\u2019s written resources. an extraordinary amount of new information becomes available daily; as a result, many consequential facts are established, changed, or added to over time."
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ],
                    "Citation support": [
                        "none"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we argue that the quality of fact verification systems should be measured by how well they adjust to new evidence. in this way, we seek to advance fact verification by requiring that models remain reliable and robust to the change present in practical settings"
                    ],
                    "Model Means": [
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ]
                },
                {
                    "Quotes (what)": [
                        "to this end, we focus on fact verification with contrastive evidence. that is, we infuse the standard fact verification paradigm with challenging cases that require models to be sensitive to factual changes in their presented evidence (hereon referred to interchangeably as \u201ccontext\u201d). we present vitaminc, a new large-scale fact verification dataset that is based on factual revisions to wikipedia. the key concept is exemplified in figure 1: there a factual revision yields a contrastive pair of contexts that are nearly identical in language and content\u2014except that one context refutes the given claim, while the other supports it."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                },
                {
                    "Quotes (why)": [
                        "this type of contrastive structure exposes existing deficiencies in model behavior."
                    ]
                }
            ]
        }
    },
    "https://link.springer.com/content/pdf/10.1007/978-3-030-72240-1.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted external fact-checking",
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval",
                        "human in the loop"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores",
                        "triage claims",
                        "gather and present evidence"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the mission of the checkthat! lab is to foster the development of technology that would enable the automatic verification of claims. automated systems for claim identification and verification can be very useful as supportive technology for investigative journalism, as they could provide help and guidance, thus saving time [20,32,34,58]."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "a system could automatically identify check-worthy claims, make sure they have not been fact-checked already by a reputable fact-checking organization, and then present them to a journalist for further analysis in a ranked list."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "human in the loop"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "additionally, the system could identify documents that are potentially useful for humans to perform manual fact-checking of a claim, and it could also estimate a veracity score supported by evidence to increase the journalist\u2019s understanding and the trust in the system\u2019s decision."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval",
                        "human in the loop"
                    ],
                    "Application Means": [
                        "triage claims",
                        "gather and present evidence",
                        "provide labels/veracity scores"
                    ]
                }
            ]
        },
        "year": 2021,
        "annotator_2": {
            "narratives": [
                {
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval",
                        "human in the loop"
                    ],
                    "Application Means": [
                        "triage claims",
                        "gather and present evidence",
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "type": "assisted external fact-checking"
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "systems for claim identification and verification can be very useful as supportive technology for investigative journalism, as they could provide help and guidance, thus saving time [34,45,47,97,54]. a system could automatically identify check-worthy claims, make sure they have not been fact-checked already by a reputable fact- checking organization, and then present them to a journalist for further analysis in a ranked list. additionally, the system could identify documents that are potentially useful for humans to perform manual fact-checking of a claim, and it could also estimate a veracity score supported by evidence to increase the journalist\u2019s understanding and trust in the system\u2019s decision."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval",
                        "human in the loop"
                    ],
                    "Application Means": [
                        "identify claims",
                        "gather and present evidence"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in the 2021 edition of the checkthat! lab, we feature three tasks: 1. check- worthiness estimation, 2. detecting previously fact-checked claims, and 3. pre- dicting the veracity of news articles and their domain. in these tasks, we focus on (i) tweets, (ii) political debates and speeches, and (iii) news articles. more- over, besides arabic and english, we extend our language coverage to bulgarian, spanish, and turkish. we further add a new task (task 3) on multi-class fake news detection for news articles and topical domain identification, which can help direct the article to the right fact-checking expert[68]."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "human in the loop"
                    ],
                    "Application Means": [
                        "triage claims"
                    ]
                }
            ]
        }
    },
    "https://arxiv.org/pdf/2106.05707.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "automated external fact-checking",
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "type": "vague identification",
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "interest in automating fact verification has been growing as the volume of potentially misleading and false claims rises [graves, 2018], resulting in the development of both fully automated methods (see thorne and vlachos [2018], zubiaga et al. [2018], hardalov et al. [2021] for recent surveys) as well as technologies that can assist human journalists [nakov et al., 2021]."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (why)": [
                        "fact verification has attracted a lot of attention in the machine learning and natural language processing communities, as it is one of the key methods for detecting misinformation"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "each claim is annotated with evidence in the form of sentences and/or cells from tables in wikipedia, as well as a label indicating whether this evidence supports, refutes, or does not provide enough information to reach a verdict."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ]
                }
            ]
        },
        "year": 2021,
        "annotator_2": {
            "narratives": [
                {
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "type": "automated external fact-checking"
                },
                {
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "type": "assisted external fact-checking"
                },
                {
                    "Model Means": [
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "type": "vague identification"
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "interest in automating fact verification has been growing as the volume of potentially misleading and false claims rises [graves, 2018], resulting in the development of both fully automated methods (see thorne and vlachos [2018], zubiaga et al. [2018], hardalov et al. [2021] for recent surveys) as well as technologies that can assist human journalists [nakov et al., 2021]."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "this has been enabled by the creation of datasets of appropriate scale, quality, and complexity in order to develop and evaluate models for fact extraction and verification, e.g. thorne et al. [2018], augenstein et al. [2019]. most large-scale datasets focus exclusively on verification against textual evidence rather than tables. furthermore, table-based datasets, e.g. chen et al. [2020a], assume an unrealistic setting where an evidence table is provided, requiring extensions to evaluate retrieval [schlichtkrull et al., 2020]."
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we introduce a novel dataset and benchmark, feverous: fact extraction and verification over unstructured and structured information, consisting of claims verified against wikipedia pages and labeled as supported, refuted, or not enough information. each claim has evidence in the form of sentences and/or cells from tables in wikipedia. figure 1 shows two examples that illustrate the level of complexity of the dataset. a claim may require a single table cell, a single sentence, or a combination of multiple sentences and cells from different articles as evidence for verification. feverous contains 87,026 claims, manually constructed and verified by trained annotators."
                    ],
                    "Model Means": [
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                }
            ]
        }
    },
    "https://arxiv.org/pdf/2012.00614.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted external fact-checking",
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "evidence retrieval",
                        "human in the loop"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "with the easy availability of information through the internet and social media, claims of unknown veracity manipulate public perception and interpretation. misinformation and disinformation are particularly pressing issues for the climate change debate. they have confused the public, led to political inaction, and stalled support for climate-change mitigation measures [2, 3, 4, 5]"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (why)": [
                        "to counter the influence of potentially false claims on the formation of public opinion on climate change, researchers and experts began to manually assess claims\u2019 veracity and publish their assessments on platforms such as climatefeedback.org and skepticalscience.com. recently, new literature on algorithmic fact-checking has emerged, using machine learning and natural language understanding (nlu) to work on this problem from different angles"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we believe that technology cannot and will not in the foreseeable future replace human fact-checkers. but it can help to provide relevant, reliable evidence for humans to make better decisions about the veracity of a claim"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "evidence retrieval",
                        "human in the loop"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ],
                    "type": "assisted external fact-checking",
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "with the easy availability of information through the internet and social media, claims of unknown veracity manipulate public perception and interpretation. misinformation and disinformation are particularly pressing issues for the climate change debate. they have confused the public, led to political inaction, and stalled support for climate-change mitigation measures [2, 3, 4, 5]."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "to counter the influence of potentially false claims on the formation of public opinion on climate change, researchers and experts began to manually assess claims\u2019 veracity and publish their assessments on platforms such as climatefeedback.org and skepticalscience.com."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (why)": [
                        "we believe that technology cannot and will not in the foreseeable future replace human fact-checkers. but it can help to provide relevant, reliable evidence for humans to make better decisions about the veracity of a claim"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "human in the loop",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this work, we focus on building a dataset of real-world claims specifically on climate change. we collect 1,535 claims on the internet. for each claim, we algorithmically retrieve the top five relevant evidence candidate sentences from wikipedia by the use of nlu where humans annotate each sentence as supporting, refuting, or not giving enough information to validate the claim. we call this database of 7,675 annotated claim-evidence pairs the climate-fever dataset.1."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/2020.fever-1.5.pdf": {
        "annotator_1": {
            "narratives": [],
            "quotes": []
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "type": "automated external fact-checking",
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "type": "vague opposition",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "recently, a number of works have found that lms store a surprising amount of world knowledge, focusing particularly on the task of open-domain question answering (petroni et al., 2019; roberts et al., 2020)."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we explore whether we can leverage the knowledge in lms for fact checking."
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we propose an approach (fig. 1b) that replaces the document retriever and evidence selector models in traditional fact-checking (fig. 1a) with a single language model that generates masked tokens."
                    ],
                    "Quotes (why)": [
                        "evidence retrieval, classification"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/W18-5517.pdf": {
        "annotator_1": {
            "narratives": [],
            "quotes": []
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ],
                    "type": "vague debunking"
                }
            ],
            "quotes": [
                {
                    "Quotes (what)": [
                        "our approach to fever is to fix the most obvious shortcomings of the baseline approaches to retrieval and entailment, and to train a sharp entailment classifier that can be used to filter a broad set of retrieved potential evidence. for the entailment classifier we compare decompos- able attention (parikh et al., 2016; gardner et al., 2017) as implemented in the official baseline, esim (chen et al., 2017), and a transformer net- work with pre-trained weights (radford et al., 2018)."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ]
                }
            ]
        }
    },
    "https://arxiv.org/pdf/2106.03794.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted external fact-checking",
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval",
                        "human in the loop"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "type": "vague opposition",
                    "Data Actors": [
                        "media consumers"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "type": "automated external fact-checking",
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the proliferation of disinformation and misinformation on the web is increasing at a scale that calls for the automation of the slow and laborintensive manual fact-checking process (vosoughi et al., 2018)."
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (why)": [
                        "there is a need for automated fact-checking tools to assist professional fact-checkers and the public in evaluating the veracity of claims that are propagated online in news articles or social media"
                    ],
                    "Data Actors": [
                        "professional journalists",
                        "media consumers"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ]
                },
                {
                    "Quotes (what)": [
                        "ideally, a fact-checking pipeline will address several tasks: 1) consider real-world claims, 2) retrieve relevant documents not bounded to a known document collection (e.g., wikipedia) and which contain information to validate the claim, 3) select evidence sentences that can support or refute the claim and 4) predict the claim veracity based on this evidence"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ]
                }
            ]
        },
        "year": 2021,
        "annotator_2": {
            "narratives": [
                {
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Application Means": [
                        "gather and present evidence",
                        "supplant human fact-checkers",
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "type": "automated external fact-checking",
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ]
                },
                {
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "gather and present evidence",
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "type": "assisted external fact-checking"
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the proliferation of disinformation and misinfor- mation on the web is increasing at a scale that calls for the automation of the slow and labor- intensive manual fact-checking process (vosoughi et al., 2018)."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "new york times reports that \u201cphysicians say they regularly treat people more inclined to believe what they read on facebook than what a medical professional tells them.\u201d disinformation is even more acute around the recent covid-19 pandemic. as a result, there is a need for automated fact-checking tools to assist professional fact-checkers and the public in evaluating the veracity of claims that are propagated online in news articles or social media."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "news article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "ideally, a fact-checking pipeline will address several tasks: 1) consider real-world claims, 2) retrieve relevant documents not bounded to a known document collection (e.g., wikipedia) and which contain information to validate the claim, 3) se- lect evidence sentences that can support or refute the claim and 4) predict the claim veracity based on this evidence."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "identify claims",
                        "gather and present evidence"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we propose a novel semi-automatic method to build a fact-checking dataset for covid-19 (covid-fact) with the goal of facilitating all the above tasks."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "identify claims",
                        "gather and present evidence"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ]
        }
    },
    "https://arxiv.org/pdf/2102.00816.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "despite the success and popularity of online media, the suitability and rapidly-spreading nature of micro-blogs fosters the emergence of various rumors [1, 3]. individuals encountering rumors on social media may turn to other sources to evaluate, expose, or reinforce rumors [6, 28]. the rapid and wide spread of rumors can cause various far-reaching consequences, for example, during the 2016 u.s. presidential election, 529 different rumors about donald trump and hillary clinton spread on social media [21] and reached millions of voters swiftly"
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers",
                        "anecdote"
                    ]
                },
                {
                    "Quotes (what)": [
                        "on a related note, if the veracity status is confirmed to be false, the rumor can then be considered as fake news. rumor handling research efforts cast four main elements: rumor detection, rumor tracking, rumor stance classification, and rumor veracity classification [44]."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                }
            ]
        },
        "year": 2021,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score stance",
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "type": "vague identification",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "despite the success and popularity of online media, the suitability and rapidly-spreading na- ture of micro-blogs fosters the emergence of various rumors [1, 3]."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "individuals encountering rumors on social media may turn to other sources to evaluate, expose, or reinforce rumors [6, 28]. the rapid and wide spread of rumors can cause various far-reaching conse- quences, for example, during the 2016 u.s. presidential election, 529 different rumors about donald trump and hillary clinton spread on social media [21] and reached millions of voters swiftly. hence, these rumors could potentially influence the election [3]. more re- cently, the rapid spread of rumors about 2019 novel coronavirus [7, 34, 35] (some of which are verified to be very dangerous false claims [20], e.g., those that suggest drinking bleach cures the illness [40]) has made social media companies such as facebook to find more effective solutions [43]."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "on a related note, if the veracity status is confirmed to be false, the rumor can then be considered as fake news. rumor handling research efforts cast four main elements: rumor detection, rumor tracking, rumor stance classification, and rumor veracity classification [44]. a typical rumor classification system includes all the four elements."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "classify/score stance"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we propose vroc, a tweet-level text-based novel rumor clas- sification system based on variational autoencoders. vroc realizes all four tasks in the rumor classification system and achieves high performance compared to state-of-the- art works."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "classify/score stance"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/2020.findings-emnlp.309.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague opposition",
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the proliferation of social media platforms and\r digital content has been accompanied by a rise in\r deliberate disinformation and hoaxes, leading to\r polarized opinions among masses."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (why)": [
                        "with the increasing number of inexact statements, there is a\r large interest in a fact-checking system that can verify claims based on automatically retrieved facts\r and evidence."
                    ],
                    "Model Means": [
                        "evidence retrieval",
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ],
                    "type": "vague debunking",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the proliferation of social media platforms and digital content has been accompanied by a rise in deliberate disinformation and hoaxes, leading to polarized opinions among masses. with the increasing number of inexact statements, there is a large interest in a fact-checking system that can verify claims based on automatically retrieved facts and evidence."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "hence, while providing very useful starting points for the community, fever is mostly restricted to a single-hop setting and existing multi- hop qa datasets are limited by the number of reasoning steps and the word overlapping between the question and all evidence. an ideal multi-hop example should have at least one piece of evidence (supporting document) that cannot be retrieved with high precision by shallowly performing direct semantic matching with only the claim."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we try to address these issues by creating hover (i.e., hoppy verification) whose claims (1) require evidence from as many as four english wikipedia articles and (2) contain significantly less semantic overlap between the claims and some supporting documents to avoid reasoning shortcuts.we create hover with 26k claims in three stages."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "gather and present evidence"
                    ]
                }
            ]
        }
    },
    "https://arxiv.org/pdf/1906.09198.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted external fact-checking",
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "human in the loop",
                        "justification/explanation production"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores",
                        "gather and present evidence"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "type": "vague debunking",
                    "Model Owners": [
                        "social media companies"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "justification/explanation production"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores",
                        "gather and present evidence"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "due to the increase of sources spreading false information, computational fact checking has been proposed to support journalists and social media platforms with automatic verification of textual content [1]."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Owners": [
                        "social media companies"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "assuming entities and relations involved in \u201cworth-checking\u201d claims have been identified [9, 11], kgs are exploited to compute the veracity of claims expressed as structured data."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we use existing kgs as sources of evidence, together with logical reasoning to make fact checking decisions. the key idea is to assess as true or false a given claim and to provide human-interpretable explanations for such decision in the form of supporting and contradicting evidence"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "justification/explanation production"
                    ],
                    "Application Means": [
                        "gather and present evidence",
                        "provide labels/veracity scores"
                    ]
                }
            ]
        },
        "year": 2019,
        "annotator_2": {
            "narratives": [
                {
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "type": "assisted external fact-checking"
                },
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "type": "vague identification"
                },
                {
                    "Model Means": [
                        "justification/explanation production"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores",
                        "gather and present evidence"
                    ],
                    "type": "vague debunking"
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "due to the increase of sources spreading false information, computational fact checking has been proposed to support journalists and social media platforms with automatic verification of textual content [1]."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific article"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we focus on claims that contain factual statements, such as \u201cwilliam durant was the founder of chevrolet\u201d, and their verification against reference data, i.e., knowledge graphs (kgs). assuming entities and relations involved in \u201cworth-checking\u201d claims have been identified [9, 11], kgs are exploited to compute the ve- racity of claims expressed as structured data."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "the key idea is to assess as true or false a given claim and to provide human-interpretable explanations for such decision in the form of supporting and contradicting evidence."
                    ],
                    "Model Means": [
                        "justification/explanation production"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores",
                        "gather and present evidence"
                    ]
                },
                {}
            ]
        }
    },
    "https://aclanthology.org/2020.acl-main.539.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague opposition",
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "fact checking for textual statements has emerged as an essential research topic recently because of the unprecedented amount of false news and rumors spreading through the internet (thorne et al., 2018; chen et al., 2019; goodrich et al., 2019; nakamura et al., 2019; krysci \u00b4 nski et al. \u00b4 , 2019; vaibhav et al., 2019)"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this work, we study fact checking, with the goal of automatically assessing the truthfulness of a textual statement."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "type": "vague opposition",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "fact checking for textual statements has emerged as an essential research topic recently because of the unprecedented amount of false news and rumors spreading through the internet (thorne et al., 2018;chen et al., 2019; goodrich et al., 2019; nakamura et al., 2019; krys \u0301cin \u0301ski et al., 2019; vaibhav et al., 2019)."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "previous research"
                    ]
                },
                {
                    "Quotes (why)": [
                        "online misinformation may manipulate people\u2019s opinions and lead to significant influence on essential social events like political elections (faris et al., 2017)"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "previous research"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this work, we study fact checking, with the goal of automatically assessing the truthfulness of a textual statement."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                },
                {
                    "Quotes (what)": [
                        "however, modeling logical operations is an essential step towards the modeling of complex reasoning and semantic compositionality. figure 1 shows a motivating example for table-based fact-checking, where the evidence used for verifying the statement comes from a semi-structured table. we can see that correctly verifying the statement \u201cin 2004, the score is less than 270\u201d requires a system to not only discover the connections between tokens in the statement and the table, but more importantly understand the meaning of logical operations and how they interact in a structural way to form a whole."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/2021.acl-long.62.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "fake news can easily misguide public opinion, cause the crisis of confidence,\r and disturb the social order (vosoughi et al., 2018).\r it is well known that fake news exerted an influence\r in the past 2016 us presidential elections (allcott\r and gentzkow, 2017). thus, it is very important\r to develop effective methods for early fake news\r detection based on the textual content of the news\r document."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "anecdote"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we propose a novel end-to-end graph neural model comparenet which compares the news to the external knowledge through entities for fake news detection."
                    ],
                    "Model Means": [
                        "evidence retrieval",
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                }
            ]
        },
        "year": 2021,
        "annotator_2": {
            "narratives": [
                {
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "type": "automated external fact-checking"
                },
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "type": "vague identification"
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "with the rapid development of the internet, there are increasingly huge opportunities for fake news production, dissemination and consumption. fake news are news documents that are intentionally and verifiably false, and could mislead readers (allcott and gentzkow, 2017). fake news can easily misguide public opinion, cause the crisis of confidence, and disturb the social order (vosoughi et al., 2018). it is well known that fake news exerted an influence in the past 2016 us presidential elections (allcott and gentzkow, 2017). thus, it is very important to develop effective methods for early fake news detection based on the textual content of the news document."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific articles"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we propose a novel end-to-end graph neural model comparenet which compares the news to the external knowledge through entities for fake news detection."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                }
            ]
        }
    },
    "http://www.vldb.org/pvldb/vol13/p2508-karagiannis.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted internal fact-checking",
                    "Data Subjects": [
                        "technical writers"
                    ],
                    "Data Actors": [
                        "technical writers",
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "human in the loop"
                    ],
                    "Application Means": [
                        "maintain consistency with kb"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "data is often disseminated in the form of text reports, summarizing the most important statistics. for authors of such documents, it is time-consuming and tedious to ensure the correctness of each single claim. nevertheless, erroneous claims about data are not acceptable in many scenarios as each mistake can have dire consequences. those consequences reach from retractions (in case of scientific papers [18]) to legal implications (in case of business or health reports [3])."
                    ],
                    "Data Subjects": [
                        "technical writers"
                    ],
                    "Data Actors": [
                        "technical writers"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we present scrutinizer, a system that helps teams of fact checkers to verify consistency of text and data efficiently [26]"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Application Means": [
                        "maintain consistency with kb"
                    ]
                },
                {
                    "Quotes (what)": [
                        "scrutinizer generates suggestions for query translations\r via classifiers."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Data Actors": [
                        "technical writers",
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "human in the loop"
                    ],
                    "type": "assisted internal fact-checking"
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "data is often disseminated in the form of text reports, summarizing the most important statistics. for authors of such documents, it is time-consuming and tedious to ensure the correctness of each single claim. nevertheless, erroneous claims about data are not acceptable in many scenarios as each mistake can have dire consequences. those consequences reach from retractions (in case of scientific papers [18]) to legal implications (in case of business or health reports [3])."
                    ],
                    "Data Subjects": [
                        "technical writers"
                    ],
                    "Data Actors": [
                        "technical writers"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ],
                    "Citation support": [
                        "scientific articles"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we present scrutinizer, a system that helps teams of fact checkers to verify consistency of text and data efficiently [26"
                    ],
                    "Data Actors": [
                        "professional journalists",
                        "technical writers"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "human in the loop"
                    ],
                    "Application Means": [
                        "maintain consistency with kb"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ]
                },
                {
                    "Quotes (why)": [
                        "our goal is to minimize overheads for human workers. those overheads are determined by multiple factors."
                    ],
                    "Data Actors": [
                        "professional journalists",
                        "technical writers"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ]
                },
                {
                    "Quotes (what)": [
                        "scrutinizer generates suggestions for query translations via classifiers."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        }
    },
    "http://ceur-ws.org/Vol-2125/paper_143.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted external fact-checking",
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "triage claims"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the ordinary internet user, however, contends with an overwhelming amount of information, which makes the task of determining the accuracy and integrity of the claims all the more onerous. additionally, users usually want their beliefs to be confirmed by information [18,34]. the confluence of vast amounts of information and such confirmation bias, thus, can create a society where unverified information runs amok masquerading as facts."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers",
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (why)": [
                        "comprehensive manual fact-checking is highly tedious and, in light of the sheer volume of information, infeasible. to overcome this hurdle, several approaches to automated fact-checking have been proposed in the nascent field of computational journalism [5,8]."
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this work, our focus is on recognizing \u201ccheck-worthy\u201d statements. accurate identification of such statements will benefit the fact-checking and verification processes that follow, independent of the specific techniques used therein."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "triage claims"
                    ]
                }
            ]
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [
                {
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "triage claims"
                    ],
                    "type": "assisted external fact-checking",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the ordinary internet user, however, contends with an overwhelming amount of information, which makes the task of determining the accuracy and integrity of the claims all the more onerous. additionally, users usually want their beliefs to be confirmed by information [18,34]. the confluence of vast amounts of information and such confirmation bias, thus, can create a society where unverified information runs amok masquerading as facts"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific articles"
                    ]
                },
                {
                    "Quotes (why)": [
                        "comprehensive manual fact-checking is highly tedious and, in light of the sheer volume of information, infeasible. to overcome this hurdle, several approaches to automated fact-checking have been proposed in the nascent field of computational journalism [5,8]."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific articles"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this work, our focus is on recognizing \u201ccheck-worthy\u201d statements. accurate identification of such statements will benefit the fact-checking and verification processes that follow, independent of the specific techniques used therein."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "triage claims"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/D19-6616.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague opposition",
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the nowadays vast amounts of textual information, its ease of sharing and its error pronesses\r call for automatic means of fact checking (thorne\r et al., 2018a)."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "past work"
                    ]
                },
                {
                    "Quotes (what)": [
                        "automated fact checking is the assignment of a truth value to a given (factual) statement, also referred to as a claim. such an assignment by itself lacks interpretability, thus it is desirable to have access to the evidence used to reach an assignment (vlachos and riedel, 2014)."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ]
                }
            ]
        },
        "year": 2019,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "type": "vague opposition",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the nowadays vast amounts of textual information, its ease of sharing and its error pronesses call for automatic means of fact checking (thorne et al., 2018a)."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "previous work"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we present our system description for the builder phase of the second version of this challenge (fever 2.0)."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/2020.acl-main.97.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague opposition",
                    "Data Subjects": [
                        "social media users",
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval",
                        "justification/explanation production"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the increasing popularity of social media has brought unprecedented challenges to the ecology of information dissemination, causing rampancy of a large volume of false or unverified claims, like extreme news, hoaxes, rumors, fake news, etc. research indicates that during the us presidential election (2016), fake news accounts for nearly 6% of all news consumption, where 1% of users are exposed to 80% of fake news, and 0.1% of users are responsible for sharing 80% of fake news (grinberg et al., 2019), and democratic elections are vulnerable to manipulation of the false or unverified claims on social media (aral and eckles, 2019), which renders the automatic verification of claims a crucial problem"
                    ],
                    "Data Subjects": [
                        "professional journalists",
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we propose a transparent and interpretable scheme that incorporates decision tree model into co-attention networks, which not only discovers evidence for explainable claim verification (section 4.4.3) but also provides interpretation for the discovery process of evidence through the decision conditions (section 4.4.2)."
                    ],
                    "Model Means": [
                        "evidence retrieval",
                        "classify/score veracity",
                        "justification/explanation production"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval",
                        "justification/explanation production"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores",
                        "gather and present evidence"
                    ],
                    "type": "vague debunking",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the increasing popularity of social media has brought unprecedented challenges to the ecology of information dissemination, causing rampancy of a large volume of false or unverified claims, like extreme news, hoaxes, rumors, fake news, etc. research indicates that during the us presidential election (2016), fake news accounts for nearly 6% of all news consumption, where 1% of users are exposed to 80% of fake news, and 0.1% of users are responsible for sharing 80% of fake news (grinberg et al., 2019), and democratic elections are vulnerable to manipulation of the false or unverified claims on social media (aral and eckles, 2019), which renders the automatic verification of claims a crucial problem."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific paper"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we propose a transparent and interpretable scheme that incorporates decision tree model into co-attention networks, which not only discovers evidence for explainable claim verification (section 4.4.3) but also provides interpretation for the discovery process of evidence through the decision conditions (section 4.4.2)."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval",
                        "justification/explanation production"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores",
                        "gather and present evidence"
                    ]
                }
            ]
        }
    },
    "https://arxiv.org/pdf/2009.07698.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague debunking",
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores",
                        "identify multimodal inconsistencies"
                    ],
                    "Ends": [
                        "limit ai-generated misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the rapid progression of generative models in both computer vision (goodfellow et al., 2014; zhang et al., 2017, 2018; choi et al., 2018) and natural language processing (jozefowicz et al., 2016; radford et al., 2018, 2019) has led to the increasing likelihood of realistic-looking news articles generated by artificial intelligence (ai). the malicious use of such technology could present a major societal problem. zellers et al. (2019) report that humans are easily deceived by its ai-generated propaganda."
                    ],
                    "Data Subjects": [
                        "professional journalists"
                    ],
                    "Ends": [
                        "limit ai-generated misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we present the first line of defence against neural fake news with images and captions"
                    ],
                    "Ends": [
                        "limit ai-generated misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "premised on the assumption that the adversarial text generator is unknown beforehand, we propose to evaluate articles based on the semantic consistency between the linguistic and visual components"
                    ],
                    "Application Means": [
                        "identify multimodal inconsistencies"
                    ]
                },
                {
                    "Quotes (what)": [
                        "besides evaluating the semantic relevance of images and captions to the article, didan also exploits the co-occurrences of named entities in the article and captions to determine the authenticity score."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "type": "vague debunking",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit ai-generated misinformation"
                    ],
                    "Application Means": [
                        "identify multimodal inconsistencies"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the rapid progression of generative models in both computer vision (goodfellow et al., 2014; zhang et al., 2017, 2018; choi et al., 2018) and natural language processing (jozefowicz et al., 2016; rad- ford et al., 2018, 2019) has led to the increasing likelihood of realistic-looking news articles generated by artificial intelligence (ai). the malicious use of such technology could present a major societal problem. zellersetal.(2019)report that humans are easily deceived by its ai-generated propaganda."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit ai-generated misinformation"
                    ],
                    "Citation support": [
                        "scientific articles"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we present the first line of defence against neural fake news with images and captions."
                    ],
                    "Ends": [
                        "limit ai-generated misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "premised on the assumption that the adversarial text generator is unknown beforehand, we propose to evaluate articles based on the semantic consistency between the linguistic and visual components."
                    ],
                    "Application Means": [
                        "identify multimodal inconsistencies"
                    ],
                    "Ends": [
                        "limit ai-generated misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "to circumvent this problem, we present didan, a simple yet surprisingly effective approach which exploits possible semantic inconsistencies between the text and image/captions to detect machine- generated articles."
                    ],
                    "Application Means": [
                        "identify multimodal inconsistencies"
                    ],
                    "Ends": [
                        "limit ai-generated misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "besides evaluating the semantic relevance of images and captions to the article, didan also exploits the co-occurrences of named entities in the article and captions to determine the authenticity score. the authenticity score can be thought of as the probability that an article is human-generated."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide labels/veracity scores"
                    ]
                },
                {
                    "Quotes (what)": [
                        "to study this threat, we construct the neural- news dataset which contains both human and machine-generated articles."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/2020.emnlp-main.628.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague opposition",
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "type": "scientific curiosity",
                    "Data Actors": [
                        "scientists"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "Ends": [
                        "develop knowledge of nlp/language"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "with the overwhelming information available on the internet, fact verification has become crucial for many applications such as detecting fake news, rumors, and political deception (rashkin et al., 2017; thorne et al., 2018; goodrich et al., 2019; vaibhav et al., 2019; krysci \u00b4 nski et al. \u00b4 , 2019), among others"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (why)": [
                        "in addition to its importance in applications, the task presents research challenges of fundamental interests\u2014the problem inherently involves both informal inference based on language understanding (dagan et al., 2005; maccartney and manning, 2009, 2008; bowman et al., 2015, 2016) and symbolic operations such as mathematical operations (e.g., count and max)."
                    ],
                    "Data Actors": [
                        "scientists"
                    ],
                    "Ends": [
                        "develop knowledge of nlp/language"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we construct the graph attention verification networks, which are designed to fuse differ- ent sources of evidences from verbalized pro- gram execution, program structures, and the original statements and tables, to make the fi- nal verification decision"
                    ],
                    "Model Means": [
                        "evidence retrieval",
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Model Owners": [
                        "scientists"
                    ],
                    "Ends": [
                        "limit misinformation",
                        "develop knowledge of nlp/language"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "type": "scientific curiosity"
                },
                {
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "type": "vague opposition",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "with the overwhelming information available on the internet, fact verification has become crucial for many applications such as detecting fake news, rumors, and political deception (rashkin et al., 2017; thorne et al., 2018; goodrich et al., 2019; vaibhav et al., 2019; krys \u0301cin \u0301ski et al., 2019), among others."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific articles"
                    ]
                },
                {
                    "Quotes (why)": [
                        "in addition to its importance in applications, the task presents research challenges of fundamental interests\u2014the problem inherently involves both informal inference based on language understanding (dagan et al., 2005; maccartney and manning, 2009, 2008; bowman et al., 2015, 2016) and symbolic operations such as mathematical operations (e.g., count and max)."
                    ],
                    "Model Owners": [
                        "scientists"
                    ],
                    "Ends": [
                        "develop knowledge of nlp/language"
                    ]
                },
                {
                    "Quotes (what)": [
                        "to effectively enable symbolic operations and integrate them into language-based inference models, we propose a framework centered around programs, i.e., logical forms that can be executed to find evidences from structured data."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ]
                },
                {
                    "Quotes (what)": [
                        "built on that, we propose graph-based verification net- work to fuse different sources of evidences from verbalized program execution, together with the original statements and tables, to support the final verification decision."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/2020.lrec-1.849.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "automated external fact-checking",
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "avoid biases of human fact-checkers",
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the advancements in information technology have led to a rapid accumulation of textual content available online. while this has many positive implications, we are faced with the challenge of sifting truth from falsehood. fact checking, the task of determining whether a given claim is true or false, has thus become an active area of research recently (vlachos and riedel, 2014)"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "past work"
                    ]
                },
                {
                    "Quotes (why)": [
                        "manual fact checking is not scalable due to the limitations imposed by the speed and capacity of human fact checkers (sharma et al., 2019). as importantly, it is susceptible to human biases (ciampaglia et al., 2015)."
                    ],
                    "Ends": [
                        "avoid biases of human fact-checkers"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (why)": [
                        "researchers began to address the shortcomings by automating the fact checking process"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "this is the largest fact checking dataset of real claims and evidence documents to date; it will allow the development of fact checking systems that can effectively process claims that occur in the real world."
                    ],
                    "Model Means": [
                        "evidence retrieval",
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "type": "automated external fact-checking",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the advancements in information technology have led to a rapid accumulation of textual content available online. while this has many positive implications, we are faced with the challenge of sifting truth from falsehood."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (why)": [
                        "initial approaches to fact checking were manual, as done on websites such as politifact.com, factcheck. org, and snopes.com. as expected, manual fact checking is not scalable due to the limitations imposed by the speed and capacity of human fact checkers (sharma et al., 2019). as importantly, it is susceptible to human biases (ciampaglia et al., 2015)."
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "avoid biases of human fact-checkers"
                    ]
                },
                {
                    "Quotes (why)": [
                        "researchers began to address the shortcomings by automating the fact checking process."
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this work, we present wikifactcheck-english1, a novel large-scale corpus for fact checking. it consists of 124,821 triples consisting of a real-world claim, its context and a cited evidence document extracted from the en- glish wikipedia; 34,783 of the triples are accompanied by a manually written claim that is refuted by the given evidence document (see an example entry in table 2). this dataset was designed to be as realistic as possible, so that the fact-checkers trained on this dataset can handle real claims effectively."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "this is the largest fact checking dataset of real claims and evidence documents to date; it will allow the development of fact checking systems that can effectively process claims that occur in the real world."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/2021.findings-acl.226.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the rapid growth of social media has created fertile soil for the emergence and fast spread of fake\r news (zhao et al., 2015), resulting in serious consequences. for example, during the u.s. 2016 presidential election, the most popular fake news was\r more widely spread than the most popular authentic news on facebook, which confused people and\r broke the authenticity balance of the news ecosystem (shu et al., 2017). to mitigate the negative\r effects caused by fake news, it is crucial to detect\r fake news on social media automatically."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                }
            ]
        },
        "year": 2021,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "type": "vague identification",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the rapid growth of social media has created fertile soil for the emergence and fast spread of fake news (zhao et al., 2015), resulting in serious conse- quences. for example, during the u.s. 2016 presidential election, the most popular fake news was more widely spread than the most popular authentic news on facebook, which confused people and broke the authenticity balance of the news ecosystem (shu et al., 2017). to mitigate the negative effects caused by fake news, it is crucial to detect fake news on social media automatically."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific articles"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we propose a novel end-to-end approach to detect fake news on social media only using the text and the attached image, without any extra information and auxiliary tasks."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/2021.acl-long.133.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit ai-generated misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "in recent years, generative neural network models in natural language processing (zellers et al.,\r 2019) and computer vision (choi et al., 2018) have\r become the frontier for malicious actors to controllably generate misinformation at scale. these\r realistic-looking ai-generated \u201cfake news\u201d have\r been shown to easily deceive humans, and it is,\r thus, critical for us to develop robust verification\r techniques against machine-generated fake news\r (tan et al., 2020; zellers et al., 2019; kaliyar\r et al., 2020)."
                    ],
                    "Ends": [
                        "limit ai-generated misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we propose a new task: finegrained, knowledge element-level cross-media information consistency checking"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                }
            ]
        },
        "year": 2021,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "type": "vague identification",
                    "Ends": [
                        "limit ai-generated misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "in recent years, generative neural network models in natural language processing (zellers et al., 2019) and computer vision (choi et al., 2018) have become the frontier for malicious actors to controllably generate misinformation at scale. these realistic-looking ai-generated \u201cfake news\u201d have been shown to easily deceive humans, and it is, thus, critical for us to develop robust verification techniques against machine-generated fake news (tan et al., 2020; zellers et al., 2019; kaliyar et al., 2020)."
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit ai-generated misinformation"
                    ],
                    "Citation support": [
                        "scientific articles"
                    ]
                },
                {
                    "Quotes (what)": [
                        "in this paper, we propose a new task: fine- grained, knowledge element-level cross-media information consistency checking. the task involves treating the entire multimedia news article as one whole interconnected claim, where the goal is to detect misinformative kes across the image, caption, and body text, as revealed by inconsistencies with respect to itself, or to background knowledge. this ke-level detection approach directly points out the fake pieces of information in the news, allowing for better explainability."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "limit ai-generated misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "a major challenge to performing ke level misinformation detection is the lack of training data. hence, we additionally propose a novel approach to generate noisy training data automatically since existing fake news generators (zellers et al., 2019) do not track the specific pieces of information generated that are fake."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                }
            ]
        }
    },
    "https://arxiv.org/pdf/1902.11116.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted knowledge curation",
                    "Data Actors": [
                        "engineers and curators"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Citation Support for Narratives": [
                        "sense of threat"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "wikipedia is playing an increasingly important role as a \u201cneutral\u201d arbiter of the factual accuracy of information published in the web. search engines like google systematically pull content from wikipedia and display it alongside search results [38], while large social platforms have started experimenting with links to wikipedia articles, in an effort to tackle the spread of disinformation [37]."
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (why)": [
                        "in reality, almost all wikipedia articles contain at least some unverified claims, and while high quality articles may cite hundreds of sources, recent estimates suggest that the proportion of articles with few or no references can be substantial [35]."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we explore this problem throughout this paper by focusing on two tasks: (1) citation need: identifying which statements need a citation. (2) citation reason: identifying why a citation is needed."
                    ],
                    "Data Actors": [
                        "engineers and curators"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                }
            ]
        },
        "year": 2019,
        "annotator_2": {
            "narratives": [
                {
                    "Data Actors": [
                        "engineers and curators"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ],
                    "type": "assisted knowledge curation",
                    "Citation Support for Narratives": [
                        "scientific research"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "wikipedia is playing an increasingly important role as a \u201cneutral\u201d arbiter of the factual accuracy of information published in the web. search engines like google systematically pull content from wikipedia and display it alongside search results [38], while large social platforms have started experimenting with links to wikipedia articles, in an effort to tackle the spread of disinformation [37]."
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ],
                    "Citation support": [
                        "scientific articles"
                    ]
                },
                {
                    "Quotes (why)": [
                        "while the role citations serve to meet this requirement is straight- forward, the process by which editors determine which claims require citations, and why those claims need citations, are less well understood. in reality, almost all wikipedia articles contain at least some unverified claims, and while high quality articles may cite hundreds of sources, recent estimates suggest that the proportion of articles with few or no references can be substantial [35]. while as of february 2019 there exists more than 350, 000 articles with one or more {citation needed} flag, we might be missing many more."
                    ],
                    "Data Actors": [
                        "engineers and curators"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we explore this problem throughout this paper by focusing on two tasks: (1) citation need: identifying which statements need a citation. (2) citation reason: identifying why a citation is needed."
                    ],
                    "Data Actors": [
                        "engineers and curators"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims"
                    ],
                    "Ends": [
                        "increase veracity of published content"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/2020.findings-emnlp.43.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted external fact-checking",
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "triage claims",
                        "identify claims"
                    ],
                    "Citation Support for Narratives": [
                        "automation"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "misinformation is being spread online at ever increasing rates (del vicario et al., 2016) and has been identified as one of society\u2019s most pressing issues by the world economic forum (howell et al., 2013)."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (why)": [
                        "the rate at which misinformation is introduced and spread vastly outpaces the ability of any organization to perform fact checking, so only the most salient claims are checked. this obviates the need for being able to automatically find check-worthy content online and verify it."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "each task is concerned with a shared underlying problem: detecting claims which warrant further verification"
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "triage claims"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "triage claims",
                        "identify claims"
                    ],
                    "type": "assisted external fact-checking",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "misinformation is being spread online at ever increasing rates (del vicario et al., 2016) and has been identified as one of society\u2019s most pressing is- sues by the world economic forum (howell et al., 2013). in response, there has been a large increase in the number of organizations performing fact checking (graves and cherubini, 2016)."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific articles"
                    ]
                },
                {
                    "Quotes (why)": [
                        "however, the rate at which misinformation is introduced and spread vastly outpaces the ability of any organization to perform fact checking, so only the most salient claims are checked. this obviates the need for being able to automatically find check-worthy content online and verify it."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "triage claims"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (what)": [
                        "each task is concerned with a shared underlying problem: detecting claims which warrant further verification."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "identify claims",
                        "triage claims"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/2020.coling-main.476.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague debunking",
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide aggregates of social media comments",
                        "vague persuasion"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "online rumor perhaps is one of the most prevalent social diseases in the era of social media. an immediate\r example we are witnessing is the unprecedented information disorder represented by various rumors,\r conspiracy theories, hoaxes, fake news, etc. in parallel with the worldwide pandemic of covid19. in\r different places, a number of people were hospitalized or even died for drinking bootleg alcohol to prevent\r coronavirous infection, resulting from a false rumor attack on gullible public claiming that \u201csmoking,\r methanol or cocaine can cure for the virus\u201d"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "anecdote"
                    ]
                },
                {
                    "Quotes (why)": [
                        "automatic rumor debunking is at the core of battle against\r such massive disorder of information especially in the midst of crisis."
                    ],
                    "Application Means": [
                        "vague persuasion"
                    ]
                },
                {
                    "Quotes (what)": [
                        "the stances relative to parent nodes implying the underlying credibility\r of the claim."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide aggregates of social media comments"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide aggregates of social media comments"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "type": "vague debunking",
                    "Data Subjects": [
                        "social media users"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "online rumor perhaps is one of the most prevalent social diseases in the era of social media. an immediate example we are witnessing is the unprecedented information disorder represented by various rumors, conspiracy theories, hoaxes, fake news, etc. in parallel with the worldwide pandemic of covid19. in different places, a number of people were hospitalized or even died for drinking bootleg alcohol to prevent coronavirous infection, resulting from a false rumor attack on gullible public claiming that \u201csmoking, methanol or cocaine can cure for the virus\u201d1. automatic rumor debunking is at the core of battle against such massive disorder of information especially in the midst of crisis."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat+news article"
                    ]
                },
                {
                    "Quotes (why)": [
                        "rumor debunking aims to determine the veracity of a given topic or a claim. fact-checking websites, such as snopes.com and politifact.com, employ manual verification and investigative journalism, which is prone to low efficiency and poor coverage."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "to this end, we propose to enhance the representation by exploring the stances towards the same target utilizing the associated contextual information."
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "provide aggregates of social media comments"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/W18-5526.pdf": {
        "annotator_1": {
            "narratives": [],
            "quotes": []
        },
        "year": 2018,
        "annotator_2": {
            "narratives": [],
            "quotes": [
                {
                    "Quotes (what)": [
                        "the system is evaluated on the correct labeling of the claims as \u201csupports,\u201d \u201crefutes,\u201d or \u201cnot enough info\u201d (nei) as well as on valid evidence to support the label (except in the case of \u201cnei\u201d). each claim can have multiple evidence sets, but only one set needs to be found so long as the correct label is ap- plied. figure 1 gives an example of a claim along with the evidence sets that support it, as well as a claim and the evidence that refutes it. we split the task into three distinct modules, with each module building on the data of the pre- vious one."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/2020.fever-1.2.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "automated external fact-checking",
                    "Model Means": [
                        "classify/score stance",
                        "classify/score veracity"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Citation Support for Narratives": [
                        "scientific research"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "although fake news is not an emerging phenomenon and has been documented throughout\r history, the prevalence and wide spread of misinformation over the internet has captured significant\r proportion of public attention in recent years. this\r is in part linked to the low barrier for content generation through the advent of the internet and social media (allcott and gentzkow, 2017) and the\r fact that false news spread faster than true news\r (vosoughi et al., 2018) rendering it increasingly\r dangerous to public discourse."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific papers"
                    ]
                },
                {
                    "Quotes (why)": [
                        "since manual fact-checking by human experts\r does not scale well with the amount of information shared on the web, there is a growing body\r of work in recent years aimed at developing automatic tools to target fake news, misinformation\r and credibility of content on social media in general (rubin et al., 2016; el ballouli et al., 2017;\r baly et al., 2018a,b; wang et al., 2018; saleh et al.,\r 2019; zhang et al., 2019)"
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "Citation support": [
                        "past work"
                    ]
                },
                {
                    "Quotes (what)": [
                        "this work explores the application of tex- tual entailment in news claim verification and stance prediction using a new corpus in ara- bic."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "classify/score stance"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Application Means": [
                        "supplant human fact-checkers"
                    ],
                    "type": "automated external fact-checking",
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Model Means": [
                        "classify/score stance",
                        "classify/score veracity"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "although fake news is not an emerging phenomenon and has been documented throughout history, the prevalence and wide spread of misinformation over the internet has captured significant proportion of public attention in recent years. this is in part linked to the low barrier for content generation through the advent of the internet and social media (allcott and gentzkow, 2017) and the fact that false news spread faster than true news (vosoughi et al., 2018) rendering it increasingly dangerous to public discourse. the widespread exposure in the u.s. for example has been reported by researchers who found that the average american encountered between one and three stories from known publishers of fake news during the month before the 2016 election (allcott and gentzkow, 2017)."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "scientific articles"
                    ]
                },
                {
                    "Quotes (why)": [
                        "since manual fact-checking by human experts does not scale well with the amount of information shared on the web, there is a growing body of work in recent years aimed at developing au- tomatic tools to target fake news, misinformation and credibility of content on social media in gen- eral (rubin et al., 2016; el ballouli et al., 2017; baly et al., 2018a,b; wang et al., 2018; saleh et al., 2019; zhang et al., 2019)."
                    ],
                    "Application Means": [
                        "supplant human fact-checkers"
                    ]
                },
                {
                    "Quotes (what)": [
                        "as such, this work contributes to recent efforts targeting arabic by introducing a new publicly available corpus in arabic that is suitable to study claim verification and semantic entailment (katz, 1972)."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "classify/score stance"
                    ]
                }
            ]
        }
    },
    "https://link.springer.com/content/pdf/10.1007/978-3-030-58219-7.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "vague identification",
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (what)": [
                        "the purpose of the 2020 edition was to foster the development of technology that would enable the (semi-)automatic verification of claims posted in social media, in particular twitter"
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Application Means": [
                        "identify claims"
                    ]
                },
                {
                    "Quotes (what)": [
                        "the four\r tasks are defined as follows:\r task 1. check-worthiness estimation for tweets. predict which tweet from a\r stream of tweets on a topic should be prioritized for fact-checking.\r task 2. verified claim retrieval: given a check-worthy tweet, and a set of claims\r previously checked, determine whether the claim in the tweet has been factchecked already.\r task 3. evidence retrieval. given a check-worthy claim in a tweet on a specific topic and a set of text snippets extracted from potentially-relevant web\r pages, return a ranked list of evidence snippets for the claim.\r task 4. claim verification. given a check-worthy claim in a tweet and a set of\r potentially-relevant web pages, estimate the veracity of the claim.\r task 5. complements the lab. it is as task 1, but on political debates ad speeches\r rather than on tweets: given a debate segmented into sentences, together with\r speaker information, prioritize sentences for fact-checking."
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ]
                }
            ]
        },
        "year": 2020,
        "annotator_2": {
            "narratives": [
                {
                    "Data Subjects": [
                        "social media users",
                        "public figures/politicians"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "triage claims",
                        "identify claims"
                    ],
                    "type": "vague identification",
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "the purpose of the 2020 edition was to foster the development of technology that would enable the (semi-)automatic verification of claims posted in social media, in particular twitter.2 we turn our attention to twitter because information posted on that platform is not checked by an authoritative entity before publi- cation and such information tends to disseminate very quickly.3 moreover, social media posts lack context due to their short length and conversational nature; thus, identifying a claim\u2019s context is sometimes key for enabling effective fact- checking [13]."
                    ],
                    "Data Subjects": [
                        "social media users"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "past work"
                    ]
                },
                {
                    "Quotes (what)": [
                        "the full identification and verification pipeline is displayed in figure 1. the four tasks are defined as follows: task 1 check-worthiness estimation for tweets. predict which tweet from a stream of tweets on a topic should be prioritized for fact-checking. task 2 verified claim retrieval: given a check-worthy tweet, and a set of previously- checked claims, determine whether the claim in the tweet has been fact- checked already. task 3 evidence retrieval. given a check-worthy claim in a tweet on a specific topic and a set of text snippets extracted from potentially-relevant webpages, return a ranked list of evidence snippets for the claim. task 4 claim verification. given a check-worthy claim in a tweet and a set of potentially-relevant web pages, estimate the veracity of the claim. task 5 complements the lab. it is as task 1, but on political debates ad speeches rather than on tweets: given a debate segmented into sentences, to- gether with speaker information, prioritize sentences for fact-checking."
                    ],
                    "Data Subjects": [
                        "public figures/politicians",
                        "social media users"
                    ],
                    "Model Means": [
                        "classify/score veracity",
                        "evidence retrieval"
                    ],
                    "Application Means": [
                        "identify claims",
                        "triage claims"
                    ]
                }
            ]
        }
    },
    "https://aclanthology.org/2021.acl-short.86.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "assisted external fact-checking",
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "curbing the spread of fake news and misinformation on the web has become an important societal\r challenge."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (why)": [
                        "with the aim of assisting fact-checkers, researchers in nlp have sought to develop computational approaches to fact-checking (vlachos and riedel, 2014; wang, 2017; perez-rosas et al. \u00b4 , 2018)."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Citation support": [
                        "past work"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we release a multilingual fact-checking benchmark x-fact, which includes 31,189 short statements labeled for factual correctness"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        },
        "year": 2021,
        "annotator_2": {
            "narratives": [
                {
                    "Ends": [
                        "limit misinformation"
                    ],
                    "type": "assisted external fact-checking",
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "curbing the spread of fake news and misinforma- tion on the web has become an important societal challenge. several fact-checking initiatives, such as politifact,1 expend a significant amount of manual labor to investigate and determine the truthfulness of viral statements made by public figures, organi- zations, and social media users. of course, since this process is time-consuming, often, a large num- ber of falsified statements go unchecked."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ],
                    "Citation support": [
                        "sense of threat"
                    ]
                },
                {
                    "Quotes (why)": [
                        "with the aim of assisting fact-checkers, researchers in nlp have sought to develop computational approaches to fact-checking (vlachos and rie- del, 2014; wang, 2017; pe \u0301rez-rosas et al., 2018). many such works use the fever dataset, which contains claims extracted from wikipedia documents (thorne et al., 2018)."
                    ],
                    "Data Actors": [
                        "professional journalists"
                    ],
                    "Model Means": [
                        "human in the loop"
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (why)": [
                        "although misinformation transcends countries and languages (bradshaw and howard, 2019; islam et al., 2020), much of the recent work focuses on claims and statements made in english. developing automated fact checking (afc) systems in other languages is much more challenging, the primary reason being the absence of a manually annotated benchmark dataset for those languages."
                    ],
                    "Ends": [
                        "limit misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "we release a multilingual fact-checking benchmark x-fact, which includes 31,189 short statements labeled for factual correctness and covers 25 typologically diverse languages across 11 language families. x-fact is an order of magnitude larger than any other multilingual dataset available for fact checking"
                    ],
                    "Model Means": [
                        "classify/score veracity"
                    ]
                }
            ]
        }
    },
    "https://arxiv.org/pdf/2104.05893.pdf": {
        "annotator_1": {
            "narratives": [
                {
                    "type": "adversarial research",
                    "Data Actors": [
                        "scientists"
                    ],
                    "Model Means": [
                        "generate claims"
                    ],
                    "Application Means": [
                        "produce misinformation"
                    ],
                    "Citation Support for Narratives": [
                        "investigative"
                    ],
                    "Ends": [
                        "limit ai-generated misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "misinformation has reached new heights as sophisticated ai-based tools have come into the spotlight."
                    ],
                    "Ends": [
                        "limit ai-generated misinformation"
                    ]
                },
                {
                    "Quotes (why)": [
                        "real images of people and events get reappropriated and used out of context to illustrate false events and misleading narratives by misrepresenting who is in the image, what is the context in which they appear or where the event takes place."
                    ],
                    "Ends": [
                        "limit ai-generated misinformation"
                    ]
                },
                {
                    "Quotes (what)": [
                        "here, we explore whether such a threat can be automated. we show that real world images can be automatically matched to captions to generate false but compelling news stories, a threat scenario that may lead to larger-scale image repurposing."
                    ],
                    "Data Actors": [
                        "scientists"
                    ],
                    "Model Means": [
                        "generate claims"
                    ],
                    "Application Means": [
                        "produce misinformation"
                    ]
                }
            ]
        },
        "year": 2021,
        "annotator_2": {
            "narratives": [
                {
                    "Data Subjects": [
                        "technical writers"
                    ],
                    "Model Means": [
                        "generate claims"
                    ],
                    "Application Means": [
                        "produce misinformation"
                    ],
                    "Citation Support for Narratives": [
                        "investigative"
                    ],
                    "type": "adversarial research",
                    "Ends": [
                        "limit ai-generated misinformation"
                    ]
                }
            ],
            "quotes": [
                {
                    "Quotes (why)": [
                        "misinformation has reached new heights as sophisticated ai-based tools have come into the spotlight. for instance, it has become easy to generate images of people who \u201cdo not exist\u201d1 and create realistic deepfakes of existing people (victor, 2021). recent language models have become better at fooling people into believing that generated texts are from real people (hao, 2020). however, simple and cheap image repurposing remains one of the most widespread and effective forms of misinfor- mation (fazio, 2020)."
                    ],
                    "Ends": [
                        "limit ai-generated misinformation"
                    ],
                    "Citation support": [
                        "scientific articles"
                    ]
                },
                {
                    "Quotes (what)": [
                        "specifically, real images of people and events get reappropriated and used out of context to illustrate false events and mis- leading narratives by misrepresenting who is in the image, what is the context in which they appear, or where the event takes place. this method is effective since augmenting a story with an image has been shown to increase user engagement and make false stories seem true (fenn et al., 2019). here, we explore whether such a threat can be au- tomated. we show that real world images can be automatically matched to captions to generate false but compelling news stories, a threat scenario that may lead to larger-scale image repurposing."
                    ],
                    "Data Subjects": [
                        "technical writers"
                    ],
                    "Model Means": [
                        "generate claims"
                    ],
                    "Application Means": [
                        "produce misinformation"
                    ]
                }
            ]
        }
    }
}