Is the future of fact checking automated?

Ten years ago, the concept of fact-checking in real time was perhaps a dream. The process takes time and effort, ranging from identifying claims from hourly broadcasts, cross-referencing them with available data and interviewing experts to establish facts.

But very soon, direct fact checks can flash in the chyrons under the faces of politicians and experts debating on television, all done with minimal human input.

The UK-based fact-checking organization, Full Fact, has been implementing forms of automated fact-checking since 2013. Outlet currently uses an alpha version of three automated tools working in tandem to detect and categorize real-time claims, cross-reference them with existing fact-checks and even display relevant figures for statistical claims to help fact checkers quickly assess their accuracy. Although the tool is mostly internal so far, Full Fact is collaborating with other fact-checking organizations to expand its use across languages, regions, and cultures.

“Our intention is really to bring these tools to as many users as possible, and in particular to take them to language users who have historically been understaffed in automated fact checking due to the difficulties associated with multilingual tools,” said Kate Wilkinson, senior product manager at Full Fact.

The first tool deals with damage detection. Users select a channel, whether it is BBC News or a number of others, and can watch a live transcript of what is being broadcast. As soon as a sentence appears that contains a statement – verifiable or not – the tool recognizes and labels it as belonging to one or more categories: quantity, support, rules, correlation, personal (someone does or believes in something), predictions and voices.

“The problem we’re trying to solve is that we want to make it as easy as possible for fact-checkers to identify the most important things to fact-check on a given day,” Wilkinson said.

The second tool addresses the issue of how to increase the scope of existing fact checks. This tool cross-references claims identified from the claims detection tool with existing fact checks performed by Full Fact. Here is an example from last week of the tool in action; it immediately warned fact-checkers when British Prime Minister Boris Johnson reiterated a false claim.

“Anyone involved in fact checking knows that a fact check can take 15 minutes, or it can take two weeks, depending on how easy it is to find the data you need,” Wilkinson said. “But once all that effort and that work has been invested, the result is a fact check. And unfortunately, it’s rare that just publishing a fact check kills a false statement.”

Wilkinson compared false statements in information ecosystems with weeds in a garden.

“If you go out into your garden and you do some gardening and you pull out the big weed, it won’t stop weeds in your garden,” she said. “The moment the next weed emerges, pull it out and pull it out and pull it out until the weeds slowly over time no longer come back.”

The third tool, state-checker, allows fact-checkers to quickly cross-reference direct claims about statistics with officially available statistics. So far, this tool is limited to statistics on inflation and unemployment.

“We know there are certain topics for debate, especially around statistics, where we can dramatically reduce the time it takes to verify a statistical claim,” Wilkinson said. “If you used our tool to monitor statements under a live address, it would identify and show you all the statements that could be considered, it would tell you if any of these statements have been fact-checked before and have shown “And if there was a statistical claim about unemployment or inflation, it would tell you if that claim was correct or incorrect.”

In recent years, Full Fact, with financial support from Google.org, has explored how artificial intelligence could help tackle false information and make a concerted effort to scale its tools for multilingual, global use.

“Many of the tools that have been built have been built with one context, social environment, environment or culture in mind,” Wilkinson said. “But what we are trying to achieve in this project is to serve a global audience.”

Wilkinson said the tools will soon be available to fact-checking organizations around the world and is currently being tested by the UK Statistical Office (it uses the tool to be alerted when official statistics are misused).

The fact check shops Chequeado in Argentina and Africa Check, headquartered in South Africa, implement versions of the same technology.

Leave a Comment