Fact Checking And the False Equivalence Dilemma

  • Share
  • Read Later
Justin Sullivan / Getty Images - Brooks Kraft / Corbis for Time

Last week I wrote a cover story for TIME about the factual deceptions of both Barack Obama and Mitt Romney. Many readers, particularly Obama supporters, were outraged that the story did not clearly state that one of the candidates was misleading more than the other. Peter Hart, writing for the liberal Fairness and Accuracy in Reporting, summed up many of the tweets I received. “The article kicks off with a hefty helping of false balance–the tendency to see all problems as coming more or less equally from both sides,” he wrote. “One of the most common problems with media factchecking is the need to always be balanced–no matter what is happening in reality.”

I don’t disagree with Hart’s underlying point. The role of the press in these situations is one of accountability. As a reporter, I should be calling out factual errors and deceptions of public officials, which TIME did in the magazine, both in my story and a sidebar by Alex Altman. That way, voters will be more informed about what is actually happening, which makes for an efficient democracy. It also could increase what political scientists sometimes call “reputational cost” for politicians who mislead. In a perfect world, the cost would be higher for the side that is deceiving more. It therefore follows that the press should try to figure out who is worse.

(PHOTOS: Political Pictures of the Week, Sept. 28 – Oct. 4)

I would love to be able to tell you that Mitt Romney is misleading more than Barack Obama or vice versa. (Aides to both campaigns have certainly made their cases to me.) The problem is that there is no existing mechanism for carrying this sacred duty out in real time. Furthermore, I feel I can say with confidence that the likelihood that someone believes they know who is misleading more is directly related to their own partisan feelings in this campaign. There are just too many subjective judgements that have to be made to come to any conclusion, and as I point out in my piece, we are predisposed to forgive those deceivers that share our worldviews and punish those who do not.

About a week before my piece came out, I attended a panel discussion at the National Press Club with the top fact checkers in journalism: Bill Adair of Politifact, Brooks Jackson of FactCheck.org, Glenn Kessler of the Washington Post and Jim Drinkard of the Associated Press. I asked them all if they could make a judgement about which campaign was worse. All of them said they could not. (You can watch a C-Span video of this exchange here.)

“I don’t know of any objective way to measure that,” said Jackson. “Even if we could come up with a scholarly and factual way to say that one candidate is being more deceptive than another, I think we probably wouldn’t just because it would look like we were endorsing the other candidate.”

(MORE: If Obama Won’t Defend the Last Four Years, Why Would America Give Him Another Four?)

Drinkard elaborated. “It’s a real continuum,” he said. “There are a lot of misdemeanors as well as the felonies that go on. And how [do] you devise some kind of a rating system for that?” Kessler said his experience in Washington had taught him that there was no such thing as an honest candidate. “Politicians in both parties will stretch the truth if it is in their political interest,” he said. “There is no difference between the two on that score.”

Some have tried to count up the fact-checking ratings on these sites to determine which candidate is worse. But all the fact checkers agreed that this is a flawed methodology, since it pulls from a tainted sample compiled unevenly by the fact checkers themselves. In addition, not all deceptions are equally deceiving, and different people will reach different judgements about which is worse. Do you think it was worse for President Obama to claim that Romney supports outlawing abortion even in cases of rape and incest, when Romney does not? Or for Romney to claim that Obama plans to give welfare recipients a check without any work requirement, when he does not? I don’t know how to answer that question.

Even if we pretended that these ratings gave an objective view of the campaigns, and that the gradations made by fact checkers held some scientific consistency, the differences are not as stark as partisans assume. According to one site, of the 427 rulings on Obama campaign statements, Politifact has found 119–or 28%–to be mostly false, false or “pants on fire.” Of the 183 ratings on Romney campaign statements, Politifact has found 79–or 43%–to be mostly false, false or “pants on fire.” So maybe Romney has more deception per utterance? Or maybe Obama has more total deception? I can’t tell you from those numbers.

(MORE: Lessons Learned from Watching the Presidential Debate–On Mute)

Kessler at the Washington Post has what he calls the Pinocchio tracker, which gives you the average number of Pinocchio’s for a given politician for the statements he has reviewed. Obama gets an average of 2.04 Pinocchios out of 4, while Romney gets an average of 2.35 Pinocchios out of 4. Romney has had 10 statements that received the maximum of 10 Pinocchios, compared to six statements for Obama that received the maximum. Does this mean anything? According to Kessler, not really.

In my reporting for the story, the closest I came to a system of determining worse and better on the candidate level came from Kathleen Hall Jamieson, a professor at the Annenberg Public Policy Center, who has pioneered much of the academic work on deception in presidential campaigns. She said what you really wanted to measure was consequential deceptions, meaning the level of deception that moved voters. One way of doing this would be to score every campaign ad that runs in a cycle for deception, and then weight the ads by the number of people who see them. It’s a fine idea, but difficult to do in real time, when the reputational cost is the highest for the campaigns.

In the end, my story did not make a judgement about which campaign was worse. It focused instead on the dynamics that allow politicians to get away with hypocrisy: Expressing outrage at an opponent’s deception while continuing to deceive on their own. But that doesn’t mean others can’t succeed where I came up short. If anyone has an idea for how to do this, please do drop a line. There is a good chance a working system could improve the quality of our political debates.

MORE: Obama, Romney Twist Two Crucial Issues in First Debate