Progressive Politics in Minnesota, the Nation, and the World

Occam's Razor: GOP Pants on Fire Ratings Deserved

Category: GOP Politics

Posted: 02/11/11 15:01

by Dave Mindeman

Eric Ostermeier, over at the Smart Politics blog, generally puts together a fact based composite for any premise that he asserts. And his stats are compelling to his point.

However, sometimes he just needs to apply Occam's razor...(a principle that suggests we should tend towards simpler theories.)

In one of his recent examinations he wonders if there is a "left" leaning bias to the St. Petersburg Times, Politifact. The column that rates politician statements for accuracy.

Ostermeier outlines what bothers him:

A Smart Politics content analysis of more than 500 PolitiFact stories from January 2010 through January 2011 finds that current and former Republican officeholders have been assigned substantially harsher grades by the news organization than their Democratic counterparts.

One of the legitimate questions that Ostermeiers asks is in regards to the criteria of how Politifact decides which statements to examine. Which would be a legitimate question if there is supposed to be a concrete method for selecting statements. There isn't. The same question could be asked of Ostermeier as to his own selection process for the statistical data he analyzes.

You might also ask if they simply select GOP politicians more than Democrats but the answer there is that the selection is split nearly down the middle...50-50.

For those current or former political officeholders, PolitiFact has generally devoted an equal amount of time analyzing Republicans (191 statements, 50.4 percent) as they have Democrats (179 stories, 47.2 percent), with a handful of stories tracking statements by independents (9 stories, 2.4 percent).

Yet, as Ostermeier points out, the GOP statements get the worst ratings the most often.

Occam's razor. The GOP get the worst ratings because they make the worst statements.

Ostermeier concentrates on making Politifact defend their selection process, but overlooks the facts about the statements themselves. Could it be possible that the Republicans make more outrageous and indefensible assertions?

Politifact is certainly going to be drawn to statements that get the most attention and the more outrageous the statement, the more attention it gets.

Maybe Ostermeier should try another analysis. Maybe he should put Michele Bachmann into a separate category. If you remove her statements from the Republican side of the ledger, the balance will shift heavily to the left.

Occam's razor. Michele Bachmann can't tell the truth.

comments (9) permalink

DaveM

02/12/11 09:06

You are working on a study that will establish....???

Hmmm. Why bother with the study since you have already framed your conclusion. Seems like a lot of bother to have to spend time searching for corroborrating evidence. You missed your calling buddy. You should be working for the pharmaceutical industry. They pay big money for that type of reasoning.

Bryan White @ subloviate.blogspot.com/

02/12/11 05:03

DaveM, I think you're probably right that PolitiFact chooses statements from the left with an eye toward balancing the numbers of statements they rate. It bad if they rate more Republican statements, and they're probably sensitive to that appearance even if they don't care about being perceived as biased based on the severity of the ratings for conservatives. But if they're doing that then they do plainly have an ideological selection bias.

And if it makes you feel better, I am currently working on a study based on a much different method than Ostermeier's that will establish a strong probability of content bias in addition to the selection bias.

As with Ostermeier's study, it will leave open possibilities other that ideological bias. But Occam's razor isn't going to favor those alternatives.

"Unless there is a method to run every political statement through a "truth meter" you can always make an argument that the selection process is biased."

Yes, I believe that's what I said. :-) Though randomization of a limited sample is an acceptable way to mitigate the bias.

You may have noticed that I do case studies of PolitiFact stories in my spare time (and yes I definitely employ selection bias). I've located a sobering number of egregious errors in PolitiFact's fact checks, errors of all shapes and sizes. Both parties are affected at times, but the sheer number of large errors adversely affecting conservative figures bolsters the argument that PolitiFact is left-tilted.

DaveM

02/12/11 03:33

Whether or not the selection bias is relevant is really up to the reader. Personally, I would say that Politifact could find a lot more instances of "pants on fire" statements by Republicans than they do. They ignore the obvious untruths - (Michele Bachmann could fill a book with errors and distortions). Politifact generally chooses statements in which they can find statistical analysis that can lead to a stronger burden of proof. You may choose to believe that they are scouring the internet for Republican misdeeds, but I would contend that their main problem is finding a balance of Democratic statements that match the factual errors of the GOP. Unless there is a method to run every political statement through a "truth meter" you can always make an argument that the selection process is biased. The outrageous fabrications publicized during the Health Care debate by GOP opponents could have filled volumes alone.

And I still contend that if Bachmann's loose cannon statements were left out of the mix, then the Republicans would actually fare better in the other comparisons. Just a thought.

Bryan White @ subloviate.blogspot.com/

02/12/11 03:20

DaveM wrote:
"Most reporters choose their own stories. If they do a good job of reporting the story, do they still have to prove that they have no inherent bias?"

Come to think of it, the above suggests that you don't quite get selection bias. A story can easily manifest selection bias regardless of whether the reporting in the story is fair. Thus it doesn't matter whether the reporting is fair or not in finding a significant selection bias.

In experimentation, selection bias is pretty much automatic where no attempt is made to simulate random selection of samples. Whether the selection bias in story selection favors one party over another in the end result is a separate question. Ostermeier's work strongly suggests that we need much more than the automatic presumption of selection neutrality. After all, doesn't it make sense on the face of it for left-leaning reporters and editors to show more interest in statements that they find questionable? And doesn't Adair say as much (quoted by Ostermeier in his original post) regarding the types of stories that attract their interest?

Bryan White @ subloviate.blogspot.com/

02/12/11 02:34

DaveM wrote:

"I fail to understand why Politifact has to "prove" anything."

PolitiFact doesn't have to prove anything just like Fox News doesn't have to prove anything. It's a matter of public perception. As Eric pointed out, PolitiFact frames its data to encourage readers to draw conclusions about political figures that would follow *if* story selection was random. But the story selection isn't random. PolitiFact needs to justify itself only if it isn't comfortable with a partisan Fox-style audience (albeit gravitating toward the opposite political pole).

You appear to misunderstand/misapply Occam's razor, by the way. Why is it simpler to *find a large group of entities tend to lie more than another large group of entities* than to *suppose that a much smaller group of entities (PolitiFact journalists) display bias in their work*?

DaveM.

02/11/11 17:18

I fail to understand why Politifact has to "prove" anything. They examined political statements that interested them. Most reporters choose their own stories. If they do a good job of reporting the story, do they still have to prove that they have no inherent bias? Maybe I should assume you have an inherent conservative bias because your analysis deals with GOP favorable data?? But I don't, because I think you use the data in a broad enough sense that it tells something regardless of the outcome.

Eric Ostermeier @ www.smartpoliticsblog.org

02/11/11 17:09

>It would seem that most of the data says there is
> none.

What data is that? You can't use the data I published from coding PolitiFact's stories itself (noting the site attributes more false statements to the GOP) as proof that the GOP lies more and thus there is de facto no selection bias. That's circular reasoning. Note: My report did not definitively prove there is such bias, but the data published shifts the burden, I would argue, to PolitiFact. And greater transparency in their selection methodology would shine a light on this very question.

DaveM.

02/11/11 16:45

Then why ask title your post with "selection bias"? It would seem that most of the data says there is none.

Eric Ostermeier @ www.smartpoliticsblog.org

02/11/11 16:29

> Could it be possible that the Republicans
> make more outrageous and indefensible
> assertions?

FYI: this very possibility was in fact addressed in my report:

"One could theoretically argue that one political party has made a disproportionately higher number of false claims than the other, and that this is subsequently reflected in the distribution of ratings on the PolitiFact site."

« First « Previous

Next » Last »

Progressive Politics in Minnesota, the Nation, and the World

Occam's Razor: GOP Pants on Fire Ratings Deserved

Calendar

Archive

Categories

Latest posts

Comments

Search posts

Links

RSS Feeds