National Journal Mis-Identifies Top Liberals, Conservatives in Congress Once Again: The Radical Harry Reid Edition

It happens every year, or at least so it seems (I don’t always pay attention, there are just too many lies out there).  The National Journal puts out its list of top liberals and conservatives in Congress, based on its arbitrarily-tweaked ranking methodology, and lo and behold some Democratic leader invariably shows up as “the most liberal Democrat in Congress.”  When John Kerry was running for President in 2004, he was “the most liberal Democrat”–NOT! as Scoobie Davis and others showed, including a trio of political scientists from Princeton & Stanford in a 24-page paper, “‘The Most Liberal Senator’?: Analyzing and Interpreting Congressional Roll Calls”. When Obama was the candidate in 2008, lo and behold, he was “the most liberal Democrat”–NOT! as I myself explained at some length.

Now it’s Harry Reid’s turn! That’s right, Harry Reid! More liberal than Russ Feingold last year, and just as liberal Bernie Sanders! TPM summarizes the National Journal‘s picture-heavy, fact-lite story. They’re slightly less off-the-mark with identifying the most conservative members, but since “conservative” isn’t used constantly as a demonizing term, this has less of impact politically.

Here’s TPM’s summary of NJ‘s multiple-way ties for first place “most liberal” and “most conservative” in the House and Senate, compared with their actual rankings in order via the DW-Nominate ranking system regarded as the gold standard amongst serious academics, and based on aggregating all but the most lopsided floor votes–with less than 0.5% on the losing side. (Website here.) Unfortunately, the DW-Nominate rankings for 2010 haven’t been released yet, so we have to use 2009 figures, but as we’ll see in a moment, comparing DW-Nominate rankings to NJ‘s 2009 rankings doesn’t change things very much at all (even though Reid didn’t make the “most liberal” ranking then). Reid, of course, isn’t even in the more liberal half of the Democratic caucus, while two others on the list–Cardin and Stabenow–are right on the cusp:

So, out of 9-way tie for first, National Journal managed to get exactly two Senators that ranked in the top 9 according to the DW-Nominate scale. The average DW-Nominate rank for these senators was 18.3, compared to an average rank of 5 for the actual group of the 9 most liberal senators. That’s 3.7 times higher than it should be, if National Journal‘s rankings were accurate. If we look back at National Journal‘s rankings for 2009, we discover that they did a better job in terms of getting a lower average score–but with a smaller list, so the end result is that their average was actually even higher–4.2 times higher than it should have been. So they were, in fact less accurate than the measure we get by comparing this year’s rankings with last years DW-Nominate data:

Things are even worse on the House side. Comparing the 2010 “most liberal” list to actual DW-Nominate rankings for 2009, there is not a single member on National Journal‘s list of 7 who scored in the top 7. Being from Southern California myself, I found it particularly amusing that Linda Sanchez and Judy Chew were on the list–while no one in the know in LA would even rate them as the most liberal representative from California, much less Los Angeles. Maxine Waters (ranked #3 by DW-Nominate) would take that honor easily on both counts. Pete Stark and Bob Filner (#s 6 & 7 respectively) would not be far behind statewide, while Diane Watson (#25) would also be well ahead of Chu in the LA Area. (Two other Californians–Lynn Woolsey and George Miller come in at #16 and #20 respectively.) As far as making direct comparisons are concerned, the average for this list of 7 is 23.7, 5.9 times higher than it should be:

But that’s positively psychic compared to how well National Journal did in 2009, with an average of 39.0 coming it at 8.7 times what it should have been:

Interestingly, the National Journal is far more accurate when it comes to ranking conservatives. But that still leaves them plenty wide of the mark. Their 2010 Senate top rankings gave us two of the top 8 slots in the 8-way tie for first, and their average score was “just” 3.1 times higher than it should be. Notable outliers: Johns McCain (#28) and Thune (#24).

Back in 2009, there was none of this group tie stuff, so just to make comparisons possible, I’ve listed the top five together, and of those, National Journal managed to three senators who actually belonged in the top 5. The average score was less than twice what it should have been:

They must have had Dione Warwick working for them that year!

Switching over to the House side, the gap between liberal and conservative ranking accuracy narrows considerably, even though they managed to get one of the top 5 named in their 5-way tie for first. Still, they ended up with an average 5.9 times higher than it should have been–the same as for liberals:

Once again, National Journal did even worse, comparing it’s 2009 picks to the 2009 DW-Nominate data, but the drop-off was not as bad as on the liberal side, so comparatively they did less poorly: “just” 6.5 times to high, compared to 8.7:

Still, one has to think they’d do better by just putting up pictures on a wall and playing darts.