(Editor's Note: This piece is coauthored by Philip A. Wallach and first appeared at Legbranch.com on June 26, 2018.)
Our recent piece, “Congress is Broken. But don’t blame polarization,” provoked a number of useful discussions about how political scientists should interpret the results of empirical models like DW-NOMINATE. When used improperly, these models distort our understanding of reality rather than illuminating it. Consequently, we believe that political scientists should not use the data produced by such models uncritically. These models should instead be seen as tools that can help foster a better understanding of Congress, but only when appropriate. To that end, we seek to clarify and deepen our original discussion in this post.
1. What does roll-call polarization mean?
What can polarization be about, if not ideology? Sometimes, parties are engaged in a zero-sum competition over who will benefit from patronage, with few “ideological” implications at all. Other times, parties may frame votes for purely tactical advantage. But assuming that NOMINATE scores are measures of ideology reduces votes over all such conflicts to being indicative of ideology, analogous to a vote to increase or decrease the size of the welfare state. In short, NOMINATE treats all roll call votes as ideological, regardless of their underlying meaning. According to Frances Lee, “The methodologies scholars use to infer the policy distance between the parties cannot distinguish between partisan and ideological conflict.”
This is the ground on which our critique of the way in which political scientists use NOMINATE is based. In that way, we are building on Richard Bensel’s critique of NOMINATE. Bensel writes,
The problem is not with the construction of the algorithm which will always and infallibly produce statistical artifacts but lies, instead, with yawning chasm between those artifacts and the language in which we try to interpret them. … The basic problem is that the ideological beliefs of members of Congress have often been conceived as tautologically identical with the statistical artifacts produced by the NOMINATE algorithm.
Bensel cautions us against thinking of the famous two-dimensional dot plots that are output by the NOMINATE algorithm as composites of each member’s voting pattern. Rather, they depend on the totality of the dataset. Excluding a single member or vote would change the whole picture. As we said in our piece, then, the roll-call-based NOMINATE scores are reflections of what gets voted on much more than they are capable of reflecting what is really in legislators’ heads as they decide how to approach the voting chances they are given.
When we look at political scientists’ attempts to create other measures of ideology, the utterly confused picture that emerges strongly supports our revisionist interpretation of the roll-call-based scores. Chris Tausanovitch and Christopher Warshaw recently showed that six independently constructed measures, each purporting to identify candidates’ ideology, are very poorly correlated with each other – except for the basic feature of being able to reliably identify partisanship. (“[T]wo measures can be expected to explain only 33% of the variance in one another for Democrats and 18% for Republicans.”) Their bottom line is striking: “Overall, the measures we examine only marginally improve on candidates’ party identification for predicting their roll call behavior.” NOMINATE scores do possess some predictive power—but that is no surprise. It is equivalent to saying that roll-call voting in the future will resemble roll-call voting in the past, and it gives no warrant for an ideological interpretation. Indeed, in light of the mixed up picture from the other measures, we should be wary of leaping to an ideological interpretation.
On Twitter, Boris Shor responded to critiques of the standard interpretation even in light of Tausanovitch and Warshaw’s findings: “What they're trying to do is see how much non-chamber measures help predict Congressional roll call behavior, beyond party. If what I want is to characterize chamber polarization, I would use roll call based measures.” Wallach responded: “‘If what I want is to characterize chamber polarization’...But is that what we want?”
In other words, if this were really about members’ fixed and durable ideological commitments, why would the overall picture painted by these many measures be so jumbled?
2. Object all you like—but aren’t the members with the extreme NOMINATE scores actually the extreme ideologues?
One passage in our original piece is in need of additional clarification:
The … “first dimension” mapping for each legislator is generally interpreted as representing the legislator’s ideology on the “liberal-conservative” dimension. By construction, this assumes that when two legislators vote in opposite ways most of the time, it is because one of them is “very liberal” and one is “very conservative.” But nothing in the algorithm actually supports using these labels instead of, say, “very likely to toe the Democratic line” and “very likely to toe the Republican line.” Many times, toeing the party line will mean acting out a certain ideological agenda — but many times, it won’t.
It may be intuitive to read, “toe the party line” as meaning, “go along with the leadership.” On that reading, what we said is definitely not right. Party leaders do not end up with the most extreme NOMINATE scores. In the 114th Congress, Mitch McConnell scored 0.404 and Paul Ryan 0.555, with plenty of members in both chambers to their right.
However, what we intended to convey was that those members far to the extreme are often conforming their voting records to the demands of their party’s hard-core bases—the kinds of people who write the platforms. (Leaders are often talented at fervently declaring their allegiance to this base even while cutting a more accommodating path in practice.) As we say, those on the far extremes may sometimes be justly described as “ideological,” but to identify “what Republican activists demand” with “conservative ideologue” is to engage in a purely circular logic. If American politics since 2016 has taught us anything, it should be that the power of conservative ideology as a controlling force has been grossly oversold, and in general collapsing the distinction between “partisan” and “ideological” is unhelpful.
It also requires an unjustified leap to say that because a member seeks to appease this constituency in their roll-call voting decisions, this shows that they are intrinsically ideological. Depending on the nature of their district, some members may see voting to please the base as much more important than others; whether this reflects their own internal beliefs, or their strategic considerations, is difficult to say.
To see why the picture is more complicated than the “extreme ideologues” interpretation suggests, let’s briefly look at a few examples. Are the members with extreme scores clearly “liberal” and “conservative” ideologues? How about Senators Bernie Sanders (-0.509), Cory Booker (-0.498), Ben Sasse (0.833), and Mike Lee (0.861)? The algorithm tells us that the two Senators on the left vote very differently than the two Senators on the right. Fair enough.
The ideological interpretation of the algorithm’s outputs tells us that they are among the members least likely to see eye-to-eye on anything. But here’s where it’s so crucial to focus on what gets left out of the dataset. As it happens, Cory Booker and Mike Lee have frequently teamed up in recent years, making serious pushes on a wide variety of matters including criminal sentencing reform, devolution of medical marijuana policy to the states, agricultural conservation, and stopping corruption in agricultural programs. None of those bills ever received votes. This kind of activity, which so clearly suggests that these members are something other than dogmatic ideologues, is simply invisible to NOMINATE. To the extent there are unrealized possibilities for cooperation on matters that transcend the positions locked into the parties’ platforms, the roll-call-based measures just aren’t going to tell us.
The examples of Sanders and Sasse are also instructive. Both have voting records to please their respective bases, but that hardly gives an adequate description of their respective places in their coalitions. Both are critics of their party establishments and are outside-the-box thinkers. The ideological interpretation of their NOMINATE scores seeks to deny the importance of these facts. As tweeters aplenty have suggested: if these members are so committed to different ways of thinking, why do they end up voting so conventionally? That reflexive belief that what really matters about a politician is what will show up in the roll call votes is myopic and limiting—as Matt Glassman eloquently argued in a slightly different context.
3. Are Democrats and Republicans today more ideologically divided than ever?
As we note in the piece, DW-NOMINATE’s portrait of historic polarization in the current Congress is offered as evidence in support of the narrative that currently dominates political commentary. In short: whereas the parties were once ideologically mixed coalitions, they are today well-sorted along ideological lines, and each side (but especially Republicans) has come to embrace a more extreme ideological position over time. As a result, the prospects for political compromise are dim.
For this logical chain to hold, two assumptions are necessary. First, that policy positions retain their ideological valences indefinitely. Second, that the division captured by DW-NOMINATE in one Congress is comparable in magnitude to the division in any other. Both are obviously false. As David Bateman, Joshua Clinton, and John Lapinski argue, taking ideological continuity seriously over time would lead to the obviously absurd prediction that today’s Democrats would vote against federal protection for freedmen during Reconstruction. Similarly, we’d be led to think that divisions over the enforcement of the Voting Rights Act today are of comparable importance to the divisions over whether African-Americans should be allowed to vote at all. As they point out, fixing Democrats as always on the left, and Republicans as always on the right, gives NOMINATE scores a partisan meaning that automatically predominates over any ideological one. Where parties’ relationships to ideological positions are in flux, relying heavily on NOMINATE is likely to produce confused analysis.
The standard, NOMINATE-backed story also tends to put so much focus on interparty difference that it also tends to distract from the importance of intraparty divisions. When we seek to understand some of the leading issues of the 115th Congress, it is internal divisions between Republicans that have often been decisive.
On healthcare, Republicans came in having promised for half a decade to repeal Obamacare, and yet it quickly became clear that there was significant disagreement among Republican members as to how much this could be taken seriously or literally now that they were in power. Leadership failed to forge a major reform that could garner majority support, and eventually they opened the door to a member-brokered compromise. It (surprisingly) passed the House and came close to succeeding in the Senate—but it did fail, and of course it was not anything like a full repeal. Attempts to find out just how much change a bipartisan coalition of legislators would be interested in making have been successfully discouraged.
And on immigration, the reality is that neither Speaker Paul Ryan, R-Wis., nor Senate Majority Leader Mitch McConnell, R-Ky., nor Minority Leader Chuck Schumer, D-N.Y., tried particularly hard to resolve the issue. The reason? Both parties are deeply divided internally when it comes to immigration policy and forcing action on the issue would have only exposed those divisions. For this reason, party leaders refrained from doing so. They merely pretended to create the conditions for open-ended debate while actually ensuring that nothing with a chance of getting 60 votes would ever receive consideration. Gridlock over immigration can thus be interpreted as arising out of a leadership response to intraparty divisions, not ideological polarization.
4. What role is issue suppression playing?
That brings us to the issue of issue suppression, which is central to our view of the current situation. Leadership shapes the agenda to exclude cross-cutting issues (or, as with immigration, to give them only nominal chances) in order to differentiate their partisan brand and thereby maximize their electoral advantage. (Or so the story goes…)
Let us offer an example where issue suppression has been remarkably successful over the last decade, during majority control by both parties. In the wake of the financial crisis, there were huge reservoirs of political energy looking for retribution against America’s big banks, who had profited from bad practices and then been rewarded with government support. Retaliation would almost certainly have been a political winner, whether in the form of breaking up the businesses somehow or dramatically raising their required levels of capital. Our major piece of post-crisis financial regulation, the Dodd-Frank Act, took half-hearted approaches on both counts (by imposing the modest and largely ineffectual Volcker Rule that bans banks from proprietary trading, and by vaguely pushing for higher capital requirements). The policy merits of the more dramatic moves are highly debatable, but that debate has rarely taken place in Congress, and certainly none of the proposed legislative vehicles for dramatic action ever received extended consideration. Neither the Bailout Prevention Act of 2015, co-sponsored by Elizabeth Warren (D-MA) and David Vitter (R-LA), nor the Terminating Bailouts for Taxpayer Fairness Act of 2013, cosponsored by Sherrod Brown (D-OH) and Vitter, ever made it out of committee.
From our perspective, this is no puzzle: Leaders can see that their members are internally divided on these questions, and they have no interest in seeing whether a complex, ideologically heterodox coalition could form. They might well offer policy justifications for their opposition, but the coalition-preservation explanation is almost certainly the real cause. As Lee Drutman explored, congressional leadership has generally done a masterful job suppressing issues that activate a pro- and anti-establishment cleavage, which cuts across the two parties. (Here is an attempt by Harry Enten to fit this concern into DW-NOMINATE’s framework—but note that labeling the second dimension in this way is just as arbitrary as labeling the first dimension as liberal-conservative ideology.)
Drutman argues that the political status quo they are protecting is overdue for disruption, for substantive reasons. But the case can also be framed in more procedural-institutional terms. A Congress that follows its leaders as they flee from the most controversial issues of the day, in the name of party unity, is not ultimately a healthy participant in our system of government.
We are hardly the only ones to think this way. Dougherty, Lynch, and Madonna analyze all House roll call votes from 1875 to 1997 and find evidence that restrictive rules are successfully used to reduce the dimensionality of recorded votes. They offer a number of citations to other work that explores how parties use agenda power to reduce observable dimensionality, and note Cox and McCubbins’ assertion that “the first commandment of party leadership” is, “Thou shalt not aid bills that will split thy party.”
5. What is at stake here?
Are ideologues breaking our politics? Is the disappearance of moderate members the reason we seem so unable to compromise? Are American politics doomed to futility as long as “our” members find themselves across the aisle from “those crazies”?
To each of these questions, the ideological interpretation of contemporary polarization answers yes—and we would answer no. Rather than seeing deeply entrenched ideological polarization as the root cause of our political dysfunction, we see contemporary congressional polarization as a symptom of a misguided conception of politics that has come to dominate our discourse. We believe that difference and disagreement are the basic stuff of democratic politics—and that our Madisonian political system has, in the past, successfully worked through greater differences than we see today. But today our leaders seem too eager to redefine the polity in order to banish disagreement, either through respectability politics or manipulation of the voting electorate or just through a once-and-for-all decisive win in the next election.
Stay tuned as we work to develop all of these ideas, and a sincere thanks to all of our critics for your helpful comments.