The Joy Of Opening Time Capsules

James Hanley in reply to George Turner says:

George,

First, are you going to accept my challenge or do I have to start making poultry sounds?*

Now on to business.

those are simple polls, not in-depth ones with controls and follow-up.

I don’t think you know what you’re saying when you use the word “controls.” There are no control groups in polling. What “controls” do you mean? And, seriously, you don’t know what these professional polling firms are doing.

Follow-up can be useful, but it’s not required. Some people just aren’t going to answer the poll, no matter how many times you follow up.

What happened with the 280 people who quit responding, and how are they different from the 90 who still answer? If the drop-outs aren’t statistically random in voting behavior, then the new polls have picked up a bias compared to the old polls, and we haven’t corrected for it yet.

And yet you’re sure that you know what the correction would be? You don’t know that it’s not relatively random. You seem to assume liberals are still pretty eager to give their time to a pollster while conservatives are refusing to answer. It’s not that simple. A lot of it is general unwillingness to participate because where once upon a time it was relatively new and seemed as though one was playing a part in the process, now it’s just another irritation to add to the excessive television ads and flossy flyers in your mail. And much of it is about cell phones, which allow people to refuse to answer if they don’t recognize the number (it’s a bit harder, psychologically, for some people to hang up once they’ve answered than it is to not answer at all). But as cell phones have become more ubiquitous, it’s hard to say that their users aren’t essentially random. But to the extent there is a bias there, it’s younger more liberal people that tend to be cell users, so the polls might actually undercount liberals slightly as a consequence.

My point is that you’re exactly right when you say we don’t know. But all your prior arguments have suggested the polls favoring Obama are wrong, so you must think that you actually do know how falling response rates affect the polls. But you don’t.

And for god’s sake, man, who do you think has put more effort into thinking about this problem and correcting it? Some conservative political blogger or the people who depend for it on their livelihood? You’re a conservative, so surely, like me, you believe in the power of the market. Why are you suddenly assuming the market must have failed here, and a bunch of conservative politicos really know better than the people who are actually in the business?

I’ll add this. A couple of years ago I went to a conference of the American Association of Political Consultants. I listened and I talked to folks. Both the Democratic and Republican campaign professionals talk exactly the same about strategy, and both groups rely on the professional polling organizations. I also have a friend who’s been a professional campaign consultant, for Republicans and for Republicans only, and we talk a lot about these things. I’ll tell you this straight up and you can quote me on it–you’re not going to get a Republican political consultant in private to agree that the polls are so biased that a Republican candidate can actually be expected to erase a 4+ point deficit.

You’re listening to people who have a political purpose in making this argument. You’re not actually talking to campaign and polling professionals.

But if you’re really so confident in your story, please accept my challenge and commit to admitting your error if Obama wins. Because if you’re right, you have nothing to worry about, and it will only be me that has to make a public confession.
_________________________
*Which, by the way, was the quickly forgotten Beach Boys follow up to their greatest studio album of all time.Report

James Hanley in reply to George Turner says:

George, those samples are still far more random than any individuals’ “I know all these people who are going to vote this away” samples.

And you’re still assuming you know which direction the polls are biased. That’s the real issue here–you’re extrapolating from well known problems to implied claims about what those problems mean for the outcome. But you don’t actually know that, and you don’t even have enough confidence in your own claims to even agree to admit your wrong prediction if it turns out wrong.

And let’s not forget where this discussion started, with your false claims that a good survey would cost $10 million so the professional orgs aren’t doing them but the campaigns are. From the beginning here you’ve spewing nonsense based in ignorance with a handful of not exactly obscure problems of survey research.

I could go well beyond you in explaining the problems of survey work, such as the effect of wording, question order, etc. But that’s not necessary. All that’s necessary is to point out that despite the problems inherent in survey research, it still does a better job predicting outcomes than those who complain it’s no good, because it’s methodology is still more rigorous than the methodology you, Koz, and Van Dyke are using.

But if you really do believe, man up and stick it to me by suckering me into committing myself to admitting my error. Talk is cheap, and if someone’s not willing to do anyt hint besides talk, it’s a pretty reliable sign that they know how weak their argument really is.Report