Thursday, September 20, 2012

Why the Marquette poll is such an outlier


The Marquette Poll is even screwier than Mike Flynn of Breitbart suggests. 

First of all, it looks like it's a twofer, a questionnaire that covers two topics in one survey. The first part is a horse race survey of Wisconsin, but the second part is all about global warming/climate change. 

Maybe because the people behind the global warming part wanted a more representative survey for their purposes, the RV screening is really loose: 94% of those contacted said they were registered to vote and the remainder were asked a follow-up question about whether they they planned to register to vote. All 41 said they did plan to register. This 100% registration compares with about 68% of the voting age population of Wisconsin that was registered to vote in 2010 according to the Statistical Abstract. 

So, right away, one suspects that maybe 1/4 to 1/3 of the respondents may have lied about their self-reported registration status. Essentially, then, its a poll of adults, not RVs, not a great sample for a horse race survey, but one that works just fine for a global warming/climate change survey. Anyway, of the 705 'registered voters' in the survey, 601 or 85% qualified as likely voters by answering that they were absolutely certain to vote. In 2010, again according to the Statistical Abstract, 70% of registered voters in Wisconsin voted but only 54% of the voting age population did, which is essentially what this sample, per above, represents. So, when the Marquette survey says registered voters, think adults and when it says likely voters, think registered voters plus a bunch of liars (in 2010, only 68% of the voting age population in Wisconsin was registered to vote). 

Two of the proofs of the validity of these new definitions is that there was no difference between the registered voter base (54% to 40% for Obama-Biden) and the likely voter base (54% to 39% for Obama-Biden) and that the Party ID results (with leaners) were +11 Democrat.
Besides using a lot of foreplay questions (e.g., what do you think of the Tea Party?), and asking state horse race questions before the presidential horse race question (the 16th question on the survey), the base definitions largely determine the strong outlier results from this survey. 

But, heh, I think the global warming/climate change results from the Marquette survey are probably perfectly representative of what Wisconsin adults think on that topic.

Friday, August 10, 2012

Summer Vacation Peak Arrives and Republicans Disappear from National Samples


Three recent presidential preference polls conducted in late July and early August have had ending samples that were strongly skewed toward Democrat identifiers.

• The Pew Poll, conducted July 16-26, 2012, among 2,508 adults, including 1,956 registered voters,  had an 18-point Democrat advantage: 43% D, 25% R, 32% I
• A PPP poll of NC PPP surveyed 813 likely North Carolina voters from August 2nd to 5th had a 12-point Democrat advantage: 45% D, 33% R, 21% I likely voters
• A recent Fox News poll based on landline and cell phone interviews from August 5 to August 7 among 930 randomly-chosen registered voters nationwide had a +9 Democratic advantage.

While some may believe these huge skews to be intentional on the part of the pollsters, a more realistic   explanation is seasonality.  The main reason for these strong Democratic skews, we think, is that these 3 surveys (and more to come in peak early/mid August vacation weeks) are surveying into the teeth of the American Summer Vacation Sampling Frame, the high season period when many Americans go away for vacation. 

This leads to what Kish (1995)1 cited as one of the main problems of sampling frames: "missing
elements: some members of the population are not included in the frame".

Although American Summer Vacation Sampling Frame problems are not nearly as severe as in the European case (where much of market research shuts down in August), at Moran Group, we've found that 'missing elements'  can  definitely be a problem in July & August surveys, especially when it comes to interviewing more upscale and professional respondents.

Indeed, one of the reasons that the American Summer Vacation Sampling Frame problem may be particularly acute in 2012 is found in a recent Harris Interactive survey cited in Business News Daily, which shows that the members of the US population who are not included in the 2012 Summer Sampling Frame may be particularly upscale this year
While only six in 10 adults planned at least one trip through August, which was down from 2009 and 2010 numbers these vacationers are planning on spending an average of $3,136 during vacations this year, nearly double the 2010 average of $1,627
Since school openings tend to be a big driver of family Summer vacation schedules and vary regionally, the problems associated with the American Vacation Summer Sampling Frame really won't go away until school starts: 34% of school districts open before August 19, 27% open 8/20 to 8/26,  45% open 8/27 to 9/2 and 21% open after 9/2. 

For purists like me out there, we should be back to the standard American Sampling Frame in polls taken after Labor Day.

1. (Leslie Kish (1995). Survey sampling. Wiley. ISBN 978-0-471-10949-5. Retrieved 11 January 2011.)

Friday, August 3, 2012

Jenna Endorses Mitt


Jenna Jameson, hopefully, for traffic's sake, the first of many Porn Stars for Mitt.
Read more at link