Polls, people, and panic

When the DesMoines Register poll for Iowa done by Ann Seltzer, considered the Gold Standard not just for that state but among state pollsters, was released last night, Twitter went on fire.  In 2016 Seltzer had released a late Iowa poll that showed Trump up by 9, a margin that he exceeded by several points in the actual vote. Then it was one of the few state polls showing late movement to Trump. People were panicking — was there again late movement to Trump?

If one looked at the poll a bit more closely, one began to ask questions. Remember, as good as any pollster is what is reported is considered a 95% confidence interval  that is, the actual figures reported will be within the Margin of Error (MoE) also reported 19 of 20 times, and once out of 20 may be well outside the MoE.  I did not look inside the poll, but several things being reported about it caught my attention.

First, the figures being reported for the 4 Congressional Districts seemed odd, especially the first, which showed the current Democratic Congresswomen losing by 15 points in a district she carried in 2018 by 5 points to flip the seat.  Next there was a 25 point swing among independent voters statewide from the last poll.  Finally, if there were a late swing towards Trump in IA, one would expect to see similar movement — even if not as large — in the neighboring states of WI and MN, and no one else of repute was showing that kind of movement.

But then I saw tweets about N=800, and when I looked more closely it seemed that the reporting on for of the CDs was based on N=200.  I believe it was our own David Nir noted that at this site they would not consider those CD polls reliable, since DK requires a minimum sample of 300 to be reliable.

I checked on what the MoE would be on a sample size of 200  —  it is 6.93%. That is NOT a meaningful MoE.  And as I noted in comments to a few people, if the same data was being used for both the statewide and CD figures, realistically one might also argue that the N=800 of the statewide figures should also be considered as realistically a combination of four n=200 samples, meaning it too had MoE of 6.93%, which btw was about identical to the advantage it gave Trump over Biden statewide.  I remarked to my wife that I would be skeptical unless it were confirmed by other reliable (NOT Trafalgar, Susquehanna, or Rasmussen) pollsters.  I also noted the near simultaneous final state polls from CNN that showed a very different pattern — no real movement towards Trump. And what we have seen in today’s final New York Times state polls done by Siena, with margins for Biden basically at (or in the case of WI) exceeding the national averages for those states, I am willing to largely ignore this DMR poll, except to say that it does seem to confirm some movement for Ernst in the Senate race, but that might be an artifact of Greenfield being off the trail for several days (?exposure to Covid?).

So let me offer a few thoughts about the Times polls below.