Friday, September 13, 2013

Don’t believe the headlines about polls

Politico has a story reporting on a new NBC/WSJ poll.  The headline reads “GOP ON RISE”.  If you read the Politico article, you will come away with the impression that there has been a surge of support among Americans for Republicans and a dramatic drop in support for Democrats.  Lucy McCalmont, the author of the Politico story, writes:

“More Americans think that the Republicans are doing a better job on the economy, foreign policy and reducing the federal deficit..The GOP has an edge of 7 percentage points over the Democrats on the issue of foreign policy.  This is up from 2006, when the GOP was behind 9 percentage points…Democrats only hold a 18 percentage point lead over the GOP in looking out for the middle class, the lowest margin in decades.  The Democrats also see the lowest margin ever on dealing with health care, at 8 percentage points.”

So, according to McCalmont, the Democrats are in deep trouble.

But, wait a minute.  Let’s take a look at the actual poll results.

The poll was conducted between September 5-8, 2013 by Hart Research Associates/Public Opinion Strategies and involved 1000 adults and 300 cell phone only respondents.  The margin of error was +/- 3.1%.  So far, so good.  Nothing wrong there.

Now to understand any poll you need to pay close attention to the “margin of error.”   A 3% +/- margin of error means that there could be as much as a 6 point swing in actual results.  For example, a poll that shows Democrats at 51% and Republicans at 49% would not mean anything at all.  The margin of error tells us that actual support for Democrats could be as high as 54% (+3%) or as low as 48% (-3%) and ACTUAL support for Republicans could be as high as 52% or as low as 46%.  In other words, always look at poll results as the mid-point of a range (the margin of error, which usually is 3% to 4% in a national poll.)

With “margin of error” in mind, let’s dig into the numbers for the NBC/WSJ poll.

As McCalmont reports, Democrats have a 17 point edge over Republicans on the issue of looking out for the middle class.  And, that is down from the early 1990s.  However the edge is close (within the poll’s margin of error) to where it has been since the mid-1990s.  Additionally, the Democratic Party decline (if it is one) is not because of a Republican gain.  Republicans are about where they have been since the mid-1990s.  Democrats are down from a 22 point advantage in February of this year.  However the movement has not been toward Republicans—they remain exactly where they were at 24%.  What’s happened is that more people say both parties are the same or say they are unsure.  But, as I said the movement that has occurred is within the margin of error—3.1%--so the poll does not show enough movement to indicated statistically that anything has changed at all.  And, there is NO trend in the poll results.  The Democrats in the last three polls have gone from a 19 point advantage to a 22 point advantage to a 17 point advantage now.  That’s a zigzag and probably just statistical noise.

In respect to dealing with healthcare, the Democratic advantage has declined but not because of a significant Republican gain—which is within the margin of error—or a statistically significant Democratic decline.  Also, there is NO TREND but rather a zigzag pattern with the Democrats going from 39% to 41% and back to 37% in the last three polls.  We have the same statistically insignificant zigzag pattern when it comes to dealing with the economy and on the issue of reducing the federal debt.

In reference to dealing with foreign policy, the last three polls had the Democrats at 36%, 35%, and now down to 26%.  At first glance that would look like a clear movement away from Democrats and toward Republicans.  However, the two previous polls were conducted in 2005 and 2006, during the Bush administration when Americans were turning against George W. Bush.  If you look at the results going back to the 1990s, Democrats actually do BETTER on foreign policy than they did at that time and Republicans are doing about the same or worse.  Support for the Republicans on foreign policy is actually much less than it was in the early 2000s.  In fact 43% of Americans thought Republicans did a better job on foreign policy then as opposed to just 33% now.

In short, if you examine the actual results of this NBC/WSJ poll, you can’t detect a lot of change—most results are pretty much within the margin of error so any differences are likely just statistical noise and there is no statistical trend away from Democrats and toward Republicans although there may be a slight movement toward dissatisfaction with BOTH parties.

Bottom Line:  Don’t believe the headlines about poll results.  Take a look at the actual results AND keep the “margin of error” firmly in mind.

You can read the Poltico story at the link below and examine the actual NBC/WSJ poll results by clicking on the other link.

No comments: