With statistics, political polls and bikinis, it's not what they show that matters

There’s a good reason to be skeptical of polls produced by campaigns.

Unlike media and academic polls, campaign polls are designed to do more than reveal. They’re designed to help a candidate who is paying for them win the election.

That’s why I’m usually skeptical about releases – usually partial releases – of poll results by campaigns. And because we’re skeptical, campaigns rarely release them anymore.

Last week, however, it happened twice and for the same reason. First, an internal poll for 3rd Congressional District Democratic candidate Denny Heck found its way to the Washington, D.C., newspaper Roll Call. It showed Heck within three points of Republican Jamie Herrera. That challenged earlier polls (and the primary results) giving Herrera a huge lead.

Then, 9th District U.S. Rep. Adam Smith’s campaign sent out a memo that reported the Democrat with a 19-point lead – 54 percent to 35 percent – over Republican Dick Muri (plus or minus 4.9 percentage points). That memo came out shortly after KING-TV released its SurveyUSA poll showing Muri-Smith too close to call.

The KING-TV poll was a shocker because Smith enjoys a large fundraising edge and did comfortably well in the primary (54.5 percent when combined with a Green Party candidate’s vote vs. 45.5 percent for the two Republicans).

Smith’s campaign understood the effect of the KING-TV poll and quickly moved to counter it with a memo from its pollster. Our rule of thumb with any poll is to know the exact wording of the question, the dates the survey was conducted, the number of voters interviewed and the margin of error. Smith’s pollster – Benenson Strategy Group of Washington, D.C. – did that for this question asked of 400 likely voters Sept. 18-20:

“Given this, if the election for the U.S. Congress was held today, and the candidates were Democrat Adam Smith or Republican Dick Muri, for whom would you vote?”

Did you catch the problem? I didn’t until an interested bystander called to point out two short but very significant words.

“Given this.”

Given this? Given what?

Those words suggest this question came later in the poll, most likely after the survey taker had read a statement designed to alter the opinion of the voter. For example, the survey might want to test how attacking Muri on support for the Bush tax cuts might resonate with voters.

A good poll first asks a straight head-to-head question. Only after it establishes that baseline can it measure how different messages or attacks move the numbers.

This sort of message testing – distinct from so-called push polls that are not polls but rather disguised attempts to persuade – is common. It’s how campaigns assess whether a tactic will help or backfire.

But the resulting numbers are not accurate measures of voter opinion and shouldn’t be portrayed as such.

So the big question is, Which numbers did the Smith campaign release? Did it release the results of the first horse race question or a tainted question asked later in the poll?

When I reached Pete Brodnitz of Benenson, he said he hadn’t noticed the phrasing of the question on his own memo until I called.

“You picked up on a real problem with the memo, but it’s not a real problem with the poll,” Brodnitz said, blaming a bad “cut-and-paste job.”

“I can assure you I’m annoyed with whoever put the memo together,” he said.

Well, that makes two of us.

Brodnitz said the memo’s numbers – the 19-point Smith lead – came from an unbiased question. Only later, after potential messages were tested, was the question asked again, this time prefaced by “Given this … .”

We’ll have to see more polls – and perhaps the actual election results – to gauge whether the KING-TV poll or the Benenson poll is the outlier. By then, this all will be forgotten.

But it does serve as a reminder that polls are only as accurate as the methodology used. And yes, sometimes they are used to spin as much as they are to reveal.

Peter Callaghan: 253-597-8657 peter.callaghan@thenewstribune.com blog.thenewstribune.com/politics