By Corey Cook
I am writing in regard to Reed Nelson’s story “’Poll’ showing 73 percent approval for Mayor Lee was flawed.” As one of the two authors of the survey, I am deeply disappointed in the many insinuations in the article and the author’s cavalier abandonment of evidence or reason in order to make his politically expedient, but otherwise inane, point.
In fact, the author is so quick to dismiss the findings of the study, which is based upon accepted methodology, and which had nothing to do with mayoral approval scores, that he actually misses the entire thrust of the study – that voters in San Francisco are deeply ambivalent about the current environment, concerned about the affordability crisis, and not trusting of local government to come up with a solution.
You’d think the Bay Guardian might find that an interesting subject. Under a previous editor I have little doubt it would have. Instead, the author mind numbingly asserts that the mayor’s approval rate – a largely irrelevant number – is clearly overinflated and the survey must then be “bogus” (meaning fake or phony). While other scholars might find the popular characterization of their work as “fake” somewhat amusing. I do not.
The author relies on two main sources to claim that an on-line panel survey is “bogus”, the New York Times “style guide” and the “website publication” of Southeast Missouri State University Political Scientist Russell D. Renka, who is neither a survey researcher nor a political methodologist, and who does not seem to have published anything in this field (or even in political science based on his on-line vita), but who does seem to have a fairly robust home page that includes cute photos of his grandkids.
It’s not the kind of “source” that I would utilize to deride another academic’s work as “bogus”, and I could suggest some other (actual) publications to consider, including Harvard political scientist Stephen Ansolabehere’s peer reviewed article in Political Analysis titled “Does Survey Mode Still Matter?” from 2011 that compares national surveys fielded at the same time over the Internet (using an opt-in Internet panel), by telephone with live interviews (using a national RDD sample of landlines and cell phones), and by mail (using a national sample of residential addresses).
The authors of that study conclude that “comparing the findings from the modes to each other and the validated benchmarks, we demonstrate that a carefully executed opt-in Internet panel produces estimates that are as accurate as a telephone survey and that the two modes differ little in their estimates of other political indicators and their correlates.” But unfortunately that peer reviewed publication by a Harvard political scientist seems to contradict the simple assertion that a survey result the author doesn’t like must be phony.
Let me say that I don’t considered this issue “settled” in the scholarly community, but it is far from the case that serious on-line panel surveys ought to be derided as “bogus.” My preference would be to do a 1,200 person phone survey. If the Bay Guardian would like to commission such a survey, I would enjoy working with you on that project. But given the various cost limitations that preclude such a robust research design, this is not an altogether bad alternative.
That said, feel free to poke at the methodology and suggest that the numbers for Lee might not reflect that of the overall population because of the timing of the survey or because it was only conducted in English (though I’d disagree with you there – that likely holds down his numbers), or frankly just that surveys do often get it wrong. Even the best random sample is outside the margin of error one time out of 20 according to basic probability theory.
But the other thing I’d like to draw your attention to is that you’ve missed the entire point of the survey. Why do you focus on mayoral approval when it’s a survey about attitudes towards affordability and tech? In fact the article notes that “(i)nterestingly, the USF “poll” also found that 86 percent of respondants (sic) said that lack of affordability was a major issue in the city, while 49.6 percent of that same group considered housing developers to be most at fault for the astronomical real estate prices.” So apparently that part of the survey wasn’t bogus.
Here were our four findings:
* San Franciscans are of two minds: a clear majority of respondents say the city is going in the right direction, yet affordability is seen as a significant, and newly exacerbated problem.
* Most respondents see the tech boom as most strongly helping tech executives and workers. Though there is little sense that respondents and their families benefit from the tech boom, a clear majority say that tech is also good for other white collar workers and the city overall.
* The public strongly supports the idea that the city government ought to enact policies to preserve affordability but were skeptical of public officials’ ability to deal with these issues.
* Despite these concerns, there was little interest in making it harder for tech companies to come to San Francisco. For now, keeping the economy strong appears to be the priority, and we expect that feelings about the economy will likely stave off a substantial political “backlash” at least at the present time.
While Ed Lee has high approval scores, they are tepid – much more “good” than “excellent”. And those numbers erode on affordability, what the voters regard as the city’s most important issue. And we found that people don’t articulate a high degree of trust in mayor in dealing with affordability. Yes, they trust him more than they do others (like developers, or the Board of Supervisors), but not much. This survey help me understand what happened on the 8 Washington vote. Voters like the mayor, as they do Newsom incidentally, but don’t buy their argument that the development would address housing affordability. His popularity didn’t have coattails on this issue.
It strikes me as a real missed opportunity for your journalists to trash the poll, based on really flimsy grounds, rather than address it’s important, and yes, ambivalent findings.
Sincerely,
Corey Cook, Ph.D.