But has polling become just another weapon in the dark arsenal of modern politics, alongside robocalls, micro-targeting and attack ads? In this new age of elections as crowd control — the techniques of which Stephen Harper has so thoroughly mastered — do pollsters measure public opinion or try to lead it? Are they being used by special interest groups? Can they be trusted?
Arguably the best pollster around, Nik Nanos of Nanos Research offered me some observations on the business while I was researching my new book, Party of One. One of the things he told me was that “any contact” with the public by pollsters can be unethical. You either speak truth to power or you don’t. You either deliver the reality sandwich — or you give a massage.
“With a major pollster,” Nanos said, “what you see in the news is only the surface. Beneath is an implied reference to deep data and knowledge of trends, which ‘validates’ the insights … The real test is who has had it right for the last five elections — accurate cultural memory. Invariably, the people who fit that category operate as quality silos.”
What it really comes down to is that there are pollsters and there are real pollsters. Real pollsters are not in the business of building anyone’s numbers. They have solid track records. They have what the business calls “full-scope” control over their product — the call centre, the focus group, questionnaire writing, interpretation.
They do not sub-contract out their work, but create high quality, independent “silos” free of cross-pollination. They get paid for their work and they aren’t in the business of getting it wrong. If a politician is getting “free” polling — and a lot of them are — they’re not getting it from the pros like Nanos, Decima, Ipsos or Environics.
By comparison, a “fair-weather” pollster has no control over field work. In fact, some pollsters use another firm’s call centre to generate their polls, which leads to an oddity: two ‘different’ polls based on the same data. That can easily lead to false trends, which in turn can be used to cynically manipulate public opinion. Not an unimportant thing during an election campaign.
Sometimes, the fair-weather pollster is also a campaign manager. There is lots of power in that approach, lots of money to be made. But the mission changes with the dual role. The pollster who is also campaign manager sets his sights on leading public opinion, not recording it.
The favourite tool of fair-weather pollsters is the random-dialed Interactive Voice Response, or IVR call. It is a relatively weak research tool since there’s not much depth involved in programming a demon dialer and asking the respondent to press a couple of numbers on a phone keypad. The result is often a shallow assessment of public opinion because IVR isn’t designd to handle open-ended questions — or even confirm who is responding to the survey.
In a nutshell, the reason fair-weather pollsters like the IVR method so much is that in the voter contact world, it means mass contact and mobilization of voters — the strong suit of the Conservatives. So call it what it is — telemarketing, not research. To do professional polling, there is really no substitute for a live interviewer talking to a live subject. “Telephone interviews are still the best way to get a read on public opinion,” Nanos said.
But what about the integrity of the questionnaire? Nik Nanos is the last person you would have expected to find in front of his profession’s complaint panel. But that’s exactly where he ended up over a complaint from the Canadian Labour Congress. The CLC accused Nanos of loading the dice in a poll he conducted on behalf of LabourWatch, which trumpeted a conclusion that 83 per cent of Canadians agreed that unions should be forced to make full financial disclosure.
In reporting the poll, Nanos left out the preamble to one question that the labour organization felt was clearly “priming” the answer from respondents. In another instance, a question was entirely left out of the report on the poll — a process documented by researchers at the University of Regina in an outstanding report called Working in the Shadows for Transparency.
LabourWatch bills itself as a non-partisan organization that just happens to want to take a sledgehammer to unions — rather like the Harper government. For that reason, LabourWatch fully supports Bill 377, the “private member’s” bill heartily endorsed by the Harper PMO. The group used the Nanos poll, as did the government, to advance this piece of legislation.
Bill 377 is really just a re-run, unamended, of the same bill that was sent back to House of Commons by the Senate in June 2013 with substantial changes. The Harper government simply returned the bill to the Senate in its original form in October 2013. The way organized labour saw it, Harper was simply using polls as cover to reintroduce union-crushing legislation.
Former Conservative senator Hugh Segal pulled no punches, describing Bill 377 this way: “Use of unprecedented time allocation on private members’ bills would justifiably produce a firestorm. Bill 377 was badly-drafted legislation, flawed, unconstitutional and technically incompetent when it was amended the last time. Unamended, it has not now become perfect simply because one senator retired to do other things.”
At a personal level, it must have been a very odd moment for Nanos when he faced the complaints panel. In 2004, Nanos was one of the founders of the Marketing Research and Intelligence Association, which acts as the conscience of the polling business on behalf of its 1,800 members. Nanos also served as the association’s president and, just four years ago, was voted an MRIA Fellow — hardly the stuff of which professional misconduct claims are made. This guy is the real deal.
While the complaint panel found “no evidence” that Nanos Research had violated MRIA’s code of conduct, it did say this: “The Complaint Panel found that the reporting of the two questions in issue, specifically the omission of question 18 from the report and the reporting of question 20 without the preamble, allowed potentially biased information to be reported by LabourWatch.”
Much less forgiving, veteran pollster Allan Gregg called the use of the Nanos poll “horrendously biased” and added: “This is not the kind of polling that people in our discipline should be doing. Clearly it’s being done by an advocacy group that’s got a particular perspective on the world and an axe to grind, and they’re using the poll not to illuminate their understanding of public opinion but as a PR [public relations] tool.”
The panel made a series of recommendations, including this one: “Nanos should inform the LabourWatch, and copy the CLC and MRIA, that the question 20 must only be reported within the complete context of the question, including the preamble … Nanos should also seek permission from LabourWatch to release the results of question 18. These results should be communicated to the CLC by letter (with a copy to MRIA).”
LabourWatch refused to comply with the MRIA complaint panel’s recommendations. Instead, they answered with another poll that purported to show that priming for an answer in a poll questionnaire was inconsequential to the result.
The message for the media in all this? Here’s what David Northrup, director of the Institute for Social Research at York University, told researchers at the University of Regina: “If the media did a better job examining the type of research that came out of pollsters, maybe pollsters would feel that they were being held to a higher standard, and to be quite frank, usually you can get away with just about anything, because the media can be sloppy when it comes to taking on pollsters.”
He who pays the pollster will always try to call the tune.
Original Article
Source: ipolitics.ca/
Author: Michael Harris
No comments:
Post a Comment