It seemed like a safe prediction.

As chief executive officer of Abacus Data, a polling company, David Coletto had followed the 2012 Alberta election closely. In the campaign’s final month, his polls and those of his competitors showed the same thing: The right-wing Wildrose Party, led by Danielle Smith, was winning. In its final survey, Abacus had Wildrose up by 10 percentage points over the Progressive Conservatives.

So on April 23, as people across the province cast their ballots, Mr. Coletto was interviewed on Sun News Network and said the obvious: Ms. Smith would be the next premier of Alberta.

Mr. Coletto recently described what happened next.

“The results started coming in,” he said, “and my face goes white …”

The PCs had won easily. Their margin of victory, almost 10 points, was the opposite of what Mr. Coletto had predicted.

That election left Canada’s polling industry shaken. It wasn’t just scrappy upstarts such as Abacus that had blown the call – veteran firms such as Leger Marketing had misfired, too.

These are not the golden days of polling, that’s for sure. There are many reasons, and they sort of overlap and intersect with each other. It’s sort of the perfect storm

Scott MacKay, president of Probe Research

Nor was 2012 an anomaly: Alberta-scale disasters have become increasingly common in the world of public-opinion research, from Israel to Scotland to the United States.

In Canada, with a federal election looming, the polling industry is in a nervous state. Its earnings are shrinking, its reputation is tarnished and its methodologies are in flux. Known for their bravado and influence, many pollsters have been left feeling vulnerable.

“These are not the golden days of polling, that’s for sure,” said Scott MacKay, president of Winnipeg-based Probe Research. “There are many reasons and they sort of overlap and intersect with each other. It’s sort of the perfect storm.”

———————

Election-related revenues are a fraction of Canada’s $500-million polling industry – less than 4 per cent, according to the Marketing Research and Intelligence Association (MRIA) – but they play an outsize role in pollsters’ fortunes.

The lead-up to a vote is a smorgasbord of free publicity for companies such as EKOS, Nanos and Ipsos.

“Doing this work is almost the equivalent of a fashion gangplank,” said Angus Reid, 67, whose storied career in polling includes the founding and selling of his eponymous company. “It’s an opportunity to show off their research.”

By the same token, the spectre of failure looms over pollsters throughout a campaign. A botched election forecast is an excruciatingly public form of failure: Curtis Brown, vice-president at Probe, admitted to having anxiety dreams about fiascos such as the 2013 B.C. vote, which his colleagues in the industry got spectacularly wrong. (Probe didn’t have a poll in the field.)

The upcoming federal election appears especially difficult to call. The most recent Nanos Research poll for The Globe and Mail essentially shows a three-way tie, with the Conservatives, NDP and Liberals separated by less than the margin of error, each hovering around 30 per cent.

But even in the best of times, gauging voter intentions is fraught with difficulties. There are the “Shy Tories,” well documented in Britain, who deny that they intend to vote Conservative until doing just that.

The “Bradley effect,” meanwhile, is named after a black candidate for governor of California who led in the polls right up to voting day before losing to a white opponent.

Polls, in the U.K. and in other places around the world, appear to be getting worse as it becomes more challenging to contact a representative sample of voters.

Nate Silver, on his website FiveThirtyEight.com

In the privacy of the polling booth, fickle hearts and cold feet often prevail. As Abacus’s Mr. Coletto put it, “Ultimately, people are really unpredictable.”

People are even less predictable if you can’t get in touch with them and pollsters are having more trouble on that score. They now find themselves in an awkward state of technological limbo: With both phone and Internet polls plagued with problems, the industry lacks a consistent, affordable way to reach a wide swath of the population.

No less an authority than Nate Silver – the writer and statistician whose nearly perfect predictions of the past two U.S. elections made him a polling superstar – has warned of a sustained dip in quality.

“Polls, in the U.K. and in other places around the world, appear to be getting worse, as it becomes more challenging to contact a representative sample of voters,” he wrote in a post on his website, FiveThirtyEight.com, after this year’s British election, when polls failed to predict a Conservative majority.

Pollsters themselves are acutely aware that the industry is amassing a growing tally of failures. The jitters have gotten so bad that even polling successes now come laced with anxiety. When Albertan voters stampeded into the NDP camp ahead of this year’s provincial election, polls accurately picked up on the stunning shift. But some in the industry were “so spooked,” Mr. MacKay said, “that they didn’t even believe their own numbers.”

“The NDP was up 15 points the day before the election,” he added, “and you saw these guys on TV and their eyes were shifting and they were saying, ‘You know, anything can happen. People change their minds.’”

Fittingly for number crunchers, the pollsters’ malaise can be quantified: In 2004, the MRIA reported that its members earned $574-million; in 2014, that number was down to $509-million.

“Our industry has definitely taken a hit,” said Sébastien Dallaire, vice-president of public affairs at Leger Marketing.

———————

It has been a gradual but painful descent for a sector that journalists and politicians once accorded a kind of mystique. In the late-1970s, the empirical weight of opinion-poll pronouncements was a new force in Canadian politics; pollsters, it was thought, wielded survey results as if they were crystal balls.

By the early 1980s, declining long-distance rates and the ubiquity of land lines led to a spike in telephone polling. Suddenly, it was cheap to call Canadians across the country, and, significantly, Canadians were picking up.

“The brilliance of polling in the 1980s is that virtually everyone was accessible through a single means of technology – the telephone,” said Andrew Laing, president of the media monitoring firm Cormex Research.

Angus Reid’s refusal rate was about 15 per cent in those days, he says – so nearly nine in 10 people who picked up agreed to answer his questions. Some people even complained when they were denied the novelty of having a pollster call them at home.

“In the early eighties, this was viewed as pretty sexy stuff,” Mr. Reid said. “Polling had sort of arrived from the margins to the mainstream and everyone loved it. It was really the salad days of the industry.”

Suddenly, pollsters such as Allan Gregg and Darrell Bricker were everywhere – advising political parties, being quoted on front pages and leading the nightly news, they had become minor celebrities.

“It was really exciting,” said EKOS Research president Frank Graves, who founded his company in 1980. “It was a period where you felt pretty comfortable that you knew how to do things, people listened … and you made a lot of money. It was a pretty good deal.”

But starting in the early 2000s, technological and social change began dulling some of the industry’s lustre.

Many cite weaker voter loyalty as a hurdle to predicting elections. “People are far more pragmatic and spontaneous than they used to be,” Probe’s Mr. MacKay said, “which is a nightmare for a pollster.”

Even more daunting, though, is the issue of getting voters to answer questions about their political preferences on the phone. More than half of Canadians under 35 don’t have a land line, and tracking down cellphone users is more expensive since they aren’t listed.

At the same time, caller ID has made it easier for land-line users to ignore pollsters. And those who do pick up often promptly hang up again, thinking they’ve encountered a hated telemarketer.

“It’s a call from someone they don’t know and it’s all the same,” Mr. Brown lamented.

Refusal rates for political polls, once below 20 per cent, now often top 90 per cent.

The growing inefficiency of paying for live interviewers has driven the industry toward less expensive techniques such as Interactive Voice Response (IVR) – robo-polling – and online panels.

In Canada, everybody just kind of runs away. It hurts the industry, but most importantly, it’s a disservice to democracy.

Darrell Bricker, CEO of Ipsos Public Affairs

That lower cost has allowed a raft of new firms such as Abacus, which does 80 per cent of its polling online, to enter the field.

“I don’t think Abacus could have started the way that we did 10 years ago,” Mr. Coletto said. “Physically, we needed some office space, a few computers and an Internet connection.… There are some barriers to entry, but they aren’t physical or capital-related.”

The old guard often treats these newcomers with thinly veiled contempt.

“There are a lot of people playing at this,” said Mr. Bricker, CEO of Ipsos Public Affairs. “You see these people pop up – who knows what their motivation is. To get a little publicity.”

Still, online polling has many credible proponents – Mr. Reid himself uses Web panels for his non-profit research foundation, the Angus Reid Institute. Boosters point to the different kinds of questions they can pose on the Web, asking respondents what they think of video clips or passages of text.

Mr. Reid, meanwhile, touted the relative intimacy of Internet questionnaires, which allow for interviews on sensitive topics such as sexual harassment that people might be uncomfortable discussing over the phone.

But finding a good response pool online is a “bit of an art form now,” Mr. Reid acknowledged. While randomness was once a watchword for pollsters – ensuring that samples weren’t self-selecting and skewed – Internet panels now solicit members with Web ads, often asking participants to complete multiple surveys in exchange for Air Miles, donations to charity, concert tickets and other goodies.

“Increasingly, people expect to be compensated for their time,” said Mr. Coletto, who conducts his polls through a panel of 500,000 people compiled by a separate market research firm.

Critics warn that polls conducted through these panels can yield warped results, since respondents have volunteered to participate and might be more opinionated than the general population, or motivated by money.

Nik Nanos, CEO of Nanos Research, which does live-agent phone polling for The Globe, said two-thirds of the polls conducted in Canada wouldn’t pass muster for publication in The New York Times, mainly because they aren’t random enough.

Online polls aren’t always wrong, he said, but they aren’t consistent either. “Do you want your survey right 19 times out of 20, or 15 times out of 20?” he asked.

Pollsters who try to take soundings from social media are greeted with even more skepticism. “Wild, voodoo polls,” Mr. MacKay called them.

“I see some of these Facebook polls and it makes me want to puke,” Mr. Reid said.

As dubious polling becomes more prevalent, the industry looks poised to have an unusually large impact on this year’s federal election. About two-thirds of voters are determined to replace the Harper government and many believe that whichever left-of-centre party looks likeliest to accomplish that goal will reap a bumper crop of strategic ballots.

That calculus will be heavily determined by polling: If the NDP, for example, leads by a healthy margin at the start of the campaign’s home stretch, lukewarm Liberals could flock to the orange tent – or vice versa.

In an effort to shore up their credibility and their bottom lines ahead of such a crucial test, Canadian pollsters have begun trying to self-police. Mr. Bricker was recently elected chairman of one such group, the Canadian Association for Public Opinion Research, which launched in June. It plans to set standards around transparency and polling methods to hold firms accountable when things go pear-shaped.

“In Canada, everybody just kind of runs away,” he said. “It hurts the industry, but most importantly, it’s a disservice to democracy.”

Others, such as Mr. Graves and Mr. MacKay, predict a return to old-school techniques such as door-to-door surveys, which they think could bolster polling’s legitimacy.

“I know it sounds primitive,” Mr. MacKay said.

Primitive maybe, but it could just prevent another Wildrose moment for pollsters.