On how playing with the census creates problems with the data, cf Canada 2011

The results of the 2011 Canadian census are in, and informative press releases have been coming out, like the release of the 24th of October outlining the changing patterns of language spoken and used in Canada.

There's a problem. I made a series of posts at Demography Matters concentrating on how the desire of the incumbent federal government in Canada to abolish the long-form census, putting questions on identity and such matters off to a voluntary form, would damage the integrity of the data series. Well, as reported by CBC, that happened.

Forgive the extended quote, but it's necessary to make the point.

Prime Minister Stephen Harper's cancellation of the long-form census has started to take a toll on Statistics Canada's data.

The agency released its final tranche of the 2011 census last week, focusing on languages, but it included a big warning that cautions data users about comparing key facts against censuses of the past.

"Data users are advised to exercise caution when evaluating trends related to mother tongue and home language that compare 2011 census data to those of previous censuses," Statistics Canada states bluntly in a box included in its census material.

[. . .]

In the past, the language questions were mostly included in the long form, which went to 20 per cent of households. When Harper cancelled the long form, several groups concerned about tracking the vibrancy of French in Canada went to court to make sure information about official language usage was properly collected.

As a result, the Harper government agreed to move the language questions to the short form, which went to 100 per cent of households.

The problem was that the language questions in 2011 were presented in a different context than they were in 2006, explained [Jean-Pierre Corbeil, the lead analyst for the languages part of the 2011 census]. In 2006, they were preceded by other questions about ethnicity and birthplace. Now, they appear suddenly after basic demographic questions.

The context of the questions has changed dramatically, likely prompting people to answer the questions truthfully, but differently, Corbeil said.

"We reviewed everything. Everything is really OK. The only thing is, we know that the responses we get are really influenced by the context and the placement in our questionnaires."

The main problems arise in how respondents reported their mother tongue and the language they spoke at home. Based on what Statistics Canada knows about immigration, there were far too many people claiming to have two mother tongues -- an official language plus a non-official language -- and speak an official language plus another language at home.

What first set off alarm bells for Corbeil was the proportion of people reporting English as a mother tongue. The raw data from the 2011 census told him it was 58 per cent. That was the same percentage as in 2006, but in the meantime, Canada had received about 1.1 million new immigrants.

And Citizenship and Immigration data, as well as Statistics Canada's own research, told him that 80 per cent of those immigrants did not have English or French as a mother tongue.

If people had responded to the 2011 census in the same way as the 2006 census, the proportion of English-speakers "would have been lower," Corbeil said.

He looked further and found more strangeness. Between 2001 and 2006, the census found there was an increase of 946,000 in the number of people who claimed a non-official language as a mother tongue. But between 2006 and 2011, that number dropped to 420,000.

That's less than half the increase noted earlier in the decade, even though immigration levels continued to rise at the same rate.