First, my apologies to French-Speaking followers since I am going to write my messages on the Scottish referendum solely in English. And apologies to English speakers too: My English writing is not perfect.
The referendum campaign for Scottish independence is under way. It is the first time where it is possible to compare a political campaign with the referendum campaign pertaining to Quebec sovereignty in 1995. The Quebec referendum was held on October 31th 1995 and ended with a very close result, i.e. 50.5% No and 49.5% Yes. The turnout was 93.5%. In both situations, the movement for Independence is taking place in a developed country that is part of the Commonwealth.
For this first post, I will compare the Scottish polls with the Quebec polls during similar periods and I will address the question of attribution of preferences to non disclosers, i.e. respondents who do not reveal a preference or state that they will not vote. In following posts, I will address the question of whether there are differences between polls' estimates according to the methodology used -- mode of administration, question wording, etc. -- and I will deal with the question of whether young people's preferences differ from older people's.
What is the state of public opinion?
a) Including non disclosers
The following graph shows the evolution of support for Scottish Independence. The non-disclosers -- all those who say they don't know how they will vote or state that they and will not vote -- are included in the graph.
It shows that although there was a possible increase in support for Independence from January to March, support seems to be going back to what it was at the beginning of the year. In addition, we see that there is much variation between the estimates from the different pollsters. The proportion of Yes is at about 35% but it varies between less than 30% and close to 43%. The proportion of No is close to 50% on average but the range of estimates is also large, i.e. 10 points. Finally, the proportion of non disclosers stays between 10% and 20% except for some specific polls where it reaches 30% or more. Those polls were generally conducted face-to-face.
If we look at a similar period in Quebec, i.e. January to June 1995 -- the referendum was to be held on October 30th -- we get a quite different portrait. Notice that since the question wording had not been decided yet at that time in Quebec, pollsters used different wordings and tested support for different options and according to different wordings. In order to have enough data as well as a relatively homogenous information, I kept the estimates for all the questions pertaining to a vote for either sovereignty per se or sovereignty with an association with the rest of Canada. Here is the portrait:
In the Quebec situation, we notice first that the Yes and No sides were closer than it is in Scotland and second, that the proportion of non-disclosers decreased in the last weeks before summer with a concurrent increase in both Yes and No support.
b) A few words about the non-disclosers
Is there a relationship between the proportion of non-disclosers in the polls and the distribution of Yes and No? The clear answer is that for these periods, there is absolutely no significant relationship between the proportion of non-disclosers and the distribution of support for Yes and No in Scotland as well as in Quebec. In Quebec however, following suggestions from researchers -- Maurice Pinard, Pierre Drouilly -- based on empirical findings, there has been a tendency to attribute more non-disclosers to the No side -- between 67% and 75% -- than to the Yes side. The rationale for this practice was that the socio-demographic profile of the non-disclosers is more similar to the profile of No voters -- more likely to be older, for example -- than of Yes voters. This practice allowed for a very good estimate of the referendum results in 1995. Without it, the predictions would not have been accurate.
Should Scottish pollsters proceed to a non proportional attribution of non-disclosers? This question should be examined. In any case, it is highly possible that the No side is underestimated since when polls are not accurate, it is usually the status quo side, the more conservative side of the electorate, that is underestimated.
c) Excluding non-disclosers, with proportional attribution
If we exclude the non-disclosers, we have to decide how to attribute them between Yes and No. The common use is to do it proportionally. The hypothesis here is that the preferences of those who do not reveal a preference are similar to to the preferences of those who do reveal them. Notice however that this means that when the proportion of No is close to 60%, proportional attribution gives to the No side a proportion that is close to the non proportional distribution that was used in Quebec. In short, non proportional attribution has a more substantial effect when the distribution of Yes/No is close to 50%.
Here is the portrait of the evolution of support for Independence in Scotland with proportional attribution of non-disclosers:
It shows the No side between 55% and 60% with the difference between Yes and No being close to 15%, and no overlap of Yes and No, whatever the poll.
Now, let's see what was the situation in Quebec at about the same period with a similar proportional attribution of non disclosers. The graph shows that at most, there was a 10 point gap between Yes and No around the middle of March. By May, the gap had closed and, by the end of June, the difference between Yes and No was 5 to 6 points .
In Quebec as in Scotland, the campaign started with a deficit for the Yes side. However, the deficit was clearly less substantial in Quebec than it is right now in Scotland. Campaign dynamics vary within and between countries. It will be interesting to follow the Scottish situation and examine the methodological questions that it will raise. Will opt-in panels -- more than two-thirds of the polls -- overestimate the Yes side? Will polls in general be accurate using a proportional attribution of non disclosers? How will young people vote? Will they be much more favorable to Independence like in 1995 in Quebec or will they have similar preferences as the other age groups, like the current situation in Quebec?
P.S. Methodological information for specialists: The lines -- the likely evolution -- are estimated using weighted local regression (loess) with Epanechnikov estimation using 65% of the points for adjustment. In various trial and error with different "smoothers", it appeared the best ans most flexible compromise and it gave very good results in the last Quebec election in 2014.