We are close to entering the last week of the campaign. I will present an update of my last week analysis but, for this post, I will mainly focus on the major differences between modes.
First an update
As we can see in this first graph, the progression of the leave side went on during last week. However, it is important to notice that the Stay side remained stable. Its support did not decrease. What happens in that the progression of the Leave side seems to come almost entirely from a decrease in the proportion of respondents who say that they don't know how to vote or that they will not vote. This proportion, as noticed in a previous post, varies substantially, from3% to 15% during the last week.
However, as I explained in preceding posts, empirically it is much more sound to attribute non-disclosers non proportionally, attributing more of them to the status quo side, in this case, Stay. If I keep the same non-proportional attribution that I used before -- 67% to stay and 33% to leave -- I get the following portrait of the situation. Support for Stay is slightly ahead of support of Leave, by about three points (it was five points last week).
It's all about modes
The first question is whether there is a difference between modes, controlling for change over time. In order to check for that, I perform a regression. The conclusions are:
- As we can easily see with the form of the curves in the preceding graphs, change over time is in the form of a reverse U (quadratic).
- For Stay with proportional attribution of non-disclosers, after taking into account change over time, web polls give on average 5.1 points less to Stay than telephone polls. The mode, by itself, explains 45% of the variation between polls (which is huge!).
- Using non-proportional attribution, the difference between modes is slightly corrected. Support for Stay in Web opt-in polls is 3.66 points lower on average than in telephone polls and mode explains 26% of the variation between polls.
If we use non-proportional attribution of non-disclosers, the portrait is similar but the endpoint estimate is slightly different, at 51.5%. However, since non-proportional attribution corrects for some of the differences between modes, there is no difference left in estimates according to mode.
Although the polls using different modes of administration do not trace the same portrait of change in support over time, it seems that, at the end, they tend to agree. So, as of now, we do not have to start a battle on who's right.
It seems to me that referendums on "independence" somewhat look alike, if one compares Quebec 1995, Scotland 2014 and the actual Brexit. During the campaign, the "change" side always gains support and, in the days before election, comes close to 50 percent . In Quebec and in Scotland, the "Law of even polls" was respected. This is, when the two sides are at par, the status quo side is likely to win. Why is that? We may speculate. It is possible that people who are for status quo tend less to reveal their preferences to pollsters or are less present in the samples. It is also possible that some people who are in favour of change are afraid of what could happen if change win by a tiny margin. they may therefore change their minds at the last minute. Anyhow, it is easier and less consequential to tell a pollster that you are going to vote for change than to do it for real. And "the message" is sent to leaders nonetheless.
Another nine days to go to see whether what happened in Quebec and Scotland will happen for the Brexit. We know however that the situation is somewhat different, in particular in the sociodemographic profile of the supporters for the two sides. Support for change in the actual campaign comes more from older people, who tend to turnout in larger proportions.
**Notice on methodology: In the graphs, each point represents a poll estimate positioned at the middle of the fieldwork; lines represent the likely change in support estimated using Loess (Epanechnikov, 0.65).
For methodologists and other interested people
A question to ask is whether there is more variation according to mode and whether there is variation within mode. The next graph shows a box-and whiskers plot of the variation according to mode in support for Stay with proportional attribution of non-disclosers. The graph again illustrates that support for Stay is estimated higher by telephone polls. However, there is not much difference between modes in the level of variation and not that many polls differ significantly from other polls using the same mode. Two poll estimates by Survation are significantly higher than the other web opt-in polls and one YouGov poll is somewhat lower. For telephone polls, ICM and ORB each have two polls that are somewhat low.
A general conclusion from this analysis would be that the major difference between modes is in the estimates -- in this case the median estimate -- not in variation. And there is not much difference within modes either.