Party leader ratings: ‘satisfied or dissatisfied’ vs ‘doing well or badly’

Anthony Wells recently looked at the party leader ratings produced by YouGov’s and MORI’s opinion polls. He noted that MORI were giving Ed Miliband higher ratings than YouGov, and argued that part of this difference is due to the questions the pollsters asked:

YouGov: Do you think Ed Miliband is doing well or badly as leader of the Labour party?
MORI: Are you satisfied or dissatisfied with the way Ed Miliband is doing his job as leader of the Labour Party?

The key, Anthony suggested, is in how different party supporters respond to these two wordings.

Miliband approval ratings in December MORI poll:
Con supporters – 25% satisfied, 61% dissatisfied
Lab supporters – 54% satisfied, 37% dissatisfied
LD supporters – 33% satisfied, 51% dissatisfied
Miliband approval ratings in December YouGov poll:
Con supporters – 8% well, 87% badly
Lab supporters – 59% well, 31% badly
LD supporters – 24% well, 63% badly

You can see where most of the difference lies – amongst Labour voters the answers are not that different, Miliband’s approval rating is in the 50s, his disapproval in the 30s. The big difference is how the supporters of opposing parties answer the question. Basically, if Conservative supporters are asked if Miliband is doing well or badly, they overwhelmingly think he is doing badly. Asked if they are satisfied or disatisfed with his leadership, a significant minority of Tory supporters say they are satisfied – presumably because they are perfectly satisfied with Labour having a leader who they think is doing badly.

So the suggestion is that while some Conservative voters will be ‘satisfied’ with Miliband doing well (the more straightforward ones), others will be ‘satisfied’ with him doing badly (the more cynical ones). If this is true, it means MORI’s satisfaction ratings are unsound, because they’re mixing together two very conflicting things rather than measuring one. In contrast, YouGov’s ‘doing well or badly’ question would be a sounder measure of (straightforward) approval.

I tested this theory, by looking into the pollsters’ data on Miliband’s ratings across the whole of 2011, seeing how Conservative and Labour voters judged him as doing well and how satisfied they were with his performance.

If the YouGov ‘doing well’ ratings are sound, then those from Conservative and Labour voters should be positively correlated – that is, both groups’ ratings of Miliband should rise and fall in a similar pattern. (Of course, there will be some things Labour voters like him doing that Conservatives won’t, and vice versa, so the correlation will be far from perfect, but there will also be things that appeal or repel across the board.)

This is exactly what I found: the correlation between how many Conservative voters say Miliband is doing well and how many Labour voters say the same is +0.46.

But if the MORI ‘satisfied’ ratings are unsound, then those from Conservative and Labour voters should be much less positively correlated than the ‘doing well’ ones are. The more straightforward Conservatives will lean the same way as Labour voters when Miliband does something good or bad, but this will be at least partly cancelled out by the cynical ones who lean the other way, only happy when he looks like a doomed bungler.

This is exactly what I found: the correlation between how many Conservative voters say they’re satisfied with Miliband and how many Labour voters say the same is just +0.04.

This supports the theory that MORI’s ‘satisfied or dissatisfied’ question doesn’t produce meaningful ratings because some people are satisfied by success and some by failure.

Here are the graphs:



A couple of notes:

I didn’t look at how Lib Dem voters rate Miliband because there are so few of them. I don’t say this to be snide; it’s just that smaller groups of people in polls produce more wildly and randomly varied results.

I didn’t look at rival supporters’ ratings of Cameron and Clegg because both pollsters ask about Cameron “as Prime Minister” rather than as Conservative leader, and MORI asks about Clegg “as Deputy Prime Minister” (YouGov asks about him as Lib Dem leader). It’s one thing to want the opposition leader to screw up his own party’s chances, but it takes a lot more cynicism to hope those in government will screw up the country. So I expect that any converse effect – Labour voters becoming more satisfied when Cameron screws up – would be much smaller. If you’re interested, you could try digging up their ratings from before 2010.