Posts Tagged 'evidence-based policy'

We need more experts participating in political debates: Continuing the legacy of Professor Carol Weiss


Iris Stucki, the winner of the 2017 Carol Weiss Prize for outstanding early career research, discusses her winning article, ‘The use of evidence in public debates in the media: the case of Swiss direct-democratic campaigns in the health policy sector’. The article is published in Evidence & Policy and is free to access for 3 months.

Arguments referring to evidence are rare in Swiss direct-democratic campaigns. I took 5030 media items and analysed how many of them made a reference to evidence. At less than 7% the result is sobering, and experts, the actors that make most use of evidence in their arguments, are also scarce.


Why is this result of deep concern?

The success of democracy depends on an informed public. To be able to make good decisions, voters need to receive information about evidence, in particular evaluation studies, showing whether a policy works or not. Of particular importance for Swiss voters in deciding about a policy is the campaign coverage by the mass media. Here, debates between actors with different interests take place and arguments from both proponents and opponents are conveyed. And here, political arguments could be substantiated by evidence.

“The success of democracy depends on an informed public.”


Political use of evidence

Political use of evidence takes place when evidence is used to legitimise a predetermined position. Political use of evidence has had negative connotations for a long time, because, research has often been intended for use in improving and adapting political measures, rather than being used in political arguments. However, the positive view of the political use of evidence recognises that evidence is open to interpretation. Against this background, the use of evaluation studies and other research to support political arguments is nothing to condemn. On the contrary, presenting different evidence-based perspectives enriches political debate. As early as 1979, Carol Weiss stated that research, to the extent that it supports the position of one group, “gives the advocates of that position confidence, reduces their uncertainties, and provides them an edge in the continuing debate”.

My analysis of the use of evidence in direct-democratic campaigns shows that evidence is almost exclusively used in a political way. The good news is that the Swiss media display proponents and opponents in their political use of evidence in a balanced way, that is, pro and con arguments are conveyed in a similar proportion. The bad news is that not all of the actors are given equal coverage. Journalists and politicians dominate the discourse, while experts, the actors most likely to ground their arguments in evidence, appear most rarely. One way to improve this situation would be for the media to integrate experts to a greater extent in their reporting. The simple solution, a fruitful collaboration between journalists and experts seems to be complicated in reality.


Knowledge-based journalism

In an ideal world of knowledge-based journalism, journalists serve as explainers of science and facilitators of evidence-based discussions while experts recognise that they have a role to play in educating the public in policy debates. However, such collaboration seems to be tough for experts especially, as they have to be convinced that they want to participate, to take position and to eventually let go of their knowledge. This is best illustrated by a statement in a discussion forum on the question why there are so few experts in political debates. One discussant said that experts have to abandon a part of their identity as scientists when intervening in the world of politics, and have to show idealist ambitions to engage in political debates.

“Ultimately, both journalists and experts are in pursuit of the same goal: an enlightened public to avoid the emergence of a post-truth democracy.”

But perhaps this is the path to take. Ultimately, both journalists and experts are in pursuit of the same goal: an enlightened public to avoid the emergence of a post-truth democracy. Thus, I close with a call for more experts to participate in political communication. I draw, again, on Carol Weiss, who recognized 20 years ago, that experts have the capacity and the responsibility to actively present evidence in the public arena and explain its scope and relevance to citizens. I am convinced that when experts who are involved in the production of evidence collaborate with journalists and publicly share analysis that is relevant in the political world, they both contribute to making democracy more evidence-prone, and citizens more enlightened.


Iris Stucki is deputy head of the Federal Office for the Equality of People with Disabilities in Switzerland. She received her PhD in Public Administration in 2016 for her dissertation on the use of evidence in direct democracy. Her research interests cover evidence-based policy making and voting behaviour.

Her article ‘The use of evidence in public debates in the media: the case of Swiss direct-democratic campaigns in the health policy sector’, published in Evidence & Policy is free to access for 3 months.


Is the new impact agenda the excuse you’ve been waiting for to use your research to make a difference?

Sharon Wright and Peter Dwyer, researching the impacts of Universal Credit since 2013 as part of the collaborative ESRC Welfare Conditionality project, reflect on their recent experience of contributing to the Universal Credit debate, to argue that impact activities can be most meaningful if they are aimed at making a difference that really matters.

Dr. Sharon Wright

Prof. Peter Dwyer

The news that research impact will account for a quarter of a unit’s score for the REF2021 research excellence rankings has piqued the interest of cash-hungry University leaders across the country.

With the most significant and far reaching impacts bringing in around £324k, pressure is building for academics to strike into uncharted knowledge-exchange territory to secure elusive high-earning 4* impact case studies.

But if the thought of money as a motivator leaves you cold – and the more familiar competing pressures of teaching, administration and research offer space for little else – is there an alternative way of looking at the new drive for impact?

“Impact activities can be most meaningful if they are aimed at making a difference that really matters.”

In October 2017, Universal Credit (UC) hit the headlines with public outrage at claimants unable to afford to eat and at risk of losing their homes because of the built-in delay of 6 weeks for the first payment.

One of the greatest injustices is that Universal Credit was sold to the electorate as a reform aimed at simplifying the system and making work pay, and as such, it was originally welcomed widely. However, design flaws are being exposed as contributing to rising foodbank use, homelessness and destitution.

House of Commons

Secretary of State for Work and Pensions, David Gauke, has been resistant to calls for urgent action to restore UC in line with its original policy aims. On 18th October 2017, a unanimous group of opposition MPs won the landmark House of Commons vote, 299 to zero, to ‘pause and fix’ the Universal Credit roll-out.

Decisive to the vote and the ongoing debate, were SNP MP Neil Gray’s authoritative parliamentary speeches, which used cutting edge research evidence, including our article on ‘Ubiquitous Conditionality’, alongside the experiences of his constituents to substantiate compelling arguments for reform:

“The Government should review the cuts to the work allowances, which are acting as a disincentive to work and making work pay less; review the cuts to housing benefit, which are driving up rent arrears […]; and review the cuts to employment support, which are denying help to those who need it most, and they should fully review and then scrap the disgusting sanctioning policy, which could have cost the life of my constituent, Mr Moran, and has cost the lives of others. That was the subject of an excellent paper by Sharon Wright of Glasgow University and Peter Dwyer of the University of York in The Journal of Poverty and Social Justice.” Read the full transcript of the debate here. 

How did we achieve this impact? Sharon met with Neil Gray on a panel discussing ‘Rethinking Poverty’ at the SNP Conference in Glasgow. Following this, she watched a clip of Neil’s first Universal Credit speech and let him know that our research published in The Journal of Poverty and Social Justice (including the article that was featured in a free collection at the time) backed up several of the points he had made. Via Twitter and email, Sharon sent Neil a link to our recent blog and responded to a follow-up query with additional research evidence. Neil then used the evidence in his subsequent speeches and said:

“Academic and well researched evidence on the impact of
Universal Credit is crucial for persuading government to
change its mind and fix the system as it is being rolled out.

Neil Gray

Sharon’s research and input has been invaluable for me in
setting out the case that I have in the House of Commons.
The government can try to dismiss or ignore political debate,
but personal testimony and independent academia is harder
to ignore.

I hope Sharon and others will continue to look at issues like
the social security ‘reforms’ so that government policy can
be effectively challenged and hopefully overturned, to help
people who desperately need that support.”

As an impact activity, the process was quick, easy and direct. The result was Neil’s exemplary use of research evidence for accurate and well-informed debate that continues to feed into meaningful changes to policy and practice.

“…exemplary use of research evidence for accurate and well-informed debate that continues to feed into meaningful changes to policy and practice.”

The focus throughout was straight-forwardly on the issues that matter. For us as academics, the current importance placed on impact activities offers legitimacy to carve out the necessary time to do exactly what we have always wanted to do – proactively engage with policy makers, in a policy field where robust evidence has gone against the grain of dominant political preferences, to use research to make a difference.


Universal Credit, ubiquitous conditionality and its implications for social citizenship from The Journal of Poverty and Social Justice, is FREE to read on Ingenta until 31 December 2017.

Policy Press has always worked closely with authors to ensure their work gains the social impact it deserves alongside academic impact. Find out more here.

Please speak to your marketing contact at Policy Press for advice and support, or email

Policy Press newsletter subscribers receive a 35% discount – sign up here.

Image: UK Parliament, ‘House of Commons: MPs debate 2013 Queen’s Speech‘ Flickr Creative Commons CC BY-NC 2.0

UK “evidence free” government since 2010: What can we learn from the Dutch analysis of manifesto pledges?

In the May issue of Evidence and Policy Sources and resources section, Mark Monaghan interviews  Netherlands Bureau for Economic Policy Analysis (CPB) Wim Suyker about the Dutch approach to manifesto pledge analysis. In the run up to the UK General Election it seems pertinent to share this interview as we all ready ourselves to make decisions on who will govern us over the next five years.

EvP 2013 [FC]The 2010 general election signalled a turning point in the fortunes of evidence-based policy-making. From a high-point under the early iterations of the New Labour government, it has become apparent that the current Coalition government have been less preoccupied with highlighting the evidence-based credentials of their policy formulation.

Recent commentaries have gone as far as to highlight the ‘strange new world of evidence-free government’. It is, of course, debatable how ‘new’ this actually is. However, a paradox of the declining prominence of evidence within government is the increasing scrutiny of evidence use from outside in the form of a seemingly increasing desire on behalf of the public to scrutinise the work of politicians.

This is possibly (probably) a response to the Parliamentary expenses scandal of 2009, but within this climate, many new initiatives such as the FactCheck blog hosted by Channel 4 and the creation of an Office for Budget Responsibility (OBR) have come to fruition. According to its web pages, the OBR was created in 2010 to ‘provide independent and authoritative analysis of the UK’s public finances’.

More recently in conjunction with the Alliance for Useful Evidence, The Conversation has launched its Manifesto Check service. Such initiatives may seem long overdue, yet they have been a common feature of the Dutch political system for many years, particularly in the work of the Netherlands Bureau for Economic Policy Analysis (CPB).

Established in 1945, the CPB is an independent organisation working ‘at the crossroads of economics and public policy’. It provides quarterly forecasts on the state of the Dutch economy. Importantly, it also offers all interested political Parties an analysis of the mainly economic effects of their manifesto pledges.

This analysis then helps shape political debate in the run up to General Elections. As we approach the 2015 General Election it is necessary to see what lessons the UK (and elsewhere) can learn from Dutch experience. In this interview Mark Monaghan discusses the CPB Manifesto Project with Wim Suyker of the CPB.

Monaghan: OK, so could you please tell our readers about the work of the CPB and your work within it?

Suyker: We are an institute employing around 100 people. We do various research projects, but we are also responsible for the official macro economic forecasts used for the budget. In that way we are comparable with the Office of Budget Responsibility in the UK. So that is our main activity. I could tell you about our history, but that is all the things that you can find on our website.

Monaghan: Could you tell us more about the Manifesto’s Project? How long has that been running and do all Parties participate?

Suyker: We have been running that since 1986. All Parties can decide if they want to join this exercise. It is on a voluntary basis in the sense that there is no law that says Parties should participate in this. It can, however, lead to awkward questions from the Press and other parties if Parties do not cooperate in the project. Last time we had ten Parties involved in this project. It was in 2012.

Monaghan: So there were ten Parties who submitted their Manifestos to yourselves, how many Parties in total contested the elections?

Suyker: There was only one Party in Parliament (Party for the Animals), that in principle didn’t participate in the project. It is not their cup of tea. There was another Party who in the election polls had 1 seat – this was the party of the Retired Persons (50PLUS) – they could have joined the project, but they decided not to do it. This was because they were very new and they didn’t have the staff etcetera.

Monaghan: OK, so it tends to be smaller, single issue Parties that don’t participate?

Suyker: Well, it depends. For instance, we have the Socialist Party. They are to the left of the Social Democratic or Labour Party and they are always very explicit in the way they decide whether to participate. ‘Is there something in it for us or not’? So the fact that basically now all Parties are participating now does not mean that it will be the same next time and it depends on the political situation. It is likely that the Parties who participated the last time will participate again in the coming projects, but it is not certain.

Monaghan: You mentioned earlier in passing the kind of, I suppose, penalty that Parties get if they fail to engage with the process, but that is not an official sanction?

Suyker: Oh no, it is more a judgement in the Press. For some parties – such as the Party for the Animals – it doesn’t hurt them by not doing it. It may hurt them if they are participating because their electorate may not like this kind of economic analysis.

Monaghan: And so that brings me on to the next question which is on what grounds are the Manifestos scrutinised? Is it just economic policy or does it cover social policies, health policies and so on…?

Suyker: Well, we have made a modification on this. In 2012 it was rather a broad exercise which we were doing with the Environmental Agency. We were also taking into account environmental impacts on policies. So we looked at fiscal and infrastructure policies. There was an analysis of energy and climate, innovation and education. You can get a flavour of it by looking through the English language summary. Having said that, there were a lot of issues tackled or topics tackled in the discussion before elections, but it is really the impact on economic growth in the short term, the impact on the Government budget in the medium term and the impact on employment in the long term that are the main issues discussed after the analysis is published.

Monaghan: So is the purpose of it to check facts?

Suyker: Perhaps I should clarify something. We don’t analyse the manifestos. The Parties make their election manifesto and on the basis of that they come to us with a list of measures that they want checking. So we don’t have to do an analysis of the Manifesto, it is for the Party to come with this list of measures, as concrete as possible, for us to look at. They may subsequently come with an additional list also.

Monaghan: So does that process take place publicly or privately?

Suyker: At that stage it is on a confidential basis. At the start of the process we make clear, by publishing on the website, what we are going to do. After that, it is down to Parties to decide whether to participate or not. We don’t comment in the Press which Party is or is not participating in the project. If a Party decides to stop its activity we will not comment on that in the Press. We will not say that a Party started the process and then quit. It is a thing for the Parties to say in the Press or not.

Monaghan: OK, and so the findings of the CPB are they published before or after the election?

Suyker: Before the election. Our aim is to publish that at least 3 weeks before the elections because we don’t want it too close to the election date. It is desirable that there is some discussion about the output we have, but it is also desirable that it is not too close to the election date.

Monaghan: Is that because you are concerned about the findings swaying public opinion in any way?

Suyker: Well in some ways for a good debate, you need some time to digest the complex issues. So I think time is desirable in a project like ours that the public can take through the Press the results, their can be some discussion about the results between the Parties and I think that is a fruitful input into the elections.

Monaghan: How do you decide the order in which the Manifestos are looked at?

Suyker: It is based on the outcome of the previous elections. The Party that has the biggest share of the electorate will be number 1 in the publication and they will be the first Party mentioned in tables and graphs etcetera.

Monaghan: So all the findings are published in one document? It is not a series of separate documents?

Suyker: No, we have one document. Last time that was 454 pages. Basically, that is it. Last times we gave a press conference at the time of publication. At the beginning of the process, we clarify the rules of the project. There is some additional material in which we do this and that is basically all the material involved.

Monaghan: So this 450 page document is then released to the Press

Suyker: Yes

Monaghan: And there is an executive summary…?

Suyker: Yes exactly. Then there are chapters. Every Party has its chapter and there is an introductory chapter which discusses the methodological issues involved.

Monaghan: Is there any sense from yourselves that the findings of the report can or do sway public opinion and is there any evidence for it?

Suyker: Well, you can see that the number of hits on our website increases a lot just after publication of the report and you can also see how it is mentioned a lot in the Press so it is clear that it is playing a role in the election process. What is also interesting is that after elections we have negotiations to form a Coalition and it is really obvious to say that the publication is an important input in this respect.

Monaghan: Is there anything specific to the discuss political system as to why this process works and would it translate to other places?

Suyker: What is peculiar or specific to the Dutch situation is that we always have a coalition government. The UK not also has its coalition government, but it is based on a senior/junior partner arrangement. Most of the time our Coalitions are different. Of course there is one Party who supply the Prime Minister but they are more equal partners. I think in this context the project fits very well.

It is the case that you also need to have authority and to be trusted by the public. I discussed this with French colleagues and for them it is hard to understand what we are doing. There needs to be consensus about how the economy works in broad terms. If you are considering a shock to expenditure there needs to be a consensus that an expansionary fiscal shock in the short-term will have a positive impact on GDP in the future. There should also be a consensus that changes in taxes and social security payments will change incentives to participate in the labour market and thus are influencing labour supply and employment in the long term that incentives on labour supply will have measurable outcomes.

It relies on a situation where there aren’t too many elements of the process that Parties don’t agree with. I think it is always for Parties to give and take. There may be elements that they like more than other elements, but they should accept the rules of the game.

Monaghan: So in your opinion, the project only works in the context that there is consensus across the political spectrum on what can be measured?

Suyker: Yes. Political Parties should accept that we are setting some rules for this project and they should be more or less happy with those rules. We always try to inform them of the rules some time before we start. We also discuss changes to the process, but there should be a consensus climate otherwise it would be very tough to do.

Monaghan: Have you ever discussed the process with counterparts in the United Kingdom?

Suyker: Well, Robert Chote [chair of the OBR] you mean?

Monaghan: Yes and others?

Suyker: We have given various presentations abroad. There are a lot of colleagues abroad who are interested in our experience and one of my colleagues this week was in Bratislava to explain the system. I’m pretty sure that if after the elections in the UK it is decided that they will have this kind of exercise then we will be in contact with the OBR.

Monaghan: So do you think this could be translated to the UK context, bearing in mind the level of consensus you suggested was necessary for the exercise to succeed?

Suyker: I think that I had more doubts about this three or four years ago, but if you have an OBR that is responsible for the economic forecasts then you have already made a big step in research-based policy making. So I think it could work well in the UK, but the Parties need to get the experience and you would have to show how there are trade offs in the process of decision-making. So I think it could work, yes.

Monaghan: OK, well I look forward to seeing whether it materialises

Suyker: Yes but from what I understand it is that after the elections Parties may change the guidance they are giving to the OBR or the Charter of Budget Responsibility of the OBR or something like that, but firstly we will have to see how the elections work out

Monaghan: Yes, would you basically see an expanded role for the OBR?

Suyker: I guess that is the most likely scenario. That is my guess, but I am not familiar enough with the situation to be sure, but the OBR is the most likely candidate to do this kind of project.

Monaghan: OK, that has been really interesting an helpful, so thank you so much for your time

Suyker: You’re welcome.

#GE2015 #whyvote #imvotingbecause

This article will be published in Open Access Sources & Resources section of the May issue of Evidence & Policy. For a free online trial to this journal, please sign up here.

The views and opinions expressed on this blog site are solely those of the original blogpost authors and other contributors. These views and opinions do not necessarily represent those of the Policy Press and/or any/all contributors to this site.

Enter your email address to follow this blog and receive notifications of new posts by email.

Twitter Updates


Creative Commons License

Republish our articles for free, online or in print.

The work on the Policy Press blog is licensed under a Creative Commons licence.

%d bloggers like this: