Was it good? How to evaluate citizens' assemblies

24. November 2021

In recent years, the number of randomly selected citizens' assemblies and similar deliberative processes has grown rapidly around the world. New figures show that this trend is not only continuing, but has gained strength and momentum in recent years. A database of the Organisation for Economic Co-operation and Development (OECD) lists nearly 600 procedures, 101 of which have taken place since 2019. It is therefore important to assess the quality of these deliberative processes as uniformly as possible with high auditing standards. The OECD published guidelines on this on 24 November 2021.

Purpose of the evaluation guidelines

The purpose of the evaluation guidelines for representative deliberative processes such as citizens' assemblies and planning cells is to support public authorities in initiating and developing better processes by setting minimum standards for their evaluation.

Within the framework of evaluations, the following questions, among others, are examined:

Does the deliberative procedure fulfil the claim of being a reflection of the population according to criteria such as age, gender, education, place of residence and migration background?
Was the question formulated for the process suitable and easy to answer?
Was an inclusive, equal, active participation of all participants guaranteed?
Was the entire spectrum of political opinion able to articulate itself?
Did the process offer equal opportunities to express one's own opinion, respectful interaction between participants, the opportunity to speak freely and be heard, and effectiveness of one's own arguments?
Was the information provided in the proceedings balanced?
Were the materials, tools and methods used helpful?
Was the facilitation valuable and successful for goal orientation as well as for structuring and bringing together the participants' contributions?
Did the participants' attitudes towards the topic change as a result of the information and discussion in the deliberation process?
Did the participants' interest in the topic and their own perception of competence change?
Were the participants generally satisfied with the event?
Was there a good connection of the procedure to the government, parliament, mayor, local council, etc.?

In order to be able to answer these questions, the participants randomly selected for citizens' assemblies, for example, are asked for feedback on their experiences in interviews and by questionnaire. Organisers of a deliberative process and political actors are also asked.

The OECD Guidelines are intended for policy-makers, evaluators and practitioners who want to evaluate the deliberative processes they have initiated, commissioned and implemented and assess them in terms of their quality. The Guidelines set a minimum standard for evaluation by providing rationales, a framework, measurement methods and evaluation questionnaires.

Evaluation as a key element of success

Evaluation of deliberative processes is a key element for their success. Timely evaluation strengthens the confidence of policymakers, the public and advocacy groups in the recommendations produced by a deliberative assembly, as it can provide evidence of the quality with which they were produced.

By evaluating deliberative procedures, the commissioning authorities demonstrate their commitment to transparency and quality. This gives the procedures greater legitimacy. Evaluation also creates learning opportunities by providing authorities and users with evidence of what went well and what did not.

Principles for conducting an evaluation

Independent evaluations are the most comprehensive and reliable method of assessing a deliberative process. For smaller and shorter deliberative processes, evaluation in the form of reports by the members and/or organisers of a deliberative process can also have a learning effect. The OECD has developed principles that can guide evaluation and ensure its quality and integrity.

The OECD strongly recommends that deliberative processes, which are becoming more widespread, be evaluated because they have an impact on the public, on policy-makers and on their decisions.

Learning from evaluations for the future

Expanding the evaluation of citizens' assemblies, planning cells and similar deliberative processes will allow policy-makers, observers and the public to assess the quality of these processes, learn from past experiences and ultimately help initiate and develop better processes in the future.