The Democratic Society (in conjunction with Snook) recently published a report for GDS to look retrospectively at the digital aspects of government consultation and make recommendations for a future model.
It was disappointing that The Consultation Institute had not been involved at greater depths. Not least, tCI have experience of evaluating online consultation tools and tCI have a robust, proven mechanism for delivering effective public consultations.
Online consultation makes sense from a number of perspectives – it is potentially more convenient, cost effective and can attain greater reach. However, current implementations do not harness the benefits of the channel well. For example, it is rarely possible to gauge if the Consultee has an informed view (i.e. if they read the supporting material and how much time they allocated to their response). I don’t think I have ever seen a tool which estimates salience based on a consultation theme or provides a measure of confidence in the responses.
I agree with the DemSoc report in as much as (historically) the variety of ways to capture views online is not very dynamic. While submitting views using video or by telephone (IVR) may create an analysis burden, more interactive methods such as drawing on a map or commenting in-line on texts are example of where the digital channel is advantageous.
Ultimately my experience of online tools is that they do not focus on good process and subsequently fall short of good practice. For example, they do not require you to set-up a consultation mandate or consider the process of stakeholder identification. They do not help you pre-select statutory Consultees. They do not help you decide if consultation is even necessary, if there are any specific legal requirements (e.g. Aarhus) or help you write well-structured questions.
Commercial software is also disappointing because it tends to be narrowly focused. Out-of-the box consultation tools rarely include stakeholder tracking capabilities, qualitative data analysis features, social media monitoring or help Consultors promote a consultation once it has been published.
There is some blurring of requirements in terms of having tools which are dialogic (and thus necessary for option development) and those which are focused entirely on structured responses. There is also confusion over the application – informal, formal, public, stakeholder.
There is further blurring of who consultation tools are for. Government has many tools at the disposal of civil servants yet my feeling is that stakeholder representatives and MPs do not have the same, devolved capabilities.
Step one for the uninformed : ready my publication for the World Bank on what constitutes meaningful consultation
What can government do?
The DemSoc report makes some good recommendations about improvements to the gov.uk consultation finder. Of course, I will add to them! Let’s take a look at the DemSoc recommendations in turn:
- The presentation of consultations on Gov.uk should be updated based on a dedicated, purpose built web page for government consultations.
Not much wrong here. There is probably sense in making the closing date more visible and listing consultations by their proximity to being closed. The most confusing part of the existing finder is that the status of each consultation (e.g. open, closed, report published) is not clear. Personally I also have no problem with showing how many responses have been received.
My recommendation would also be to look at filtering consultations by key stakeholder requirements. For example, it is impossible to tell which consultations have a requirement to capture broad-brush public opinion and those which seek a certain sector of the community, such as businesses or disability groups.
The recommendation about ‘related’ consultations is also key. The consultation finder has promise in terms of its ability to help civil servants avoid duplication and potentially merge proposed or planned consultations.
Finally, consultation alerts are very important and making this feature more prominent should definitely be a priority. However, it is the lack of alert to the output of the consultation (the report) and outcome (the decision) to Consultees which is severely lacking. I think the report alludes to this in recommendation 2(f).
- The individual consultation page template should be reordered.
Absolutely. Actually, here I endorse the use of the tCI consultation mandate which effectively covers the recommendation in 2(a). The report does miss out three essential ingredients in terms of framing a consultation – who is consulting (the ‘we’), who will be taking a decision and when the policy might be implemented by.
Consultation narratives often get detached form consultation questions and the way in which questionnaires are often formed does not allow key arguments or evidence to be presented in a way which can be back-tracked. Deep linking could help here.
There are probably lots of other element which need adding on to a summary page. There is very little in terms of explanation about data protection and what will happen to submissions in terms of transparency and data handling. There is very little presented about equalities impact and equalities monitoring.
- Better support and guidance should be available for civil servants
Preparation is key and the ability to create an (a) legal and (b) effective consultation will largely not depend on the technology used.
My view is that the consultation back-end should go further than nudge and reliance on a design manual. Why not just make all consultations comply with good practice? For example, making it impossible to create a consultation which is not open for a sufficient duration (particularly over a holiday period) or where the decision making process is not sufficiently long (and hence illegal).
Capabilities here could be comprehensive. For example, Consultors could be forced to specify any preferred option and telling Consultees about any alternatives which were discounted.
Help with question formation is particularly beneficial. For example, ensuring that questions are supplemented by contextual help (e.g. ‘tick all that apply’) and allowing Consultees to recycle generic questions which should form a part of any questionnaire (e.g. ‘about you’).
The Consultation Institute has many courses which cover good practice consultation and this rarely translates into a manual. However, my recommendation would be to turn the design manual into design tools. For example:-
- Stakeholder mapping & identification tool
- Option development & scoring tool
- Popular question and questionnaire templates
- Database of response rates by theme
- GDS should create an open and extensible core infrastructure for consultation into which consultation tools from in-house or external developers can be plugged
This is perhaps the most comprehensive recommendation. Adopting a common approach is certainly essential – it is strange that there are a mix of ways to give views – from terrible electronic ‘downloadable’ forms to Survey monkey questionnaires.
On modular design
Government could try to adopt some of the free and open source functionality that already exists for government in this domain. For example, Madison.io for commenting on text and the free eParticipation toolkit at https://opin.me.
GDS can certainly help Consultors match the type of digital engagement method required by the consultation type (e.g. via www.engagementdb.org) and subsequent tool/feature requirements.
The problem with the current proposal is that incorporating ideation and discussions takes the mission towards less formal consultation. While it is absolutely necessary for government to have access to these type of tools for engagement purposes, in this context they need to be set in the process of consultation. For example, it would be great to be able to discuss the outcome of a consultation (hence avoiding the run on the GDS blog) and it would be great to be able to use ideation to help form option development at the pre-consultation phase.
As such and under the current recommendations, choosing a ‘discussion’ consultation is not formal consultation at all – as it may not result in any policy changes or the results of which may never get heard. A clear link to the policy-making cycle is needed and Consultors need to understand the weight of their views.
Certainly it would be nice to allow Consultors to have access to a wide range of elements that they could use in a survey design. For example: maps and plans, images, videos, text narration. This could be extended to features which allowed Consultors to create paper versions which can be computer processed.
There is also an overlap in terms of democratic instruments used by citizens and lobby groups to engage with a consultation. For example, signing a petition or using FOI or parliamentary questions. Consultations need to tie-up these component parts.
Research suggests that an identifiable Consultee will increase the quality of a debate. However, there is a point at which identity stifles contribution and take-up of the opportunity. It could certainly be a disaster if a Consultee who was taking a controversial stance could be identified.
Consultors need to understand ‘who’ is speaking but only in terms of a Consultee profile (e.g. age/ethnicity) and who they represent (e.g. carer, parent etc.).
There are some very poor consultation analysis reports coming out of government. The robustness of the qualitative analysis is unclear and the presentation of quantitative results is often misleading (e.g. no actual numbers used or margins of error displayed in graphs).
The DemSoc report does well to make recommendations around standardisation. For example, presenting results question-by-question. Releasing qualitative data as open data for third party analysis would be very interesting. Similarly, the ability to ‘dig down’ into responses would be very compelling. Some of the commercial analysis software (e.g. QSR NVivo) already allows this to happen as publishable to the web.
- GDS should demonstrate the modular approach by making available at least one survey module, as a whole of what can be achieved using the core infrastructure.
I think this is possibly a red herring. Given that government is heavy on using WordPress there are many (and mature) survey plugins which could be quickly adopted. What is needed is some appraisal of each and then some in-house adaptation/adoption.
Analytics should be the focus of effort, ensuring that stakeholder interactions can be tracked (similar to functionality of commercial software such as StakeTracker or Darzin) and that there is a common system for collecting and analysing the qualitative data. Moreover, GDS could incorporate APIs form third party vendors such as theysay.io to embed natural language processing in the discovery of key insights such as sentiment.
- GDS should stimulate the existent civic tech market, releasing specifications for modules providing different types of interaction, and ensuring the release of all information required to support developers in taking advantage of the core infrastructure.
Wow, this would certainly create a hybrid tool. However, as I have discussed, it is not the tools which are the problem. For example, a typical consultation response rate is still only 1% of the local population affected – even on salient issues such as hospital closures.
I think this is too cursory and I would start by looking at the way in which digital can support the other aspects of engagement. For example, the whole issue of social listening (i.e. via social media) is somewhat washed under the carpet. Despite only 2-3% of social media traffic being useful on any particularly issue, consulting in a space or platform where audiences already exist has to be beneficial.
Eurostar use text messages to consult on passenger satisfaction after or before a journey has commenced. This combination of platform and timing yields them great results. GDS can get creative in the way that audiences are attracted-to or engage with consultations (e.g. gaining points or qudos) and by considering the ‘what’s in it for me?’.
If I were GDS, I would certainly be creating a social media listening dashboard as an outlet for extra insights. Perhaps with the help of some clever machine learning methodology such as that deployed by http://deeplinkingyouth.eu/
It is the enhancement of the consultation process that technology can support which should be seen as the real win for GDS and their online consultation parenting. While the existing feature set can be improved, standardisation is the big win for all users. There is a lot to learn about good consultation design, management and stakeholder involvement – digital can enhance consultations and help with compliance but Consultors are in desperate need of understanding good techniques and principles.
I think GDS can make a lot of gains in terms of applying technology to increase trust that consultations are not flawed. Moreover, to make consultation more appealing to both decision makers and Consultees.
The gates are open in terms of the methods and interfaces that government uses to gain views – the DemSoc report only covers online consultation but there are a plethora of other digital ways, such as using Chip and Pin machines to ask consumers simple questions at the point of purchase.
What I really think is that government needs a ‘central team’ of experts to deal with public consultation, engagement and involvement. An integrated unit to be able to manage the various consultations, analyse pubic views and feed them into policymaking. Transport for London are getting there….
Meanwhile I wish them the best of luck with any transformation and provide them with an open invite to a more lengthy discussion.