Skip to main content


Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Table 3 Overview of experiences and recommendations for evidence synthesis in controversial contexts

From: Inclusive development and prioritization of review questions in a highly controversial field of regulatory science

Measures Experiences—assessment by the authors Recommendations
Openness and inclusiveness
 Broad stakeholder invitation—any interested stakeholder can participate The absence of participant selection prevented conflicts about the composition of the stakeholder group and contributed to trust building This approach needs supporting measures to ensure that interested stakeholder representatives can participate in all subsequent steps (e.g. travel support, smaller stakeholder groups)
 Multiple ways of engagement (workshops, written consultations) Online written consultations were mainly used to introduce issues not mentioned in or to clarify or emphasize discussion topics of the face-to-face consultations This set-up is useful for clarifying issues and views and for bringing up issues not addressed during face-to-face meetings, e.g. due to lack of time. It does, however, not help in engaging individuals unable to participate in the face-to-face consultations
 Familiarizing participants with the evidence synthesis process This measure is very important for a meaningful engagement. This works better if the same representatives of a stakeholder organisation participate in the entire consultation process The engagement process must be tailored to the level of knowledge of and experience with evidence synthesis
Measures need to be in place to ensure that the same individuals can contribute over the entire engagement process (e.g. providing travel support and opting for smaller stakeholder groups)
 Procedure for written responses to stakeholder comments The measure was extensively used by stakeholders
The procedure resulted in a systematic and transparent consideration of stakeholder comments by review teams and allowed for transparency of how stakeholder inputs shaped the process
Time and resource needs are significant and therefore preclude routine application. The procedure appears suitable for sensitive or controversial topics and smaller scale evidence synthesis endeavours
 Stakeholder can influence topic selection and scope It was challenging to balance stakeholder needs with other project requirements. Many such requirements were contractual (timeline and funding) and therefore given priority Ultimately decisions about modifications of review questions and process need to stay with and be clearly accountable to the review team(s). In turn the review team(s) should make transparent the reasons for their choices
 Providing draft plans and preliminary results for stakeholder review This measure was appreciated by stakeholders whereas review team members were sometimes reluctant to disclose preliminary plans or results. Participants considered it important for open discussions and trust building Measures described above are more meaningful and credible if linked to extensive transparency requirements. Ensuring transparency is time consuming so adequate time needs to be planned in advance
 Providing detailed documentation of consultations, stakeholder comments, team responses and how stakeholder comments shaped the process This measure was appreciated by stakeholders and considered important for open discussions and trust building