Table 3 Overview of experiences and recommendations for evidence synthesis in controversial contexts

Openness and inclusiveness

 Broad stakeholder invitation—any interested stakeholder can participate

The absence of participant selection prevented conflicts about the composition of the stakeholder group and contributed to trust building

This approach needs supporting measures to ensure that interested stakeholder representatives can participate in all subsequent steps (e.g. travel support, smaller stakeholder groups)

 Multiple ways of engagement (workshops, written consultations)

Online written consultations were mainly used to introduce issues not mentioned in or to clarify or emphasize discussion topics of the face-to-face consultations

This set-up is useful for clarifying issues and views and for bringing up issues not addressed during face-to-face meetings, e.g. due to lack of time. It does, however, not help in engaging individuals unable to participate in the face-to-face consultations

 Familiarizing participants with the evidence synthesis process

This measure is very important for a meaningful engagement. This works better if the same representatives of a stakeholder organisation participate in the entire consultation process

The engagement process must be tailored to the level of knowledge of and experience with evidence synthesis

Measures need to be in place to ensure that the same individuals can contribute over the entire engagement process (e.g. providing travel support and opting for smaller stakeholder groups)

Responsiveness

 Procedure for written responses to stakeholder comments

The measure was extensively used by stakeholders

The procedure resulted in a systematic and transparent consideration of stakeholder comments by review teams and allowed for transparency of how stakeholder inputs shaped the process

Time and resource needs are significant and therefore preclude routine application. The procedure appears suitable for sensitive or controversial topics and smaller scale evidence synthesis endeavours

 Stakeholder can influence topic selection and scope

It was challenging to balance stakeholder needs with other project requirements. Many such requirements were contractual (timeline and funding) and therefore given priority

Ultimately decisions about modifications of review questions and process need to stay with and be clearly accountable to the review team(s). In turn the review team(s) should make transparent the reasons for their choices

Transparency

 Providing draft plans and preliminary results for stakeholder review

This measure was appreciated by stakeholders whereas review team members were sometimes reluctant to disclose preliminary plans or results. Participants considered it important for open discussions and trust building

Measures described above are more meaningful and credible if linked to extensive transparency requirements. Ensuring transparency is time consuming so adequate time needs to be planned in advance

 Providing detailed documentation of consultations, stakeholder comments, team responses and how stakeholder comments shaped the process

This measure was appreciated by stakeholders and considered important for open discussions and trust building