Transdisciplinary working to shape systematic reviews and interpret the findings: commentary
Environmental Evidence volume 6, Article number: 28 (2017)
Important policy questions tend to span a range of academic disciplines, and the relevant research is often carried out in a variety of social, economic and geographic contexts. In efforts to synthesise research to help inform decisions arising from the policy questions, systematic reviews need conceptual frameworks and ways of thinking that combine knowledge drawn from different academic traditions and contexts; in other words, transdisciplinary research. This paper considers how transdisciplinary working can be achieved with: conceptual frameworks that span traditional academic boundaries; methods for shaping review questions and conceptual frameworks; and methods for interpreting the relevance of findings to different contexts. It also discusses the practical challenges and ultimate benefits of transdisciplinary working for systematic reviews.
Policy dilemmas cross conventional academic boundaries. The academic response to the challenge of informing decision-making in such a context has been twofold: providing ready access to relevant scientific evidence with systematic reviews, or research syntheses, that include studies from different social, economic and geographic contexts, and draw on multiple academic disciplines; and building teams of academics and other stakeholders to address policy dilemmas by working in unconventional ways (see Box 1 for definitions). Indeed, most policy dilemmas raise many scientific questions across a range of disciplines . Early systematic reviews in environmental science were largely academic endeavours and in these circumstances the validity of the work can be undermined by lack of consensus about review questions, specifically the choice of outcomes and analysis of contextual variables . Since then, involving stakeholders in the production of systematic reviews has been seen as critical . In addition a few systematic reviewers have broadened their analysis to address both impact and explanations and meaning of impact , both change and reasons for change , and to develop a theory of change . These much needed methodological advances have important implications for delivery of services. In the health sector these implications are well illustrated by systematic reviews addressing the problems of patients offered an effective, but long and demanding, treatment for tuberculosis (TB). These reviews expose differences between the world of research, and the wider world that research is meant to serve (see Box 2).
Currently, the content of systematic reviews is largely evaluations of programmes, sometimes adapted by researchers in the field specifically to enable rigorous evaluation, with studies stripped of their organisational and socio-political context during the review process. Consequently the synthesised findings of these primary studies, with high internal validity, offer persuasive evidence of impact for policy decision-making. Yet, the partial picture this evidence presents largely ignores the policy context which risks evidence-informed policy decisions subsequently stalling with programmes failing to deliver better policy outcomes. This situation is illustrated in Fig. 1.
If systematic reviews are to address real world problems that are situated in complex systems, there is a need for systematic review designs that span academic disciplines; new ways of working to construct those designs; and methods to interpret the findings. This need is for transdisciplinary research methods—ways of working that cut across and beyond academic disciplines.
This paper offers some solutions to the challenge facing systematic reviews in environmental science, namely the need for a ‘balance… between a reductionist approach that simplifies the question but may limit both the quantity of information available and the applicability of its conclusions, and a holistic approach in which the question contains so much complexity that no studies have attempted to address it’ . In doing so it also draws on other sectors where systematic reviews were introduced to policy decision making earlier.
Here we offer three different transdisciplinary methods for producing systematic reviews: combining concepts from across and beyond academic disciplines in conceptual frameworks for systematic reviews; communication methods for working with people from across and beyond academic disciplines; and models for structuring findings to take into account contextual influences.
Conceptual frameworks to span boundaries
As systematic reviews are increasingly commissioned by policy organisations, rather than initiated by curious and reflective practitioners, the scope of individual questions addressed has broadened. For instance, a review investigating the impact of agricultural interventions on the nutritional status of children included studies from social science, agriculture, psychology, nutrition, economics and physiology . The review was structured by a theory of change conceptual framework with components that included participation in educational programmes and adoption of technology, leading to changes in diet from home produce or to enhanced household income and food purchases; and from this on to improved nutritional uptake and health status. The theory of change was used instead of a traditional systematic review (SR) ‘PI/ECO’ (population, intervention/exposure, comparison, outcomes) structure to define components of and drive the review. The approach made a large and complex review manageable and coherent, while accommodating the individual packets of evidence which were quite different in terms of question, research evidence, discipline and context.
In contrast, when policy questions seek to develop understanding rather than assess the measures of effects of an intervention, conceptual frameworks may be the output of a review, rather than used as the driver. For a review analysing qualitative studies about protected terrestrial areas, such as national parks and forests, and human well-being , the resulting conceptual framework combined dimensions of well-being (health, social capital, economic capital and environmental capital) and governance (regulation, enforcement, participatory management and empowerment) against a backdrop of human rights. The result was a conceptual framework to present a set of coherent findings from very disparate studies spanning economics, education, epidemiology, environmental science, anthropology, law, history, and public health.
Although use of conceptual models is hardly new, they may be underused. A recent mapping review of over 1000 studies examining the links between conservation activities and human health and wellbeing found very few well-articulated, detailed theories of change, despite the sometimes long and complex chains of possible interactions that were being researched .
Communication methods for shaping review questions and conceptual frameworks
The construction of review questions and use of conceptual frameworks in systematic reviews requires collaborative teams that span academic and social systems and that think critically and creatively together by managing conflict well [3, 14]. Although there is widespread support for involving stakeholders when conducting systematic reviews , current guidance is directed more towards who to engage than how to work with them creatively to shape the review. Insights about such social interactions emerged from insider research [16, 17] and reflective practice addressing the early stages of the systematic review process when refining questions and framing reviews addressing broad issues . From this insider research and reflective practice, we now recognise the parallels between shaping reviews and two other forms of creative thinking processes: qualitative analysis and non-directive counselling . While the former examines observations for patterns and meaning to make sense of data, the latter refrains from interpretation or explanation but encourages others to talk freely and discover patterns and meaning themselves to make sense of their own experience. Originally developed to help individuals address personal problems , its core element of active (or reflective) listening has been subsequently developed and applied to support creative problem solving by groups . The non-directive counselling approach has been helpful in supporting interdisciplinary review teams (inclusive of stakeholders) to solve the problem of shaping a conceptual framework for their review that will accommodate the interests of the review funder and the framings of existing relevant studies . As a stepwise process for qualitative analysis and non-directive counselling has been clarified, shared and incorporated into text books and training programmes (Box 3), we see an opportunity to clarify and practice their application for shaping systematic reviews.
However, the active listening that is at the heart of non-directive counselling brings risks. Systematic reviewers working closely with stakeholders who are bringing direct experience and strong interests risk losing their critical distance. Moreover, examining, comparing and reconciling the ideas, opinions and perspectives of different stakeholders through mutual challenge and constructive conflict  may be particularly difficult to attain when there is an imbalance in power or money, as in commissioned systematic reviews.
Models for structuring findings to take into account contextual influences
Considering the needs of multiple stakeholders is not only for the beginning of a review: there are also opportunities towards the end when interpreting emerging findings. Typically users of systematic reviews want to know how relevant the findings are to their own situation, or the populations for which they make decisions. The principle of globalising the evidence, but localising the decision  can be helped by careful description of the characteristics of the included studies, or carefully delineating the factors that might be important in contextualising the evidence, and then making sure this is systematically extracted and summarised. For example, subgroups may be distinguished by their place of residence, religion, occupation, gender, Race/ethnicity, education, socioeconomic status, and social networks and capital . This approach, with its mnemonic PROGRESS, for capturing social determinants of health, has been integrated into guidance for pre-specifying subgroup analyses in systematic reviews [28, 29]. The method is well suited to public health because it provides a framework for epidemiological analyses.
However, the PROGRESS determinants of health ignore the inner layers of individual risk factors (such as genetics, physical impairment or lifestyle factors) that feature in biology and behavioural science. They also ignore the outer layers of ecological or geological factors central to environmental science. No mention is made of intersectional theory of sociology about social identities overlapping or intersecting , perhaps because multiplying subgroup analyses reduces statistical power in epidemiology . Lastly, PROGRESS ignores any dynamics arising from: interactions between the multiple layers; the life course (age); life transitions (moving home, employment, school or leaving prison, hospital or a significant relationship); historical changes (conflicts, mass migrations (post)colonialism); or geological or climate changes (natural disasters).
A more flexible approach to investigating contextual influences or inequalities may be found in the work of Bronfenbrenner [32, 33] who conceptualised children’s lives as being shaped by environmental factors acting and interacting in a set of nested structures, from within families (at the micro level) to within their historical context (at the macro level). This has been applied to systematic reviews of research  and policy  addressing children’s rights in post-conflict areas. The potential for applying frameworks such as Bronfenbrenner’s to different systematic reviews is suggested by the various adaptations of similar ecological frameworks that can be found for primary research elsewhere, such as: environmental science ; migration studies ; and violence . We illustrate that potential in Fig. 2 by visually summarising the findings of a systematic review of qualitative studies of microfinance .
Ecological models not only offer a framework to make sense of review findings but, as they provide a way to navigate the complexity of people’s life circumstances, they also provide a framework for identifying stakeholders who can help with shaping the review or interpreting the findings. An ecological framework can be immensely beneficial when researching context-sensitive topic areas such as children, gender and the broader social, cultural and natural environments.
Practical challenges and ultimate benefits
Transdisciplinary working when conducting systematic reviews is not easy. The challenges manifest when working with contrasting paradigms, and epistemological, ontological and methodological differences. Our own experience tells us it requires time and effort to adapt to unfamiliar information resources, terminology, communication styles and research methods. Guidance is available from a systematic review which found that transdisciplinary research is enhanced by team leaders with good ideas and vision, contacts, good interpersonal skills, humility, familiarity with the disciplines and the opportunity to choose their team members and keep them all on board, and by team members with maturity, flexibility and personal commitment . Grounding the unfamiliar in social and cultural contexts recognizable to the particular review team can encourage respect for different ideologies and paradigms, and a better understanding and appreciation of disciplinary diversity. Transdisciplinary research is also helped by the physical proximity of team members, the internet and email as a supporting platform, and an institutionally conducive environment. Constructive working practices include: developing a common goal and shared vision; having clarity about, and rotation of, roles; good communication and constructive comments among team members, and importantly, a collaborative ethos of openness and sharing in learning with and from distinct disciplines.
Ideally such teams synthesise more complete evidence, more coherently, and align reviews more closely with stakeholder interests, leading to more compelling evidence. For these reasons, commissioned systematic reviews, which tend to be both complex and time-pressured, require that care be taken not only in drafting substantive content of terms of reference for the conduct of the systematic review, but also in selecting a team of reviewers well motivated to take on transdisciplinary reviews. A track record in project management, a typical requirement in requests for proposals, does little to reveal the capacity of the leader for the critical tasks of forming a team, holding it together, and resolving different points of view. Further, transdisciplinary reviews attract different stakeholders who may be driven by disparate motivations. Generally, academics tend to be comfortable ‘producing knowledge’, partly because they are rewarded by the academic structures in which they are situated for doing so. Non-academics, on the other hand, are rewarded for ‘getting things done’ and seeking practical results and impacts, which may lead to different approaches and motivations in larger and more diverse teams. Once again, the ability of a team leader to manage any resulting tension in teams with academic and non-academic members, is critical to the successful outcome of the review. Indeed, producing knowledge combined with getting things done underpin good transdisciplinary research, which is commonly assessed in terms of relevance, credibility, legitimacy and effectiveness in problem solving or social change .
Despite these challenges, transdisciplinary working, with academics and other stakeholders, has led to growing numbers of systematic reviews that address policy questions. Transdisciplinary working has also made possible the adaptation of review methods for new fields and the sharing of knowledge between experienced reviewers and novice teams who bring subject expertise to build reviewing capacity and produce learning which is empowering and reflects both the local and global.
Systematic methods for answering important questions from existing literature are well developed. These methods need to be complemented by clearer methods that emphasise the thinking and debate for developing the questions, shaping reviews and interpreting emerging findings. Such work requires crossing academic and policy boundaries, and exploring how concepts, definitions and language differ. Communication methods analogous to collective qualitative analysis or non-directive counselling look promising for refining questions and constructing conceptual frameworks collectively. Ecological models look promising for understanding the context of research findings and addressing the big questions about social change.
Whitty C. What makes an academic paper useful for health policy? BMC Med. 2015;13:301. doi:10.1186/s12916-015-0544-8.
Stewart G, Coles CF, Pullin AS. Applying evidence-based practice in conservation management: lessons from the first systematic review and dissemination projects. Biol Cons. 2005;126(2005):270–8.
Haddaway NR, Kohl C, Rebelo da Silva N, Schiemann J, Spök A, Stewart R, Sweet JB, Wilhelm R (2017) A framework for stakeholder engagement during systematic reviews and maps in environmental management. Environ Evid 2017;6:11. doi:10.1186/s13750-017-0089-8
Pullin AS, Bangpan M, Dalrymple S, Dickson K, Haddaway N, Healey J, Hauari H, Hockley N, Jones J, Knight T, Vigurs C, Oliver S. 2013. Human well-being impacts of terrestrial protected areas? Environ Evid. 2013;2:19. doi:10.1186/2047-2382-2-19. http://www.environmentalevidencejournal.org/content/2/1/19.
Leisher C, Temsah G, Booker F, Day M, Samberg L, Prosnitz D, Agarwal B, Matthews E, Roe D, Russell D, Sunderland T, Wilkie D. Does the gender composition of forest and fishery management groups affect resource governance and conservation outcomes? A systematic map. Environ Evid. 2016;5:6. doi:10.1186/s13750-016-0057-8.
Althor G, McKinnon M, Cheng SH, Klein C, Watson J. Does the social equitability of community and incentive based conservation interventions in non-OECD countries, affect human well-being? A systematic review protocol. Environ Evid. 2016;5:26.
Gough D, Oliver S, Thomas J. Introduction to systematic reviews, 2nd Edition. London: Sage; 2017.
Choi BC, Pak AW. Multidisciplinarity, interdisciplinarity and transdisciplinarity in health research, services, education and policy: 1. Definitions, objectives, and evidence of effectiveness. Clin Invest Med. 2006;29(6):351–64.
Karumbi J, Garner P. Directly observed therapy for treating tuberculosis. Cochrane Database Syst Rev 2015;CD003343(5). doi:10.1002/14651858.CD003343.pub4.
Lutge EE, Wiysonge CS, Knight SE, Volmink J, Sinclair D. Incentives and enablers to improve adherence in tuberculosis. Cochrane Database Syst Rev 2015;CD007952(7). doi:10.1002/14651858.CD007952.pub2.
Liu Q, Abba K, Alejandria MM, Sinclair D, Balanag VM, Lansang MAD. Reminder systems to improve patient adherence to tuberculosis clinic appointments for diagnosis and treatment. Cochrane Database Syst Rev 2014;CD006594(11). doi:10.1002/14651858.CD006594.pub3.
Masset E, Haddad L, Cornelius A, Isaza-Castro J. A systematic review of agricultural interventions that aim to improve nutritional status of children. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London; 2011.
McKinnon MC, Cheng SH, Garside R, Masuda YJ, Miller DC, et al. What are the effects of nature conservation on human well-being? A systematic map of empirical evidence from developing countries. Environ Evid. 2016;5(8). doi:10.1186/s13750-016-0058-7.
Larsen RK, Nilsson AE. Knowledge production and environmental conflict: managing systematic reviews and maps for constructive outcomes. Environ Evid. 2017;6:17. doi:10.1186/s13750-017-0095-x.
Cottrell E, Whitlock E, Kato E, Uhl S, Belinson S, Chang C, Hoomans T, Meltzer D, Noorani H, Robinson K, Schoelles K, Motu’apuaka M, Anderson J, Paynter R, Guise JM. Defining the benefits of stakeholder engagement in systematic reviews. Research white paper (prepared by the scientific resource center under Contract No. 290-2012-00004-C.) AHRQ Publication No. 14-EHC006-EF. Rockville: Agency for Healthcare Research and Quality. http://www.effectivehealthcare.ahrq.gov/reports/final.cfm.
Edwards B. Deep insider research. Qual Res J. 2002;2(1):71–84.
Robson C. Real world research: a resource for social scientists and practitioner-researchers. Massachussetts: Blackwell Publishers; 2002.
Oliver S, Bangpan M, Dickson K. Producing policy relevant systematic reviews: navigating the policy-research interface. Evid Policy. 2016;12(2):235–59. http://www.ingentaconnect.com/content/tpp/ep/pre-prints/content-ppevidpol1600048r2.
Rogers CR. Counseling and psychotherapy. Boston: Houghton Mifflin; 1942.
Strauss A, Corbin J. Basics of qualitative research: techniques and procedures for developing grounded theory. Sage publications, London; 1996.
Noblit G, Dwight Hare R. Meta-ethnography: synthesising qualitative studies. In: Egan G, editor. The skilled helper: a systematic approach to effective helping. California: Sage Publications; 1988.
Egan G. The skilled helper: a systematic approach to effective helping. Pacific Grove, CA, USA: Brooks/ Cole Publishing Company; 1990.
Isaksen SG. Toward a model for the facilitation of creative problem solving. J Creat Behav. 1983;17:18–31. doi:10.1002/j.2162-6057.1983.tb00971.x.
Rogers J. Coaching skills. Milton Keynes: Open University Press; 2004.
Amason AC, Hochwarter WA, Thompson KR, Harrison AW. Conflict: an important dimension in successful management teams. Organ Dyn. 1995;24:20–35.
Eisenberg JM. Globalize the evidence, localize the decision: evidence-based medicine and international diversity. Health Aff. 2002;21(3):1–17.
Evans T, Brown H. Road traffic crashes: operationalizing equity in the context of health sector reform. Int J Inj Contr Saf Promot. 2003;10(1–2):11–2.
O’Neill J, Tabish H, Welch V, Petticrew M, Pottie K, Clarke M. Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to illuminate inequities in health. J Clin Epidemiol. 2014;67:56–64.
Welch V, Petticrew M, Tugwell P, Moher D, O’Neill J, Waters E, et al. PRISMA-equity 2012 extension: reporting guidelines for systematic reviews with a focus on health equity. PLoS Med. 2012;9(10):e1001333. doi:10.1371/journal.pmed.100133.
Collins PH. Intersectionality’s definitional dilemmas. Ann Rev Sociol. 2015;41:1–20.
Burke JF, Sussman JB, Kent DM, Hayward RA. Three simple rules to ensure reasonably credible subgroup analyses. BMJ. 2015;351:h5651.
Bronfenbrenner U. The ecology of human development. Cambridge: Harvard University Press; 1979.
Bronfenbrenner U. Examining lives in context: perspectives on the ecology of human development. In: Moen P, Elder Jr GH, editors. Developmental ecology through space and time. A future perspective. Washington: American Psychological Association; 1995. p. 619–47.
Ang L. Early childhood and peace building in the Asia-Pacific Region: a literature review to inform a regional research agenda. New York: UNICEF Publication; 2014. https://www.arnec.net/wp-content/uploads/2015/04/Peace-Bldg-Report-ref.pdf. Accessed 1 April 2017.
Ang L, Oliver S. A systematic policy review of early childhood development and peace building in fourteen conflict-affected and post-conflict countries. London: UNICEF and UCL Institute of Education: University College London; 2015. Accessed 1 April 2017.
Coutts C, Hahn M. Green infrastructure, ecosystem services, and human. Health Int J Environ Res Public Health. 2015;12(8):9768–98. doi:10.3390/ijerph120809768.
Nkulu Kalengayi FK, Hurtig AK, Ahlm C, Ahlberg BM. “It is a challenge to do it the right way”: an interpretive description of caregivers’ experiences in caring for migrant patients in northern Sweden. BMC Health Serv Res. 2012;12(433). doi:10.1186/1472-6963-12-433.
Krug EG, Dahlberg LL, Mercy JA, Zwi AB, Lozano R, editors. World report on violence and health. Geneva: World Health Organisation; 2002.
Peters MDJ, Lockwood C, Munn Z, Moola S, Mishra RK. People’s views and experiences of participating in microfinance interventions: A systematic review of qualitative evidence. London: EPPI-Centre, Social Science Research Unit, UCL Institute of Education, University College London; 2016.
Choi BC, Pak AW. Multidisciplinarity, interdisciplinarity, and transdisciplinarity in health research, services, education and policy: 2. Promotors, barriers, and strategies of enhancement. Clin Invest Med. 2007;30(6):E224–32.
Belcher BM, Rasussen KE, Keshaw MR, Zornes DA. Defining and assessing research quality in a transdisciplinary context. Res Eval. 2016;25:1–17.
This paper combines six strands of thinking. PG, PH and SO analysed the limitations of practitioner focused systematic reviews within multilevel, adaptive systems. JJ and SO analysed current frameworks for equity analysis. KD, MB and SO conducted the research addressing how systematic reviewers and their stakeholder partners navigate the policy–research interface. MF and SO analysed the intellectual steps in shaping evidence gaps. LA and SO developed the multi-level ecological model that informed the early childhood development and policy-related systematic review. SO and RG applied these arguments about transdisciplinary research to environmental science. All authors read and approved the final manuscript.
We are grateful to the anonymous peer reviewers for their thoughtful comments and constructive suggestions on an earlier version of the manuscript.
The authors declare that they have no competing interests.
Availability of data and materials
This is a commentary on prior research which is cited in full.
Consent for publication
Ethics approval and consent to participate
This is a commentary on prior research published elsewhere. The ethics approval and consent procedures are reported alongside any prior research with human participants.
Much of the research underpinning this commentary (conducted by SO, KD and MB) was funded by the World Health Organization through the Alliance for Health Policy and Systems Research (research Grant No. 2013/323807-0; PO-No. 200770388). Authors SO, KD and MB have also drawn on their work supported by the UK Department for International Development (Grants: 6617, 7135). PG and PH are supported by UK aid from the UK Government for the benefit of developing countries (Grant: 5242). SO was (in part) supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) North Thames at Bart’s Health NHS Trust. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Oliver, S., Garner, P., Heywood, P. et al. Transdisciplinary working to shape systematic reviews and interpret the findings: commentary. Environ Evid 6, 28 (2017). https://doi.org/10.1186/s13750-017-0106-y