Transdisciplinary working to shape systematic reviews and interpret the findings: commentary
© The Author(s) 2017
Received: 1 April 2017
Accepted: 10 October 2017
Published: 17 November 2017
Important policy questions tend to span a range of academic disciplines, and the relevant research is often carried out in a variety of social, economic and geographic contexts. In efforts to synthesise research to help inform decisions arising from the policy questions, systematic reviews need conceptual frameworks and ways of thinking that combine knowledge drawn from different academic traditions and contexts; in other words, transdisciplinary research. This paper considers how transdisciplinary working can be achieved with: conceptual frameworks that span traditional academic boundaries; methods for shaping review questions and conceptual frameworks; and methods for interpreting the relevance of findings to different contexts. It also discusses the practical challenges and ultimate benefits of transdisciplinary working for systematic reviews.
Definitions of key terms that describe the process and products of systematically reviewing policy-relevant research
Systematic reviews of research inspect research reports using explicit, accountable and rigorous research methods 
Research synthesis aims to integrate the findings of different studies to answer the review question leading to knowledge that is greater than the sum of the individual studies 
Policy relevant Systematic reviews can be considered relevant to policy (and policy makers) when they present findings clearly for policy audiences to: illuminate policy problems; challenge or develop policy assumptions; or offer evidence about the impact or implementation of policy options; and take into account diversity of people and contexts 
Transdisciplinary research integrates the natural, social and health sciences in a humanities context, and in so doing transcends each of their traditional boundaries. It does so by scientists and other stakeholders working together beyond their traditional roles to transcend traditional boundaries to investigate systems in a holistic way 
Stakeholders in systematic reviews include any person, organisation or social group that may influence or be influenced by the process of preparing or using systematic reviews or by the decisions informed by their findings
The mismatch between the worlds of research and implementation: an example from health
The proposed solution of ‘directly observed therapy’ (DOT), a practice that involves healthcare practitioners observing patients taking their treatment, is not well supported by systematic review evidence regarding distinct approaches to implementation of directly observed therapy , including incentives and enablers , or reminders . Whilst these reviews, drawing on randomised controlled trials, provided some useful inputs to specific technical recommendations being made by the World Health Organization at the time, in broader policy terms they offer disappointing findings to national policy makers frustrated by the “real world” where: conflicts disrupting health systems; practitioners favouring patients they considered most deprived and therefore most deserving; and patients finding the timing of the treatment and incentive (a midday meal) inconvenient . Moreover, many of these reviews considered DOT without a comparator, and reviewed individual interventions alone, rather than typical packages of interventions, which is insufficient . This example highlights the importance and need to consider the ‘financial and logistical barriers to care; approaches that motivate patients and staff; and defaulter follow-up’ ; in programmes of care and the systematic reviews that inform them.
If systematic reviews are to address real world problems that are situated in complex systems, there is a need for systematic review designs that span academic disciplines; new ways of working to construct those designs; and methods to interpret the findings. This need is for transdisciplinary research methods—ways of working that cut across and beyond academic disciplines.
This paper offers some solutions to the challenge facing systematic reviews in environmental science, namely the need for a ‘balance… between a reductionist approach that simplifies the question but may limit both the quantity of information available and the applicability of its conclusions, and a holistic approach in which the question contains so much complexity that no studies have attempted to address it’ . In doing so it also draws on other sectors where systematic reviews were introduced to policy decision making earlier.
Here we offer three different transdisciplinary methods for producing systematic reviews: combining concepts from across and beyond academic disciplines in conceptual frameworks for systematic reviews; communication methods for working with people from across and beyond academic disciplines; and models for structuring findings to take into account contextual influences.
Conceptual frameworks to span boundaries
As systematic reviews are increasingly commissioned by policy organisations, rather than initiated by curious and reflective practitioners, the scope of individual questions addressed has broadened. For instance, a review investigating the impact of agricultural interventions on the nutritional status of children included studies from social science, agriculture, psychology, nutrition, economics and physiology . The review was structured by a theory of change conceptual framework with components that included participation in educational programmes and adoption of technology, leading to changes in diet from home produce or to enhanced household income and food purchases; and from this on to improved nutritional uptake and health status. The theory of change was used instead of a traditional systematic review (SR) ‘PI/ECO’ (population, intervention/exposure, comparison, outcomes) structure to define components of and drive the review. The approach made a large and complex review manageable and coherent, while accommodating the individual packets of evidence which were quite different in terms of question, research evidence, discipline and context.
In contrast, when policy questions seek to develop understanding rather than assess the measures of effects of an intervention, conceptual frameworks may be the output of a review, rather than used as the driver. For a review analysing qualitative studies about protected terrestrial areas, such as national parks and forests, and human well-being , the resulting conceptual framework combined dimensions of well-being (health, social capital, economic capital and environmental capital) and governance (regulation, enforcement, participatory management and empowerment) against a backdrop of human rights. The result was a conceptual framework to present a set of coherent findings from very disparate studies spanning economics, education, epidemiology, environmental science, anthropology, law, history, and public health.
Although use of conceptual models is hardly new, they may be underused. A recent mapping review of over 1000 studies examining the links between conservation activities and human health and wellbeing found very few well-articulated, detailed theories of change, despite the sometimes long and complex chains of possible interactions that were being researched .
Communication methods for shaping review questions and conceptual frameworks
Thinking and communication processes analogous to developing a question or conceptual framework for systematic reviewing 
• That sensitise the researchers to the landscape of interest—what is going on here, who is involved, how do they define the situation, what does it mean to them, are their definitions and meanings the same or different, what are they all doing (the same or differently) and why?
• Asking open ended questions to encourage talk and reflection on specific examples
• That explore recurring themes as stakeholders talk
• Adopting the stakeholders’ own language
• About processes, variation, connections (or assumptions) about key concepts, changes over time and pertinent structural influence
• Asking future oriented questions about how stakeholders would use the evidence
• About exceptions or contradictions; and
• Provoking thinking, demanding clarification and challenging assumptions
• About where to look for evidence and how to recognise it in different contexts
• summarising responses to confirm understanding, invite correction and introduce language that links with wider understandings
• Interrupting repetition or vague assertions
• Moving the conversation on; and
• Getting to the crux of the matter and articulating the main focus
However, the active listening that is at the heart of non-directive counselling brings risks. Systematic reviewers working closely with stakeholders who are bringing direct experience and strong interests risk losing their critical distance. Moreover, examining, comparing and reconciling the ideas, opinions and perspectives of different stakeholders through mutual challenge and constructive conflict  may be particularly difficult to attain when there is an imbalance in power or money, as in commissioned systematic reviews.
Models for structuring findings to take into account contextual influences
Considering the needs of multiple stakeholders is not only for the beginning of a review: there are also opportunities towards the end when interpreting emerging findings. Typically users of systematic reviews want to know how relevant the findings are to their own situation, or the populations for which they make decisions. The principle of globalising the evidence, but localising the decision  can be helped by careful description of the characteristics of the included studies, or carefully delineating the factors that might be important in contextualising the evidence, and then making sure this is systematically extracted and summarised. For example, subgroups may be distinguished by their place of residence, religion, occupation, gender, Race/ethnicity, education, socioeconomic status, and social networks and capital . This approach, with its mnemonic PROGRESS, for capturing social determinants of health, has been integrated into guidance for pre-specifying subgroup analyses in systematic reviews [28, 29]. The method is well suited to public health because it provides a framework for epidemiological analyses.
However, the PROGRESS determinants of health ignore the inner layers of individual risk factors (such as genetics, physical impairment or lifestyle factors) that feature in biology and behavioural science. They also ignore the outer layers of ecological or geological factors central to environmental science. No mention is made of intersectional theory of sociology about social identities overlapping or intersecting , perhaps because multiplying subgroup analyses reduces statistical power in epidemiology . Lastly, PROGRESS ignores any dynamics arising from: interactions between the multiple layers; the life course (age); life transitions (moving home, employment, school or leaving prison, hospital or a significant relationship); historical changes (conflicts, mass migrations (post)colonialism); or geological or climate changes (natural disasters).
Ecological models not only offer a framework to make sense of review findings but, as they provide a way to navigate the complexity of people’s life circumstances, they also provide a framework for identifying stakeholders who can help with shaping the review or interpreting the findings. An ecological framework can be immensely beneficial when researching context-sensitive topic areas such as children, gender and the broader social, cultural and natural environments.
Practical challenges and ultimate benefits
Transdisciplinary working when conducting systematic reviews is not easy. The challenges manifest when working with contrasting paradigms, and epistemological, ontological and methodological differences. Our own experience tells us it requires time and effort to adapt to unfamiliar information resources, terminology, communication styles and research methods. Guidance is available from a systematic review which found that transdisciplinary research is enhanced by team leaders with good ideas and vision, contacts, good interpersonal skills, humility, familiarity with the disciplines and the opportunity to choose their team members and keep them all on board, and by team members with maturity, flexibility and personal commitment . Grounding the unfamiliar in social and cultural contexts recognizable to the particular review team can encourage respect for different ideologies and paradigms, and a better understanding and appreciation of disciplinary diversity. Transdisciplinary research is also helped by the physical proximity of team members, the internet and email as a supporting platform, and an institutionally conducive environment. Constructive working practices include: developing a common goal and shared vision; having clarity about, and rotation of, roles; good communication and constructive comments among team members, and importantly, a collaborative ethos of openness and sharing in learning with and from distinct disciplines.
Ideally such teams synthesise more complete evidence, more coherently, and align reviews more closely with stakeholder interests, leading to more compelling evidence. For these reasons, commissioned systematic reviews, which tend to be both complex and time-pressured, require that care be taken not only in drafting substantive content of terms of reference for the conduct of the systematic review, but also in selecting a team of reviewers well motivated to take on transdisciplinary reviews. A track record in project management, a typical requirement in requests for proposals, does little to reveal the capacity of the leader for the critical tasks of forming a team, holding it together, and resolving different points of view. Further, transdisciplinary reviews attract different stakeholders who may be driven by disparate motivations. Generally, academics tend to be comfortable ‘producing knowledge’, partly because they are rewarded by the academic structures in which they are situated for doing so. Non-academics, on the other hand, are rewarded for ‘getting things done’ and seeking practical results and impacts, which may lead to different approaches and motivations in larger and more diverse teams. Once again, the ability of a team leader to manage any resulting tension in teams with academic and non-academic members, is critical to the successful outcome of the review. Indeed, producing knowledge combined with getting things done underpin good transdisciplinary research, which is commonly assessed in terms of relevance, credibility, legitimacy and effectiveness in problem solving or social change .
Despite these challenges, transdisciplinary working, with academics and other stakeholders, has led to growing numbers of systematic reviews that address policy questions. Transdisciplinary working has also made possible the adaptation of review methods for new fields and the sharing of knowledge between experienced reviewers and novice teams who bring subject expertise to build reviewing capacity and produce learning which is empowering and reflects both the local and global.
Systematic methods for answering important questions from existing literature are well developed. These methods need to be complemented by clearer methods that emphasise the thinking and debate for developing the questions, shaping reviews and interpreting emerging findings. Such work requires crossing academic and policy boundaries, and exploring how concepts, definitions and language differ. Communication methods analogous to collective qualitative analysis or non-directive counselling look promising for refining questions and constructing conceptual frameworks collectively. Ecological models look promising for understanding the context of research findings and addressing the big questions about social change.
This paper combines six strands of thinking. PG, PH and SO analysed the limitations of practitioner focused systematic reviews within multilevel, adaptive systems. JJ and SO analysed current frameworks for equity analysis. KD, MB and SO conducted the research addressing how systematic reviewers and their stakeholder partners navigate the policy–research interface. MF and SO analysed the intellectual steps in shaping evidence gaps. LA and SO developed the multi-level ecological model that informed the early childhood development and policy-related systematic review. SO and RG applied these arguments about transdisciplinary research to environmental science. All authors read and approved the final manuscript.
We are grateful to the anonymous peer reviewers for their thoughtful comments and constructive suggestions on an earlier version of the manuscript.
The authors declare that they have no competing interests.
Availability of data and materials
This is a commentary on prior research which is cited in full.
Consent for publication
Ethics approval and consent to participate
This is a commentary on prior research published elsewhere. The ethics approval and consent procedures are reported alongside any prior research with human participants.
Much of the research underpinning this commentary (conducted by SO, KD and MB) was funded by the World Health Organization through the Alliance for Health Policy and Systems Research (research Grant No. 2013/323807-0; PO-No. 200770388). Authors SO, KD and MB have also drawn on their work supported by the UK Department for International Development (Grants: 6617, 7135). PG and PH are supported by UK aid from the UK Government for the benefit of developing countries (Grant: 5242). SO was (in part) supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) North Thames at Bart’s Health NHS Trust. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Whitty C. What makes an academic paper useful for health policy? BMC Med. 2015;13:301. doi:10.1186/s12916-015-0544-8.View ArticleGoogle Scholar
- Stewart G, Coles CF, Pullin AS. Applying evidence-based practice in conservation management: lessons from the first systematic review and dissemination projects. Biol Cons. 2005;126(2005):270–8.View ArticleGoogle Scholar
- Haddaway NR, Kohl C, Rebelo da Silva N, Schiemann J, Spök A, Stewart R, Sweet JB, Wilhelm R (2017) A framework for stakeholder engagement during systematic reviews and maps in environmental management. Environ Evid 2017;6:11. doi:10.1186/s13750-017-0089-8 View ArticleGoogle Scholar
- Pullin AS, Bangpan M, Dalrymple S, Dickson K, Haddaway N, Healey J, Hauari H, Hockley N, Jones J, Knight T, Vigurs C, Oliver S. 2013. Human well-being impacts of terrestrial protected areas? Environ Evid. 2013;2:19. doi:10.1186/2047-2382-2-19. http://www.environmentalevidencejournal.org/content/2/1/19.
- Leisher C, Temsah G, Booker F, Day M, Samberg L, Prosnitz D, Agarwal B, Matthews E, Roe D, Russell D, Sunderland T, Wilkie D. Does the gender composition of forest and fishery management groups affect resource governance and conservation outcomes? A systematic map. Environ Evid. 2016;5:6. doi:10.1186/s13750-016-0057-8.View ArticleGoogle Scholar
- Althor G, McKinnon M, Cheng SH, Klein C, Watson J. Does the social equitability of community and incentive based conservation interventions in non-OECD countries, affect human well-being? A systematic review protocol. Environ Evid. 2016;5:26.View ArticleGoogle Scholar
- Gough D, Oliver S, Thomas J. Introduction to systematic reviews, 2nd Edition. London: Sage; 2017.Google Scholar
- Choi BC, Pak AW. Multidisciplinarity, interdisciplinarity and transdisciplinarity in health research, services, education and policy: 1. Definitions, objectives, and evidence of effectiveness. Clin Invest Med. 2006;29(6):351–64.Google Scholar
- Karumbi J, Garner P. Directly observed therapy for treating tuberculosis. Cochrane Database Syst Rev 2015;CD003343(5). doi:10.1002/14651858.CD003343.pub4.
- Lutge EE, Wiysonge CS, Knight SE, Volmink J, Sinclair D. Incentives and enablers to improve adherence in tuberculosis. Cochrane Database Syst Rev 2015;CD007952(7). doi:10.1002/14651858.CD007952.pub2.
- Liu Q, Abba K, Alejandria MM, Sinclair D, Balanag VM, Lansang MAD. Reminder systems to improve patient adherence to tuberculosis clinic appointments for diagnosis and treatment. Cochrane Database Syst Rev 2014;CD006594(11). doi:10.1002/14651858.CD006594.pub3.
- Masset E, Haddad L, Cornelius A, Isaza-Castro J. A systematic review of agricultural interventions that aim to improve nutritional status of children. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London; 2011.Google Scholar
- McKinnon MC, Cheng SH, Garside R, Masuda YJ, Miller DC, et al. What are the effects of nature conservation on human well-being? A systematic map of empirical evidence from developing countries. Environ Evid. 2016;5(8). doi:10.1186/s13750-016-0058-7.
- Larsen RK, Nilsson AE. Knowledge production and environmental conflict: managing systematic reviews and maps for constructive outcomes. Environ Evid. 2017;6:17. doi:10.1186/s13750-017-0095-x.View ArticleGoogle Scholar
- Cottrell E, Whitlock E, Kato E, Uhl S, Belinson S, Chang C, Hoomans T, Meltzer D, Noorani H, Robinson K, Schoelles K, Motu’apuaka M, Anderson J, Paynter R, Guise JM. Defining the benefits of stakeholder engagement in systematic reviews. Research white paper (prepared by the scientific resource center under Contract No. 290-2012-00004-C.) AHRQ Publication No. 14-EHC006-EF. Rockville: Agency for Healthcare Research and Quality. http://www.effectivehealthcare.ahrq.gov/reports/final.cfm.
- Edwards B. Deep insider research. Qual Res J. 2002;2(1):71–84.Google Scholar
- Robson C. Real world research: a resource for social scientists and practitioner-researchers. Massachussetts: Blackwell Publishers; 2002.Google Scholar
- Oliver S, Bangpan M, Dickson K. Producing policy relevant systematic reviews: navigating the policy-research interface. Evid Policy. 2016;12(2):235–59. http://www.ingentaconnect.com/content/tpp/ep/pre-prints/content-ppevidpol1600048r2.
- Rogers CR. Counseling and psychotherapy. Boston: Houghton Mifflin; 1942.Google Scholar
- Strauss A, Corbin J. Basics of qualitative research: techniques and procedures for developing grounded theory. Sage publications, London; 1996.Google Scholar
- Noblit G, Dwight Hare R. Meta-ethnography: synthesising qualitative studies. In: Egan G, editor. The skilled helper: a systematic approach to effective helping. California: Sage Publications; 1988.Google Scholar
- Egan G. The skilled helper: a systematic approach to effective helping. Pacific Grove, CA, USA: Brooks/ Cole Publishing Company; 1990.Google Scholar
- Isaksen SG. Toward a model for the facilitation of creative problem solving. J Creat Behav. 1983;17:18–31. doi:10.1002/j.2162-6057.1983.tb00971.x.View ArticleGoogle Scholar
- Rogers J. Coaching skills. Milton Keynes: Open University Press; 2004.Google Scholar
- Amason AC, Hochwarter WA, Thompson KR, Harrison AW. Conflict: an important dimension in successful management teams. Organ Dyn. 1995;24:20–35.View ArticleGoogle Scholar
- Eisenberg JM. Globalize the evidence, localize the decision: evidence-based medicine and international diversity. Health Aff. 2002;21(3):1–17.View ArticleGoogle Scholar
- Evans T, Brown H. Road traffic crashes: operationalizing equity in the context of health sector reform. Int J Inj Contr Saf Promot. 2003;10(1–2):11–2.View ArticleGoogle Scholar
- O’Neill J, Tabish H, Welch V, Petticrew M, Pottie K, Clarke M. Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to illuminate inequities in health. J Clin Epidemiol. 2014;67:56–64.View ArticleGoogle Scholar
- Welch V, Petticrew M, Tugwell P, Moher D, O’Neill J, Waters E, et al. PRISMA-equity 2012 extension: reporting guidelines for systematic reviews with a focus on health equity. PLoS Med. 2012;9(10):e1001333. doi:10.1371/journal.pmed.100133.View ArticleGoogle Scholar
- Collins PH. Intersectionality’s definitional dilemmas. Ann Rev Sociol. 2015;41:1–20.View ArticleGoogle Scholar
- Burke JF, Sussman JB, Kent DM, Hayward RA. Three simple rules to ensure reasonably credible subgroup analyses. BMJ. 2015;351:h5651.View ArticleGoogle Scholar
- Bronfenbrenner U. The ecology of human development. Cambridge: Harvard University Press; 1979.Google Scholar
- Bronfenbrenner U. Examining lives in context: perspectives on the ecology of human development. In: Moen P, Elder Jr GH, editors. Developmental ecology through space and time. A future perspective. Washington: American Psychological Association; 1995. p. 619–47.Google Scholar
- Ang L. Early childhood and peace building in the Asia-Pacific Region: a literature review to inform a regional research agenda. New York: UNICEF Publication; 2014. https://www.arnec.net/wp-content/uploads/2015/04/Peace-Bldg-Report-ref.pdf. Accessed 1 April 2017.
- Ang L, Oliver S. A systematic policy review of early childhood development and peace building in fourteen conflict-affected and post-conflict countries. London: UNICEF and UCL Institute of Education: University College London; 2015. Accessed 1 April 2017.Google Scholar
- Coutts C, Hahn M. Green infrastructure, ecosystem services, and human. Health Int J Environ Res Public Health. 2015;12(8):9768–98. doi:10.3390/ijerph120809768.View ArticleGoogle Scholar
- Nkulu Kalengayi FK, Hurtig AK, Ahlm C, Ahlberg BM. “It is a challenge to do it the right way”: an interpretive description of caregivers’ experiences in caring for migrant patients in northern Sweden. BMC Health Serv Res. 2012;12(433). doi:10.1186/1472-6963-12-433.
- Krug EG, Dahlberg LL, Mercy JA, Zwi AB, Lozano R, editors. World report on violence and health. Geneva: World Health Organisation; 2002.Google Scholar
- Peters MDJ, Lockwood C, Munn Z, Moola S, Mishra RK. People’s views and experiences of participating in microfinance interventions: A systematic review of qualitative evidence. London: EPPI-Centre, Social Science Research Unit, UCL Institute of Education, University College London; 2016.Google Scholar
- Choi BC, Pak AW. Multidisciplinarity, interdisciplinarity, and transdisciplinarity in health research, services, education and policy: 2. Promotors, barriers, and strategies of enhancement. Clin Invest Med. 2007;30(6):E224–32.View ArticleGoogle Scholar
- Belcher BM, Rasussen KE, Keshaw MR, Zornes DA. Defining and assessing research quality in a transdisciplinary context. Res Eval. 2016;25:1–17.View ArticleGoogle Scholar