A methodology for evaluating transdisciplinary research on coupled socio-ecological systems

Socio-ecological research, as conducted within the Long Term Ecological Research network in Europe (eLTER), is a relatively young field that studies coupled ecological and social systems to advance solutions for contemporary challenges in human-nature interactions. While many research and applied projects have been launched using a socio-ecological conceptual framework, there are few tested protocols to evaluate the effectiveness of such efforts at meeting their goals, e.g., goals relating to knowledge integration and influence on policy and practice, which distinguish this type of research. We suggest that such socio-ecological research may be conceptualized as an expression of the broader trend in science favoring transdisciplinarity, an approach that calls for research that fuses knowledge and methods from academia, practice, and broader society, with the goal of addressing shared public problems. We conducted a literature review of definitions of transdisciplinarity, and used these definitions to distill the core characteristics of transdisciplinary research. From these characteristics, we developed a list of guiding questions for conducting a second literature review, this time to select evaluation frameworks deemed suitable for assessing transdisciplinary research whose content was socio-ecological in nature. The resulting evaluative approaches were categorized into five groups: questionnaire models; mixed methods; staged environmental policymaking process review; the Research Embedment and Performance Profile approach; and case studies. Selected elements from these approaches were compiled and synthesized to create a six-stage framework for the assessment of interdisciplinary and transdisciplinary socio-ecological research projects and programs. The framework begins with qualitative analysis, followed by: quantitative analysis; data synthesis and visualization; the use of focus groups to reflect on interim conclusions, and, culmination with a final data synthesis and conclusions customized to the intended audience(s) of the evaluation. We provide an example of testing the first two stages of this framework using two Romanian Long-Term Socio-Ecological Research (LTSER) platforms.