Using collaborative approaches with a multi-method, multi-site, multi-target intervention: Evaluating the National Research Mentoring Network

Background and purpose: The NIH-funded National Research Mentoring Network (NRMN) aims to increase the representation and success of underrepresented groups (URGs) in biomedical research by enhancing the training and career development of individuals from diverse backgrounds, communities, and cultures. The national scope of NRMN, its wide array of innovative programs in mentor and mentee matching and training across the career spectrum, requires a collaborative evaluation strategy that addresses both internal and external evaluation needs. Due to the variety of programs implemented for each target group, the NRMN program is responsible for its own process and short-term outcome evaluations and the national Coordination and Evaluation Center (CEC) is responsible for assessing the medium and long-term effectiveness of the implemented strategies and program sustainability. Using a collaborative, utilization-focused evaluation framework, both internal NRMN evaluators and the CEC are working to translate findings into information that can be used to make both short term and long-term decisions about the efficacy and reach of the NRMN model. This important information can then inform efforts to institutionalize the current programs and potentially replicate them elsewhere. Program and key highlights: The overall evaluation of NRMN is guided by both outcome and process questions that are tailored for each target group. The different target groups include faculty and others who serve as mentors, mentees across academic training and career stages, and researchers without a history of independently funded research. NRMN is also building the capacity for training those pursuing biomedical careers by developing "master trainers" for both mentoring and grantsmanship programs in organizations and institutions that can support expanded training efforts aimed at diversifying the biomedical workforce. Implications: Results of this evaluation will be used to inform the design and implementation of sustainable, effective, and comprehensive mentoring and career development initiatives that promote diversity in the biomedical research workforce. Our collaborative evaluation design, theoretically-derived measurement instruments, efficient data systems, and timely reporting serve as an example of how to put evaluation principles described into practice for large, multi-site, and multi-dimensional research training programs like NRMN.