Assessing the value of a Small Grants Program for behavioral research in cancer control.

In 1999, the National Cancer Institute (NCI) issued the first Small Grants Program (SGP) for Behavioral Research in Cancer Control (R03) funding opportunity announcement for investigators new to behavioral cancer prevention and control research. We explored whether the SGP was successful in its goals to encourage new investigators from a variety of disciplines to apply their skills to and promote career development in behavioral cancer prevention and control research. A quasi-experimental design examined applicant characteristics and outcome data by award status. Propensity score matching was used to compare awardees and non-awardees with similar impact scores as a control for application quality. Awardees were more likely than non-awardees to pursue and receive subsequent funding from the NCI and publish their research. Tailored small grant programs create benefit for both promoting and retaining new investigators.

Outcome evaluation of the National Cancer Institute career development awards program.

The National Cancer Institute (NCI) career development (K) awards program supports investigators to develop their cancer research programs and achieve independence. The NCI Center for Cancer Training conducted a K program evaluation by analyzing outcomes of awardees and individuals who applied to the program but were not funded. The evaluation covered seven NCI mechanisms (K01, K07, K08, K11, K22, K23, and K25) between 1980 and 2008. Descriptive statistics and regression modeling were performed on the full cohort (n = 2,893 individuals, 4,081 K applications) and a comparison cohort described herein. K awardees proportionately received more subsequent NIH grants and authored more publications, and time to first R01 grant was unaffected. Of those not pursuing research, K awardees were more likely to participate in activities signaling continued scientific engagement. The NCI K program had a positive impact, not only on participants' biomedical research careers but also on achieving outcomes significant to the scientific enterprise.

Piloting an approach to rapid and automated assessment of a new research initiative: Application to the National Cancer Institute's Provocative Questions initiative.

Funders of biomedical research are often challenged to understand how a new funding initiative fits within the agency's portfolio and the larger research community. While traditional assessment relies on retrospective review by subject matter experts, it is now feasible to design portfolio assessment and gap analysis tools leveraging administrative and grant application data that can be used for early and continued analysis. We piloted such methods on the National Cancer Institute's Provocative Questions (PQ) initiative to address key questions regarding diversity of applicants; whether applicants were proposing new avenues of research; and whether grant applications were filling portfolio gaps. For the latter two questions, we defined measurements called focus shift and relevance, respectively, based on text similarity scoring. We demonstrate that two types of applicants were attracted by the PQs at rates greater than or on par with the general National Cancer Institute applicant pool: those with clinical degrees and new investigators. Focus shift scores tended to be relatively low, with applicants not straying far from previous research, but the majority of applications were found to be relevant to the PQ the application was addressing. Sensitivity to comparison text and inability to distinguish subtle scientific nuances are the primary limitations of our automated approaches based on text similarity, potentially biasing relevance and focus shift measurements. We also discuss potential uses of the relevance and focus shift measures including the design of outcome evaluations, though further experimentation and refinement are needed for a fuller understanding of these measures before broad application.