Project description:In this paper, we estimate the impact of receiving an NIH grant on subsequent publications and citations. Our sample consists of all applications (unsuccessful as well as successful) to the NIH from 1980 to 2000 for standard research grants (R01s). Both OLS and IV estimates show that receipt of an NIH research grant (worth roughly $1.7 million) leads to only one additional publication over the next five years, which corresponds to a 7 percent increase. The limited impact of NIH grants is consistent with a model in which the market for research funding is competitive, so that the loss of an NIH grant simply causes researchers to shift to another source of funding.
Project description:ObjectiveTo quantify randomness and cost when choosing health and medical research projects for funding.DesignRetrospective analysis.SettingGrant review panels of the National Health and Medical Research Council of Australia.ParticipantsPanel members' scores for grant proposals submitted in 2009.Main outcome measuresThe proportion of grant proposals that were always, sometimes, and never funded after accounting for random variability arising from differences in panel members' scores, and the cost effectiveness of different size assessment panels.Results59% of 620 funded grants were sometimes not funded when random variability was taken into account. Only 9% (n = 255) of grant proposals were always funded, 61% (n = 1662) never funded, and 29% (n=788) sometimes funded. The extra cost per grant effectively funded from the most effective system was $A18,541 (£11,848; €13,482; $19,343).ConclusionsAllocating funding for scientific research in health and medicine is costly and somewhat random. There are many useful research questions to be addressed that could improve current processes.
Project description:Visuospatial working memory (VSWM) is essential to human cognitive abilities and is associated with important life outcomes such as academic performance. Recently, a number of reliable measures of VSWM have been developed to help understand psychological processes and for practical use in education. We sought to extend this work using Item Response Theory (IRT) and Computerised Adaptive Testing (CAT) frameworks to construct, calibrate and validate a new adaptive, computerised, and open-source VSWM test. We aimed to overcome the limitations of previous instruments and provide researchers with a valid and freely available VSWM measurement tool. The Jack and Jill (JaJ) VSWM task was constructed using explanatory item response modelling of data from a sample of the general adult population (Study 1, N = 244) in the UK and US. Subsequently, a static version of the task was tested for validity and reliability using a sample of adults from the UK and Australia (Study 2, N = 148) and a sample of Russian adolescents (Study 3, N = 263). Finally, the adaptive version of the JaJ task was implemented on the basis of the underlying IRT model and evaluated with another sample of Russian adolescents (Study 4, N = 239). JaJ showed sufficient internal consistency and concurrent validity as indicated by significant and substantial correlations with established measures of working memory, spatial ability, non-verbal intelligence, and academic achievement. The findings suggest that JaJ is an efficient and reliable measure of VSWM from adolescent to adult age.
Project description:Across four studies, we found evidence for an implicit pro-White leadership bias that helps explain the underrepresentation of ethnic minorities in leadership positions. Both White-majority and ethnic minority participants reacted significantly faster when ethnically White names and leadership roles (e.g., manager; Study 1) or leadership traits (e.g., decisiveness; Study 2 & 3) were paired in an Implicit Association Test (IAT) rather than when ethnic minority names and leadership traits were paired. Moreover, the implicit pro-White leadership bias showed discriminant validity with the conventional implicit bias measures (Study 3). Importantly, results showed that the pro-White leadership bias can be weakened when situational cues increase the salience of a dual identity (Study 4). This, in turn, can diminish the explicit pro-White bias in promotion related decision making processes (Study 4). This research offers a new tool to measure the implicit psychological processes underlying the underrepresentation of ethnic minorities in leadership positions and proposes interventions to weaken such biases.
Project description:There has been a surge of interest in research integrity over the last decade, with a wide range of studies investigating the prevalence of questionable research practices (QRPs). However, nearly all these studies focus on research design, data collection and analysis, and hardly any empirical research has been done on the occurrence of QRPs in the context of research funding. To fill this gap, we conducted a cross-sectional pre-registered survey of applicants, reviewers and panel members from the Research Foundation-Flanders (FWO), one of the main funding agencies in Belgium. We developed a bespoke survey and further refined it through feedback from experienced researchers and a pilot study. We asked how often respondents had engaged in a series of QRPs over the last ten years. A total of 1748 emails were sent, inviting recipients to participate in the survey, complemented by featuring the survey in the FWO newsletter. This resulted in 704 complete responses. Our results indicate that such QRPs are remarkably prevalent. Of the 496 participants who answered both the applicant and reviewer track, more than 60% responded that they engaged regularly in at least one of such practices, and around 40% indicated that they engaged at least occasionally in half of the QRPs queried. Only 12% reported not to have engaged in any of the QRPs. Contrary to our hypotheses, male respondents did not self-report to engage in the QRPs more often than female respondents, nor was there an association between the prevalence of QRPs and self-reported success rate in grant funding. Furthermore, half of the respondents indicated that they doubted the reliability of the grant peer review process more often than not. These results suggest that preventive action is needed, and provide new reasons to reconsider the practice of allocating research money through grant peer review.
Project description:Disruptive advancements in science and technology often rely on new ideas and findings, which in turn brings us to focus on the value of novelty in scholarly activities. Using Web of Science publication data from European regions for the period between 2008 and 2017, this study examines, first, the impact of scientific collaboration on novelty of research. Here, five levels of collaboration are considered for each article-country, three levels of regions, and institutions, and novelty is measured with keywords information. Second, we investigate both the effect and moderating effect of research funding on novelty. Our findings show that there is a negative and significant relationship between scientific collaboration and novelty. Furthermore, funded papers show lower novelty than the unfunded, but funding does have a significant moderating effect on the relationship between collaboration and novelty. This study contributes by linking diverse levels of collaboration and funding sources to article's novelty and thus extending the scope of bibliometric research of publications.
Project description:Obtaining grant funding is a fundamental component to achieving a successful research career. A successful grant application needs to meet specific mechanistic expectations of reviewers and funders. This paper provides an overview of the importance of grant funding within medical education, followed by a stepwise discussion of strategies for creating a successful grant application for medical education-based proposals. The last section includes a list of available medical education research grants.
Project description:IntroductionTo study the effectiveness of any educational intervention for faculty requires first that they attend the training. Using attendance as a measure of faculty engagement, this study examined factors associated with the percentage of faculty in divisions of departments of medicine who attended a workshop as part of a multisite study.MethodsBetween October 2018 and March 2020, 1675 of 4767 faculty in 120 divisions of 14 departments of medicine attended a 3-hour in-person workshop as part of the Bias Reduction in Internal Medicine (BRIM) initiative. This paper describes the workshop development and study design. The number of faculty per division ranged from 5 to 296. Attendance rates varied from 2.7% to 90.1%. Taking a quality improvement approach, the study team brainstormed factors potentially related to variations in workshop attendance, constructed several division- and institution-level variables, and assessed the significance of factors on workshop attendance with hierarchical linear models.ResultsThe following were positively associated with workshop attendance rate: the division head attended the workshop, the BRIM principal investigator gave Medical Grand Rounds, and the percentage of local workshop presenters who completed training. Workshop attendance rates fell when departments identified more than five on-site study leaders.ConclusionsFactors associated with higher workshop attendance may have increased the perceived status and value of attending the workshop, leading faculty to choose the workshop over other competing demands. For future investigators studying educational interventions that require participation of faculty in clinical departments at multiple sites, this work offers several valuable lessons.
Project description:As funding is one of the key pillars of research activity, identifying the factors that impact the evaluation results in research funding competitions remains challenging, due to the heterogeneity of funding instruments. In this context, our study aims to identify the elements that ensure the application's success, comparing two perspectives: one of the applicant and the other based on the evaluation grid. The empirical investigation focuses on a survey of 243 Romanian researchers. As analysis methods, we use a binary logistic regression model to correlate the success in funding competitions for research projects with a set of factors considered determinants. The results show that the researcher's past performance influences the proposal's future performance/success, with the quality of the project director's previous publications, and its international visibility being the key drivers of successful research project applications.