Archive for the ‘NIH review process’ Tag

Top Ten Things NIH Reviewers Should NOT Say In A Review

ID-10046929

Credit: Ambro at FreeDigitalPhotos.net

The Center for Scientific Review publishes their Peer Review Notes three times a year, and the most recent issue came out yesterday. The news items are always interesting and it is worth subscribing, if you don’t already. This issue contained an item about things NIH reviewers should not say. I repeat the list in its entirety here—I thought it might be fun for my grantees to see reviewers critiqued for a change.

What do you think of this list? Have you seen one or two of these on your Summary Statements? Me personally? I have seen variations on # 2, 4, and 10 in Summary Statements, and have strongly suspected reviewers of #1 and 5. I almost fell out of my chair laughing when I read # 7, sometimes I think CSR is a little out of touch with what actually happens on Study Sections:

  1. “I didn’t read the application, but I scanned it and saw the applicant said XXX. He doesn’t know what he’s doing.” Damning statements like this can skew a review discussion over something that might be insignificant in the context of the overall application. It’s better for you to ask other reviewers who have read the application carefully what they think about XXX.
  2. “This New Investigator does not appear to be fully independent since he continues to co-publish with his fellowship mentor/department chair, or does not have designated lab space, or has not been promoted in the past several years.”  Academic research organizations have widely diverse policies for faculty advancements and lab space, and many PIs maintain productive and healthy collaborations with mentors for many years after establishing themselves as bona fide investigators. You should focus more on the investigator accomplishments, such as being the first or senior author on a significant publication or giving presentations at major scientific meetings.
  3. “This application is not in my area of expertise . . . “  If you’re assigned an application you feel uncomfortable reviewing, you should tell your Scientific Review Officer as soon as possible before the meeting.
  4. “I don’t see this basic science research affecting my clinical practice any time soon.” An application does not necessarily have to show the potential for clinical or timely impact—if the applicant doesn’t make such claims. Basic research often takes time to pay off, and you’re charged to assess the “likelihood for the project to exert a sustained, powerful influence on the research field(s) involved.” Absence of an effect on public health does not necessarily constitute a weakness in basic science.
  5. “I like this project but I’m giving it a poorer score because the applicant has too much money.” Other funding is not a scoreable matter. You should focus on the application’s scientific and technical merit. However, you can note an excessive budget request in the budget section for NIH to consider.
  6. “This application has 2 great aims and 1 bad one. I would recommend deleting Aim 3, and I can give it a 1 or 2.” You cannot trade aims with scores. The application needs to be evaluated as a whole.
  7. “This R21 application does not have pilot data, which should be provided to ensure the success of the project.” R21s are exploratory projects to collect pilot data. Preliminary data are not required, although they can be evaluated if provided.
  8. “The human subject protection section does not spell out the specifics, but they already got the IRB approval, and therefore, it is ok.” IRB approval is not required at this stage, and it should not be considered to replace evaluation of the protection plans.
  9. “This application was scored a 25 and 14th percentile last time it was reviewed . . . .” You should not mention the previous score an application got, because this could skew the review discussion. Focus on the strengths and weaknesses of the current application as well as the responses to previous critiques.
  10. “This is a fishing expedition.” It would be better if you said the research plan is exploratory in nature, which may be a great thing to do if there are compelling reasons to explore a specific area. Well-designed exploratory or discovery research can provide a wealth of knowledge.
Advertisements

Grantwriting Workshops Offered by Meg Bouvier Medical Writing

On May 8-9, The Arizona Biomedical Research Commission (ABRC) will be hosting a series of workshops on NIH grant submissions, at which I will be the featured presenter. For details and registration information, click here.

Workshops are often a cost-effective way to educate a larger group of faculty on the NIH grant process. In Phoenix next week, I will be kicking off my presentations with a popular 3 ½ hour R01 workshop, which includes a workbook that contains exercises and samples of funded grant applications. After, I will be conducting a series of one-hour breakout groups on topics including NIH submission strategies, resubmissions, mistakes commonly made by applicants, the review process, and how to choose an appropriate funding mechanism (R01, R21, or R03). Each time I present to a group, I work with the client to customize the presentations to address the needs of a particular group of attendees.

The workshops have proved quite popular with departments and institutions and can be taken for CME credit. I draw upon my experience working each year with dozens of NIH submissions and summary statements. My experience as both a bench scientist and staff writer at NIH also informs my approach to NIH grantsmanship and trainings.

Please contact us to discuss a workshop that will fit your needs and budget, and for a sampling of workshop formats and topics.

Will NIH Eliminate The Unpopular “Two Strikes You’re Out” Grant Review Policy?

In January 2009, NIH instituted a policy to sunset A2 applications, i.e. to eliminate a second resubmission. Since, investigators submit a grant application (A0) and if the application is unfunded they may submit one revised application (A1), and no more (previously, one could resubmit twice). This policy has been detested among research scientists since its inception and NIH has heard an earful of complaints about it. NIH spoke out recently on this topic. Will they consider eliminating the wildly unpopular policy? The short answer: Not bloody likely.

In a recent blog post, Sally Rockey gives some data NIH has compiled on this policy. One of the goals of the policy was to reduce the time to award, and in fact the time from A0 submission to award has been reduced from 93 weeks to 56 weeks by this policy. It is conventional wisdom that NIH grant applications, if funded, tend to be successful on their final submission, with very few succeeding as an A0. Yet surprisingly, since the policy went into effect, the proportion of funded A0 applications as compared to A1 apps has increased.

One major complaint of the policy is that it favors established investigators over new investigators. The data do not appear to support this idea, because the time-to-award for new applicants is not appreciably longer than for the entire population.

In response to the suggestion that NIH allow A2 applications only from those investigators whose apps fall just outside the payline, NIH modeled the potential impact of such a policy using FY11 data. From their model, NIH concluded that the result would be to shift awards from A0 and A1 apps toward A2 apps, which was what the policy was designed to avoid to begin with (see original blog post for details of their method.)

Rockey concludes: “Overall, these data indicate that the policy to sunset A2 applications continues to achieve the stated goals of enabling NIH to fund as much meritorious science as possible in as short a time period as possible. Any revision to the policy to allow additional resubmissions of all or a subset of A2 applications will displace equally meritorious A0 and A1 applications, and increase the time to award for many applications. For these reasons, we have decided to continue the policy in its current form.”

Sorry folks, it’s here to stay.

Reviewers of NIH Grant Submissions May Pay A Heavy Price When Their Own Submissions Are Reviewed

Grant scores are being posted right now. I was at a policy meeting last week with one of my grant clients and we waited with baited breath to see her scores. Although she has had a great deal of success on past NIH submissions, she was worried about this one, and with good reason. She recently agreed to serve as a reviewer on an NIH study section, but quickly realized that it meant that the resubmission of her own R01 competing renewal could no longer be reviewed in the most appropriate study section. Instead, it was assigned to a Special Emphasis Panel, or SEP. My client studied the list of reviewers on the SEP and learned that there was no one on the panel with the expertise needed to review her submission. She informed her SRO of this problem, but was told to sit tight and wait and see how the review went.

My client is considered an eminence in her field. She chairs a nationally-ranked academic department in her area, has landed numerous R01s, a U01, ARRA funding, and more. On her R01 competing renewal she scored in the 20th percentile, but the problems identified by the reviewers were certainly fixable and she resubmitted with hope, if not confidence. Then she accepted the position on the study section and the resubmission went to the (underqualified) SEP. The results of the resubmission were posted: Unscored.

It is hardly the first time I have heard such a story, it is just the most recent. It is considered an honor to be invited to serve on a study section. Many, like my client, choose to do so despite the time and effort involved because they feel they should give back for the many years of funding they have received from NIH. But the inadvertent result may be that the reviewer’s own submissions must be reviewed elsewhere, often on a panel that lacks the necessary expertise. As a result, my client and others have decided to terminate their service on study sections, which has the result of denying other NIH grantees reviews from those deemed the most qualified to provide them.

The current policy creates a situation where reviewers of a study section may be unable to get a fair review of their own grant applications at the most appropriate study section, thereby essentially penalizing them for service to NIH.

When discussing this problem last week in Washington with an NIH program officer, I was told to contact my local congressman. The PO felt that if we wanted to effect change to this NIH policy, the only approach was to inform a congressman that his constituents were not receiving millions in federal funding due to this policy. I plan to do so, and urge you all to do the same.

Posted October 30, 2012 by Meg Bouvier in Biomedical research, medical grant writing

Tagged with , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

FAQs on NIH Application Review

Another useful post on a recent NIGMS Feedback Loop Blog – National Institute of General Medical Sciences (Can you tell by the number of recent posts that we are past most of the Cycle III deadlines and I can finally catch up on reading/posting??)

FAQs on Application Review, Next Steps – NIGMS Feedback Loop Blog – National Institute of General Medical Sciences.

Questions include:

YouTube Video for New NIH Applicants

If you are a brand new applicant to NIH, you can find help for navigating the infamously labyrinthine federal grant application process. Last fall, the Center for Scientific Review (CSR) announced a new video to help you get started:

 

“New applicants attending our outreach talks at regional and national meetings often tell us they learn a lot from them. So more can benefit, we produced a new video that essentially puts viewers in the front row of one of these popular PowerPoint presentations.

“Our new “What Happens to Your Grant Application” video provides an overview of CSR and the peer review process. It covers more key details than our NIH Peer Review Revealed video, which presents a mock study section meeting.

View these videos HERE.

 

Other Resources for Research Mentors and Trainers

An outreach flyer associated with the video can be downloaded from our outreach publications Web page: http://cms.csr.nih.gov/publications

CLICK HERE  for a link to the original newsletter piece.

 

%d bloggers like this: