Archive for the ‘NIH review’ Tag

Top Ten Things NIH Reviewers Should NOT Say In A Review

ID-10046929

Credit: Ambro at FreeDigitalPhotos.net

The Center for Scientific Review publishes their Peer Review Notes three times a year, and the most recent issue came out yesterday. The news items are always interesting and it is worth subscribing, if you don’t already. This issue contained an item about things NIH reviewers should not say. I repeat the list in its entirety here—I thought it might be fun for my grantees to see reviewers critiqued for a change.

What do you think of this list? Have you seen one or two of these on your Summary Statements? Me personally? I have seen variations on # 2, 4, and 10 in Summary Statements, and have strongly suspected reviewers of #1 and 5. I almost fell out of my chair laughing when I read # 7, sometimes I think CSR is a little out of touch with what actually happens on Study Sections:

  1. “I didn’t read the application, but I scanned it and saw the applicant said XXX. He doesn’t know what he’s doing.” Damning statements like this can skew a review discussion over something that might be insignificant in the context of the overall application. It’s better for you to ask other reviewers who have read the application carefully what they think about XXX.
  2. “This New Investigator does not appear to be fully independent since he continues to co-publish with his fellowship mentor/department chair, or does not have designated lab space, or has not been promoted in the past several years.”  Academic research organizations have widely diverse policies for faculty advancements and lab space, and many PIs maintain productive and healthy collaborations with mentors for many years after establishing themselves as bona fide investigators. You should focus more on the investigator accomplishments, such as being the first or senior author on a significant publication or giving presentations at major scientific meetings.
  3. “This application is not in my area of expertise . . . “  If you’re assigned an application you feel uncomfortable reviewing, you should tell your Scientific Review Officer as soon as possible before the meeting.
  4. “I don’t see this basic science research affecting my clinical practice any time soon.” An application does not necessarily have to show the potential for clinical or timely impact—if the applicant doesn’t make such claims. Basic research often takes time to pay off, and you’re charged to assess the “likelihood for the project to exert a sustained, powerful influence on the research field(s) involved.” Absence of an effect on public health does not necessarily constitute a weakness in basic science.
  5. “I like this project but I’m giving it a poorer score because the applicant has too much money.” Other funding is not a scoreable matter. You should focus on the application’s scientific and technical merit. However, you can note an excessive budget request in the budget section for NIH to consider.
  6. “This application has 2 great aims and 1 bad one. I would recommend deleting Aim 3, and I can give it a 1 or 2.” You cannot trade aims with scores. The application needs to be evaluated as a whole.
  7. “This R21 application does not have pilot data, which should be provided to ensure the success of the project.” R21s are exploratory projects to collect pilot data. Preliminary data are not required, although they can be evaluated if provided.
  8. “The human subject protection section does not spell out the specifics, but they already got the IRB approval, and therefore, it is ok.” IRB approval is not required at this stage, and it should not be considered to replace evaluation of the protection plans.
  9. “This application was scored a 25 and 14th percentile last time it was reviewed . . . .” You should not mention the previous score an application got, because this could skew the review discussion. Focus on the strengths and weaknesses of the current application as well as the responses to previous critiques.
  10. “This is a fishing expedition.” It would be better if you said the research plan is exploratory in nature, which may be a great thing to do if there are compelling reasons to explore a specific area. Well-designed exploratory or discovery research can provide a wealth of knowledge.
Advertisements

Grantwriting Workshops Offered by Meg Bouvier Medical Writing

On May 8-9, The Arizona Biomedical Research Commission (ABRC) will be hosting a series of workshops on NIH grant submissions, at which I will be the featured presenter. For details and registration information, click here.

Workshops are often a cost-effective way to educate a larger group of faculty on the NIH grant process. In Phoenix next week, I will be kicking off my presentations with a popular 3 ½ hour R01 workshop, which includes a workbook that contains exercises and samples of funded grant applications. After, I will be conducting a series of one-hour breakout groups on topics including NIH submission strategies, resubmissions, mistakes commonly made by applicants, the review process, and how to choose an appropriate funding mechanism (R01, R21, or R03). Each time I present to a group, I work with the client to customize the presentations to address the needs of a particular group of attendees.

The workshops have proved quite popular with departments and institutions and can be taken for CME credit. I draw upon my experience working each year with dozens of NIH submissions and summary statements. My experience as both a bench scientist and staff writer at NIH also informs my approach to NIH grantsmanship and trainings.

Please contact us to discuss a workshop that will fit your needs and budget, and for a sampling of workshop formats and topics.

The End of Two Strikes You’re Out: Good or Bad News for Grant Applicants?

Yesterday, NIH and AHRQ announced that they had changed its resubmission policy. While a given application is still allowed only one resubmission, if you are unsuccessful on the A1, you can then submit the application again as an A0 without having to substantially redesign the content and scope of the project. This “new submission” will be reviewed without any association to the previous attempts. There will be no Introduction to the Revised Application, no explanation of how you addressed the reviewers’ concerns. Even if reviewers have seen the application in prior review cycles, they will be instructed to review it as new. (However, reviewers are human, and one wonders if this is realistic. It’s like asking a juror to ignore evidence they just heard, and we know from social psychologists that that doesn’t actually work.) Of course, the idea is that an applicant will use previous reviewer comments to strengthen the application, thereby improving their odds of funding.

Click here to read the full notice

In 2009, as part of their Enhancing Peer Review project, NIH eliminated the A2 in what has been dubbed by unhappy researchers as the “Two Strikes You’re Out” policy. NIH eliminated the A2 because meritorious research was most likely funded on the A2, which meant quite a delay to funding. The resubmission policy did indeed result in an increase in the number of awards made on A0 applications (although most funded applications are successful on the A1). About the “Two Strikes You’re Out” policy, Deputy Director Dr. Sally Rockey states: “…we heard increasing concerns from the community about the impact of the policy on new investigators because finding new research directions can be quite difficult during this phase of their career. Also, established investigators voiced concern about the need to redirect the research focus of productive labs in order to submit future NIH applications.” For one of many discussions on Dr. Rockey’s blog about the decision to sunset the A2 submission, click here.

In theory, it would appear that one could submit the same idea endlessly until one finds the best way to sell it to reviewers. The policy will no doubt appeal to many researchers, because it is now up to a researcher to decide when it is time to abandon a given project. If one waits long enough, one could submit to the same study section after most of the members have turned over, thereby having a new set of reviewers to weigh in on the project. One risk I see is that sometimes a PI is so blindly enamored with their idea that they have difficulty hearing that the reviewers are trying to tell them that the idea is simply not fundable in any form; i.e., no amount of tinkering with the writing or the details will fix it. A PI could waste a lot of time if they cannot see when it is time to abandon an unfundable project.

Judging from the comments on some of the NIH blogs (for examples, click here), many reviewers are thrilled to have more submission attempts, while others are dismayed that this marks a return to a huge number of submissions and long delays to funding. What do you think?

2013 Year In Review at NIH

Here is a link to a useful blog post written by NH Deputy Director Sally Rockey. It summarizes some of the main activities at NIH during the course of 2013. While it begins with a depressing recap of the far-reaching effects of the budget situation, it goes on to highlight some of the main goals and programs of the year. Major themes continue to include data science and efforts to diversify the scientific workforce. The blog is chock-full of hyperlinks to more information on numerous topics. If you plan to develop a long-term relationship with a federal funding agency, it is important to know its mission and funding priorities, and to familiarize yourself each year with the goals identified in the director’s appropriation report, and the suggestions made by the advisory council to the director. Read the full blog post here.

Will NIH Eliminate The Unpopular “Two Strikes You’re Out” Grant Review Policy?

In January 2009, NIH instituted a policy to sunset A2 applications, i.e. to eliminate a second resubmission. Since, investigators submit a grant application (A0) and if the application is unfunded they may submit one revised application (A1), and no more (previously, one could resubmit twice). This policy has been detested among research scientists since its inception and NIH has heard an earful of complaints about it. NIH spoke out recently on this topic. Will they consider eliminating the wildly unpopular policy? The short answer: Not bloody likely.

In a recent blog post, Sally Rockey gives some data NIH has compiled on this policy. One of the goals of the policy was to reduce the time to award, and in fact the time from A0 submission to award has been reduced from 93 weeks to 56 weeks by this policy. It is conventional wisdom that NIH grant applications, if funded, tend to be successful on their final submission, with very few succeeding as an A0. Yet surprisingly, since the policy went into effect, the proportion of funded A0 applications as compared to A1 apps has increased.

One major complaint of the policy is that it favors established investigators over new investigators. The data do not appear to support this idea, because the time-to-award for new applicants is not appreciably longer than for the entire population.

In response to the suggestion that NIH allow A2 applications only from those investigators whose apps fall just outside the payline, NIH modeled the potential impact of such a policy using FY11 data. From their model, NIH concluded that the result would be to shift awards from A0 and A1 apps toward A2 apps, which was what the policy was designed to avoid to begin with (see original blog post for details of their method.)

Rockey concludes: “Overall, these data indicate that the policy to sunset A2 applications continues to achieve the stated goals of enabling NIH to fund as much meritorious science as possible in as short a time period as possible. Any revision to the policy to allow additional resubmissions of all or a subset of A2 applications will displace equally meritorious A0 and A1 applications, and increase the time to award for many applications. For these reasons, we have decided to continue the policy in its current form.”

Sorry folks, it’s here to stay.

NIH Names New Director for the Center for Scientific Review

NIH Director Francis Collins announced yesterday tthat Richard Nakamura, Ph.D. will be the new director for the NIH’s Center for Scientific Review (CSR). Dr. Nakamura has been serving as the acting director since September 2011.

Read the full press release.

Reviewers of NIH Grant Submissions May Pay A Heavy Price When Their Own Submissions Are Reviewed

Grant scores are being posted right now. I was at a policy meeting last week with one of my grant clients and we waited with baited breath to see her scores. Although she has had a great deal of success on past NIH submissions, she was worried about this one, and with good reason. She recently agreed to serve as a reviewer on an NIH study section, but quickly realized that it meant that the resubmission of her own R01 competing renewal could no longer be reviewed in the most appropriate study section. Instead, it was assigned to a Special Emphasis Panel, or SEP. My client studied the list of reviewers on the SEP and learned that there was no one on the panel with the expertise needed to review her submission. She informed her SRO of this problem, but was told to sit tight and wait and see how the review went.

My client is considered an eminence in her field. She chairs a nationally-ranked academic department in her area, has landed numerous R01s, a U01, ARRA funding, and more. On her R01 competing renewal she scored in the 20th percentile, but the problems identified by the reviewers were certainly fixable and she resubmitted with hope, if not confidence. Then she accepted the position on the study section and the resubmission went to the (underqualified) SEP. The results of the resubmission were posted: Unscored.

It is hardly the first time I have heard such a story, it is just the most recent. It is considered an honor to be invited to serve on a study section. Many, like my client, choose to do so despite the time and effort involved because they feel they should give back for the many years of funding they have received from NIH. But the inadvertent result may be that the reviewer’s own submissions must be reviewed elsewhere, often on a panel that lacks the necessary expertise. As a result, my client and others have decided to terminate their service on study sections, which has the result of denying other NIH grantees reviews from those deemed the most qualified to provide them.

The current policy creates a situation where reviewers of a study section may be unable to get a fair review of their own grant applications at the most appropriate study section, thereby essentially penalizing them for service to NIH.

When discussing this problem last week in Washington with an NIH program officer, I was told to contact my local congressman. The PO felt that if we wanted to effect change to this NIH policy, the only approach was to inform a congressman that his constituents were not receiving millions in federal funding due to this policy. I plan to do so, and urge you all to do the same.

Posted October 30, 2012 by Meg Bouvier in Biomedical research, medical grant writing

Tagged with , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

FAQs on NIH Application Review

Another useful post on a recent NIGMS Feedback Loop Blog – National Institute of General Medical Sciences (Can you tell by the number of recent posts that we are past most of the Cycle III deadlines and I can finally catch up on reading/posting??)

FAQs on Application Review, Next Steps – NIGMS Feedback Loop Blog – National Institute of General Medical Sciences.

Questions include:

Learn About The NIH Application Review Process

Most grant applications and contract proposals to NIH are reviewed in the Center for Scientific Review. The CSR puts out a newsletter periodically with updates and tidbits for NIH applicants. It is well worth subscribing to this newsletter, as well as spending some time perusing past issues. Here is the link to the latest edition:

Latest Issue of the Peer Review Notes

http://internet.csr.nih.gov/WebAnalytics/QRRedirect.aspx?param=PRNSep12Listserv

*   How Well Is NIH Identifying and Advancing Innovative Research?

*   Former Study Section Chairs Share Advice for New Reviewers

*   Make the Best Use of the “Additional Comments to Applicant” Box

*   Who Are the Other People in Your Review Meetings?

*   CSR Reviews for the NIH-FDA Collaboration on Tobacco Control Regulatory Research

How Cover Letters Aid NIH Application Reviews

I am often asked whether one should write a cover letter to accompany an application. Many of my clients feel it is a waste of time. I completely disagree. Here is a section on the topic from the May 2011 “Peer Review Notes”, a newsletter put out by the Center for Scientific Review:

NIH encourages applicants to submit cover letters to help guide applications to our review groups and give us other information that will help us review them. A majority of applicants now take advantage of this opportunity.

 

Popular Reasons to Use a Cover Letter

• Suggest we assign your application to a review group you think is best.

• Suggest we assign your application to an NIH institute(s) or center(s) you think would be interested in your research.

• Describe the kinds of expertise needed to review your application.

• Let us know about potential reviewers who you think might be in conflict with your application.

Our scientific staff members make the final decisions after they carefully consider your suggestions and explanations.

 

Suggesting a Study Section

We designed our study sections with a deliberate amount of overlap, so more than one study section may have the expertise to review your grant application. You may express a preference, and we will work to accommodate you if possible.

 

• Check our online study section descriptions to identify a review group you think is best suited to review your application. Last year, this area of our Web site registered nearly 1.7 million page views.

• Examine recent study section rosters to help you gauge the scope of our study sections. But note that CSR study section rosters can change significantly from round to round since we recruit reviewers for a meeting based on the specific scientific content of the applications to be reviewed.

• Consider seeking guidance from a CSR scientific review officer overseeing a study section you think could best review your application. Program officers can also give guidance on suitable study sections.

 

Requesting Assignments to NIH Institute(s) and Center(s)

You can also request that your application be assigned to one or more NIH institutes or centers you think would be interested in your research. It’s usually a good idea to contact one or more NIH program officer(s) to get guidance. You can identify program officers via the NIH Institute and Center staff listings on their respective Web sites.

 

Helping Ensure Your Review is Appropriate and Unbiased

 

• Note essential expertise needed to evaluate your application in your cover letter. You

should not, however, list the names of potential reviewers.

• Identify reviewers who you think could be in conflict with your application. Learn

about these conflicts here.

 

Your scientific review officer (SRO) will consider the situation and make the final decision. If he/she agrees there is a conflict, the reviewer will not be assigned to your application and will not be in the room when it is discussed. Rosters are typically posted online 30 days before your review meetings, and if you see a reviewer on it who could be biased, contact your SRO as soon as possible.

 

Notifying NIH that You Are a Reviewer Eligible to Submit an Application Without a Deadline

Learn more about this and other uses of cover letters by checking out our new cover letter Web page.

 

 

%d bloggers like this: