Top Ten Things NIH Reviewers Should NOT Say In A Review

ID-10046929

Credit: Ambro at FreeDigitalPhotos.net

The Center for Scientific Review publishes their Peer Review Notes three times a year, and the most recent issue came out yesterday. The news items are always interesting and it is worth subscribing, if you don’t already. This issue contained an item about things NIH reviewers should not say. I repeat the list in its entirety here—I thought it might be fun for my grantees to see reviewers critiqued for a change.

What do you think of this list? Have you seen one or two of these on your Summary Statements? Me personally? I have seen variations on # 2, 4, and 10 in Summary Statements, and have strongly suspected reviewers of #1 and 5. I almost fell out of my chair laughing when I read # 7, sometimes I think CSR is a little out of touch with what actually happens on Study Sections:

  1. “I didn’t read the application, but I scanned it and saw the applicant said XXX. He doesn’t know what he’s doing.” Damning statements like this can skew a review discussion over something that might be insignificant in the context of the overall application. It’s better for you to ask other reviewers who have read the application carefully what they think about XXX.
  2. “This New Investigator does not appear to be fully independent since he continues to co-publish with his fellowship mentor/department chair, or does not have designated lab space, or has not been promoted in the past several years.”  Academic research organizations have widely diverse policies for faculty advancements and lab space, and many PIs maintain productive and healthy collaborations with mentors for many years after establishing themselves as bona fide investigators. You should focus more on the investigator accomplishments, such as being the first or senior author on a significant publication or giving presentations at major scientific meetings.
  3. “This application is not in my area of expertise . . . “  If you’re assigned an application you feel uncomfortable reviewing, you should tell your Scientific Review Officer as soon as possible before the meeting.
  4. “I don’t see this basic science research affecting my clinical practice any time soon.” An application does not necessarily have to show the potential for clinical or timely impact—if the applicant doesn’t make such claims. Basic research often takes time to pay off, and you’re charged to assess the “likelihood for the project to exert a sustained, powerful influence on the research field(s) involved.” Absence of an effect on public health does not necessarily constitute a weakness in basic science.
  5. “I like this project but I’m giving it a poorer score because the applicant has too much money.” Other funding is not a scoreable matter. You should focus on the application’s scientific and technical merit. However, you can note an excessive budget request in the budget section for NIH to consider.
  6. “This application has 2 great aims and 1 bad one. I would recommend deleting Aim 3, and I can give it a 1 or 2.” You cannot trade aims with scores. The application needs to be evaluated as a whole.
  7. “This R21 application does not have pilot data, which should be provided to ensure the success of the project.” R21s are exploratory projects to collect pilot data. Preliminary data are not required, although they can be evaluated if provided.
  8. “The human subject protection section does not spell out the specifics, but they already got the IRB approval, and therefore, it is ok.” IRB approval is not required at this stage, and it should not be considered to replace evaluation of the protection plans.
  9. “This application was scored a 25 and 14th percentile last time it was reviewed . . . .” You should not mention the previous score an application got, because this could skew the review discussion. Focus on the strengths and weaknesses of the current application as well as the responses to previous critiques.
  10. “This is a fishing expedition.” It would be better if you said the research plan is exploratory in nature, which may be a great thing to do if there are compelling reasons to explore a specific area. Well-designed exploratory or discovery research can provide a wealth of knowledge.

NIH Funding to Study Sex as a Fundamental Variable in Clinical Research

Credit: Photokanok at FreeDigitalPhotos.net

Credit: Photokanok at FreeDigitalPhotos.net

I am popping up from my mountain of R01 drafts to bring attention to an important NIH news release. Yesterday, NIH announced it has devoted over $10 million in supplemental funding for 82 grantees to explore sex differences in their clinical and pre-clinical research.

The news release states, “These awards are the latest round of funding in a program described in a May 2014 Nature commentary by [Janine Austin Clayton, M.D., NIH associate director for women’s health research] and NIH Director Francis S. Collins, M.D., Ph.D. This commentary informed NIH grantees and other stakeholders of the agency’s intent to develop policies that will require applicants to address the influence of sex in the design and analysis of biomedical research with animals and cells.”

The news release states that the goal of the supplements is to serve as “…a catalyst for considering sex as a fundamental variable in research.”

NIH began this program in FY13, initially funding 50 supplements ($4.6 million total.) The initiative has been led by the Office of Research on Women’s Health. Most of the NIH ICs have funded supplements since the inception of the program.

Historically, medical research has been conducted predominantly on white male subjects. NIH has made efforts to expand the scope of clinical research to include both sexes and to represent multiple races and ethnicities. Grantees who want to succeed in the NIH arena would be wise to incorporate such variables into current and future studies.

 

 

Rapid Advances in Ebola Research

The current Ebola outbreak is by far the largest since this hemorrhagic fever was identified in 1976. Previous outbreaks involved dozens or hundreds of infected people (click here for CDC chronology). Estimates of the current outbreak are 2,473 infections and 1350 deaths thus far. Outbreaks begin by transmission through close contact with infected animals, then rapidly spread through human communities via direct contact with bodily fluids of infected people, or through contact with items contaminated with such fluids. Once infected, case fatality is as high as 90% (click here for WHO fact sheet). There are currently no vaccines, treatments, or cures. Traditionally, outbreaks have been controlled largely by infection control measures (masks, gloves, etc.) and quarantine, and supportive care such as hydration of the infected patient.

 

Experimental Treatments: A promising drug called ZMAPP was given at Emory University to two missionaries who were infected with Ebola. Both have gotten better. The drug was also given to a Spanish priest who died soon thereafter, though the timing of drug delivery may have played a part in the drug’s efficacy in this case. As of this week, it appears to be helping three Liberian health care workers. The drug is manufactured by Mapp Biopharmaceutical Inc. It is not FDA approved at present, nor can this monoclonal antibody be produced quickly in large quantities. Other drugs are in development but have yet to show as much promise as ZMAPP. Ebola is a rare disease and affects poor countries almost exclusively, so limited funding is provided mostly by government agencies (see $28 million consortium led by Scripps and funded by NIH, and the recent $10.8 million initiative announced by Wellcome Trust and the United Kingdom’s Department of International Development.) I generally distrust .com coverage of anything related to medicine (and so should you), but this recent CNN piece on ZMAPP seems reasonable, if you would like more information.

 

Cause of the Current Outbreak: NIH announced this morning that researchers funded by NIH have used advanced genomic analysis to determine the single point of infection from an animal that led to the current outbreak, and that since that initial infection the spread has been solely human to human. Importantly, through their genetic analysis, the researchers can see how the virus has mutated since December to outsmart human immune systems. As we know, viruses are little more than tiny pieces of DNA that can mutate with diabolical speed to outsmart the comparatively slow human immune response. By understanding how infection occurs, how disease is spread, and how viruses are mutating to defy immune attack, these researchers have taken a giant step toward improved treatments and a cure. The team was led by Pardis Sabeti, MD, PhD (who not surprisingly won a highly prestigious NIH Director’s New Innovator award in 2009.)

 

Experimental Vaccines: Next week, NIAID will begin the first of several phase I clinical trials of an Ebola vaccine produced in collaboration with GlaxoSmithKline (for details, click here). They will also test an Ebola vaccine developed by the Public Health Agency of Canada and licensed to NewLink Genetics Corp. NIH will partner with a British-based international consortium to test volunteers in the UK, and in the West African countries of Gambia (with approval of local authorities) and Mali. The CDC is in discussion with Nigerian officials about testing vaccines there.

Update on the New NIH Biosketch Format

Credit: adamr at  FreeDigitalPhotos.net

Credit: adamr at FreeDigitalPhotos.net

There are changes pending for the NIH biosketch format, and I think it is good news for NIH grantees. The new NIH biosketch format will allow up to five pages for the entire biosketch, as opposed to the current four-page limit. Even better, rather than simply listing publications, the new format will give researchers the opportunity to highlight the magnitude and significance of the scientific advances associated with their discoveries and the specific role they played in those findings.

Grantees will be permitted to describe up to five of their most significant contributions to science, the influence of their contributions on their scientific field, and any subsequent effects of these contributions on the fields of medicine or technology. This will help reviewers better focus on the applicant’s most important contributions to science. Researchers also will be able to include a link to their complete list of publications in SciENcv or My Bibliography.

NIH recently launched a new round of pilot tests (here and here) to make sure the new format will work well for both applicants and reviewers. The pilot will involve surveys of both reviewers and applicants to help NIH fine tune the application instructions and guidance to reviewers. NIH plans to roll out the modified biosketch for all grant applications received for FY 2016 funding and beyond (which generally refers to applications submitted in early 2015).

To learn more about the NIH’s new Biosketch format click *here*

 

NIH Commits to 12-Year Plan for BRAIN Initiative

Credit: Koratmember at FreDigitalPhotos.net

Credit: Koratmember at FreDigitalPhotos.net

Last month a federal report was released calling for $4.5 billion in funding for brain research over the next 12 years. On June 5th, 2014 the Brain Research through Advancing Innovative Neurotechnologies (BRAIN) Initiative was presented to NIH Director Francis Collins by his Advisory Committee to the Director (ACD). The report, drafted by the ACD BRAIN Working Group, maps out a sustained commitment of $4.5 billion in new federal funding over 10 years, beginning in fiscal year 2016, to achieve seven primary goals (see bullets below).

NIH has already announced an investment of $40 million in fiscal year 2014 and President Obama has made a request for $100 million for NIH’s component of the initiative in his fiscal year 2015 budget. The working group emphasized in its report that its cost estimates assume that the budget for the BRAIN Initiative will supplement — not supplant — NIH’s existing investment in the broader spectrum of basic, translational, and clinical neuroscience research.

The NIH efforts on the BRAIN Initiative will focus on mapping the circuits of the brain, measuring the fluctuating patterns of electrical and chemical activity flowing within those circuits, and understanding how their interplay creates our unique cognitive and behavioral capabilities.

The following seven scientific goals were identified as high priorities for achieving this vision:

  • Identify and provide experimental access to the different brain cell types to determine their roles in health and disease.
  • Generate circuit diagrams that vary in resolution from synapses to the whole brain.
  • Produce a dynamic picture of the functioning brain by developing and applying improved methods for large-scale monitoring of neural activity.
  • Link brain activity to behavior with precise interventional tools that change neural circuit dynamics.
  • Produce conceptual foundations for understanding the biological basis of mental processes through development of new theoretical and data analysis tools.
  • Develop innovative technologies to understand the human brain and treat its disorders; create and support integrated brain research networks.
  • Integrate new technological and conceptual approaches produced in the other goals to discover how dynamic patterns of neural activity are transformed into cognition, emotion, perception, and action in health and disease.

The BRAIN Initiative is jointly led by NIH, Defense Advanced Research Projects Agency (DARPA) of the U.S. Department of Defense, National Science Foundation, and Food and Drug Administration. Private organizations are also committed to ensuring success through investment in the initiative.

About the ACD:

The ACD advises the NIH Director on policy matters important to the NIH mission of conducting and supporting biomedical and behavioral research, research training, and translating research results for the public. For more information on the ACD and the full agenda of this meeting, visit: http://acd.od.nih.gov/index.htm

 

 

Seven-Year, Multi-Center Clinical Trial Award 2014

Credit: terapun at FreeDigitalPhotos.net

Credit: Teerapun at FreeDigitalPhotos. net

Meg Bouvier Medical Writing is pleased to announce that our client has been awarded a seven-year Cooperative Agreement from the National Institutes of Health (NIH). The team, working at four participating medical centers, will conduct clinical trials to accelerate the discovery of treatments for critically ill patients. The work will be conducted as part of a highly prestigious, multidisciplinary clinical research consortium at NIH. While the size of the award will depend on the clinical protocols chosen, a typical seven-year clinical trial award runs in the tens of millions of dollars. Dr. Bouvier was the lead writer, editor, and advisor on the U01 submission team from Meg Bouvier Medical Writing.

NIH Common Fund Celebrates 10th Anniversary

Credit: Stuart Miles at FreeDigitalPhotos.net

Credit: Stuart Miles at FreeDigitalPhotos.net

This month the NIH celebrates the 10th anniversary of the NIH Common Fund, a funding mechanism created to support cross-cutting, trans-NIH programs that require participation by at least two NIH Institutes or Centers (ICs). These large collaborative, multi-disciplinary research projects often have the potential to encourage the development of innovative technologies and research tools that, until the development of the Common Fund, would have had difficulty meshing with the plans of any single one of the existing 27 NIH Institutes or Centers.

Over the last decade, the Common Fund has supported significant and transformative research, including the Human Microbiome Project, Big Data to Knowledge (BD2K), Extracellular RNA, Nanomedicine, Epigenomics, Undiagnosed Diseases Program, as well as the High-Risk, High-Reward Research Program that funds individual scientists with particularly innovative ideas or transformative technologies that may lack the preliminary data typically used to evaluate NIH grant applications.

To celebrate this significant milestone in the program’s history, on July 19th the NIH hosted the Common Fund Symposium featuring talks by Dr. Zerhouni, former director of the NIH (2002-2008), as well as many of the remarkable scientists who have led research projects supported by the NIH Common Fund. For those unable to attend the symposium, an archived version of the webcast is accessible to the public here. In addition, over the course of the Symposium, the winners of the first-ever Common Fund video competition were unveiled. This competition encouraged researchers to describe their work to the public utilizing wonderfully creative and often humorous methods, and are well worth a look!

 

Marijuana Is More Damaging Than You Might Have Thought

Credit: Paul at FreeDigitalPhotos.net

Credit: Paul at FreeDigitalPhotos.net

Because there is a nationwide move to legalize (or at least decriminalize) pot, there are a lot more studies on   it now. Older studies are not always relevant because there is so much more THC in today’s pot. The National Institute on Drug Abuse (NIDA) just issued a press release about findings reported in a New England  Journal of Medicine article.

Among the not surprising findings: it’s addictive, it impairs driving, and like alcohol and nicotine it’s a gate- way drug. More surprising: using marijuana as a teenager is more damaging than using it as an adult, probably because the brain is not fully formed until one’s early twenties. The damage to memory and cognition  are more pronounced when used by teens. Using it in your early teens permanently decreases one’s adult IQ, even if you don’t smoke as an adult. Another surprising finding: All users have impaired thought and memory while high, but regardless of age, the deficits actually last for days afterward. An estimated 6.5% of 12th-graders nationwide report daily pot smoking, and 60% do not perceive it as dangerous.

 

Implications for Grantees of no more “Two Strikes You’re Out”

Credit: adamr at  FreeDigitalPhotos.net

Credit: adamr at FreeDigitalPhotos.net

Ding, dong the evil resubmission policy is dead. What are the implications in terms of your grantsmanship strategy and writing?

The original intent of the ill-fated, much-hated “Two Strikes You’re Out” policy was to reduce the time from first submission to award. According to the statistics and metrics put out by NIH, it was achieving that goal. With the policy change that went into effect last month, grantees still submit an A0 (new application) and an A1 (first resubmission), with no A2 (second resubmission) available. But if they fail at both the A0 and A1, they no longer need to shelve the application forever. They can now submit the application again as an A0 to any IC or study section they choose, and the reviewers will be given no information about earlier versions or summary statements. I have yet to find a grantee who is unhappy about the change of policy. ESI and New Investigator grantees in particular have expressed relief that they will have multiple attempts to “get it right” (the incorrect assumption being that study sections are static entities and that each review will be similar to previous reviews in terms of expectations and criticisms.)

Yet, it seems to me that this resubmission policy will lead to even longer lengths of time from first submission to award than were seen when people were allowed three submissions total. In essence, the very problem that “Two Strikes You’re Out” sought to fix will be made worse than it was prior to the “Two Strikes” policy. Here is my tongue-in-cheek mathematical representation of the new resubmission policy:

(A0, A1) ∞

To my mind, this policy equates not only to longer times to award than ever before, but also to an unprecedented number of NIH submissions going forward. In recent years, the total aggregate number of R01 submissions has been fairly steady at about 28 or 29 thousand per year. I expect we will see a dramatic increase in that number in FY14 and into FY15, as all those shelved, failed submissions go back into circulation. At some point that number will decrease a bit to a steady state that is still significantly higher than 29 thousand applications per year.

Love or hate the new policy (and while I recognize that “Two Strikes” was flawed, I am firmly in the latter camp), how should you fold this information into your approach to grantsmanship?

1. Consider deferring your June 5 A0 to the Oct deadline. I suspect that everyone and his brother will be dusting off an old, unfunded application and submitting an A0 for June 5. I suspect that that will equate to a gigantic number of submissions for this cycle, and my suspicions appear to be supported by the sheer volume of A0s on which I have been contracted to work. (I haven’t seen a crushing number of applications like this since the stimulus money.)

2. Give yourself more time to upload to the grants.gov portal with any R01 deadline going forward, but particularly for the June 5 deadline. As you know, applications must be uploaded and time stamped. When traffic at the portal is heavy, there are delays in your ability to upload. Many grantees have told me they missed a deadline because they waited too long and could not upload due to traffic at the portal, or not enough time to correct error messages received. This problem will be exacerbated by the increased application volume from now on.

3. Format your application with greater care than ever. A sharp increase in submissions will likely mean that reviewers will be even more overworked than ever, and the increased time/cost of reviewing all these extra applications may mean that CSR will step up its experiments with virtual peer review. How should this affect your writing? Use lots of unique identifiers throughout the application (numbering system with multiple subheadings) to help orient reviewers who are exhausted and/or not meeting face to face while discussing your application. Make judicious use of formatting to highlight key words and phrases so that reviewers can skim and quickly grasp your main points.

4. Understand that the time from first submission to funding may be very, very long. Plan your work accordingly. Your career strategy must be able to accommodate long review times at NIH. Your career cannot come to a standstill while you wait years to find out if you will receive NIH funding. Understand that you cannot have the same science in review at two NIH ICs simultaneously. However, you can and should submit the same science to different federal agencies simultaneously (ex- NIH and HRSA, AHRQ, DoD, PCORI, NSF, etc). And of course you can blanket the private foundation landscape with the same project. (Should you receive funding from multiple entities— may you have such a problem—this conundrum can generally be worked out in consultation with the program officers.)

5. The standard for quality of a submission is likely to be raised, so you will need to write stronger applications than ever before. Historically, it seemed that if reviewers knew you had a resubmission available to you, they expected you to use it to hone and polish the submission. Because reviewers know you have endless resubmissions available, I wonder if they might raise the bar for quality of the application. They may want you to resubmit until they feel it is just right (which is a moving target, given that your original reviewer(s) may have rotated off the study section before you resubmit.) In addition, a sharp increase in the number of submissions will likely mean that competition will be stiffer, so the quality of the submission will need to be even higher to stand out from the larger crowd. Grantees will need to write their applications more thoughtfully than ever, taking extra time and care.

6. Institutions also will need to step up their game if they want to stay ahead of the competition. If your institution does not do so already, now is the time to implement Chalk Talks and Red Team Reviews (i.e., mock study sections) if you want to help your grantees succeed with the policy change. And of course, hire your grantees a professional grant writer for a few thousand dollars to improve the quality of their R01. It will improve their grantsmanship not just on this submission but on all submissions going forward, and if it helps land even one award it will have paid for itself by many orders of magnitude in the indirects received.

7. Write every submission as if it were your only shot at funding. I am afraid that less savvy grantees may think that they should jump into the game and submit an application that may not represent their best effort. After all, what harm is there in doing so if they can just keep submitting? And wouldn’t it be helpful to have multiple summary statements to hone one’s grantsmanship? Keep in mind that if you are submitting to the same study section repeatedly, they may not formally be given previous submissions and summary statements. But if you have some of the same reviewers, they will remember your past submissions, and it may color their impressions of the current submission. I liken it to a jury who hears testimony and is then instructed by a judge to ignore it in their deliberations. We know from the social psychology literature that despite our best efforts, it is not feasible to act as if we do not know something. If your assigned reviewers still serve on the study section, they may recognize your A0 as something they read in the past, and may recall their reaction to the previous submissions. Reviewers, like jurors, are human. Therefore, I maintain that grantees should write every submission as if it were their only one.

Can you think of other ways you will need to alter your approach to NIH applications because of the new resubmission policy? Do you like or dislike the new policy?

Grantwriting Workshops Offered by Meg Bouvier Medical Writing

On May 8-9, The Arizona Biomedical Research Commission (ABRC) will be hosting a series of workshops on NIH grant submissions, at which I will be the featured presenter. For details and registration information, click here.

Workshops are often a cost-effective way to educate a larger group of faculty on the NIH grant process. In Phoenix next week, I will be kicking off my presentations with a popular 3 ½ hour R01 workshop, which includes a workbook that contains exercises and samples of funded grant applications. After, I will be conducting a series of one-hour breakout groups on topics including NIH submission strategies, resubmissions, mistakes commonly made by applicants, the review process, and how to choose an appropriate funding mechanism (R01, R21, or R03). Each time I present to a group, I work with the client to customize the presentations to address the needs of a particular group of attendees.

The workshops have proved quite popular with departments and institutions and can be taken for CME credit. I draw upon my experience working each year with dozens of NIH submissions and summary statements. My experience as both a bench scientist and staff writer at NIH also informs my approach to NIH grantsmanship and trainings.

Please contact us to discuss a workshop that will fit your needs and budget, and for a sampling of workshop formats and topics.

%d bloggers like this: