Foundation Scheme Pilot 1: What? Who? When? How?

1. Foundation Scheme Pilot 1:What? Who? When? How? Lori Burrows, PhD CIHR University Delegate June 12, 2014

2. What? First Pilot competition of CIHR’s new Foundation Scheme: 5-7 year grants aimed at scientific LEADERS. If successful, you cannot hold other CIHR open grants as PI during the span of your Foundation grant. You can be a co-investigator on other Foundation or Project grants. Existing grants will be rolled into your Foundation grant. If unsuccessful, you WILL NOT LOSE EXISTING GRANTS. Many elements of this competition remain uncertain, because they are still in development and may yet change based on the results of ongoing pilots.

3. example of ‘still in development’ “The adjudication criteria considered at Stage 1 and 2 are different. A detailed description of these criteria can be found within the CIHR Adjudication and Selection Process Manual for the 2014 Foundation Scheme Competition. (link to come)”

4. Money available for 2014/15 The combined total amount available for 2014-15 Transitional OOGP and Foundation Pilot is ~$500M. The two competitions overlap for the first time…unclear how this will play out in terms of total number funded. 120 to 250 Foundationgrants may be funded;‘most Foundation grant budget requests will fall within a range of $50K to $1.5M per annum’. The budget request should be consistent with the applicant's previous CIHR open grant research funding history. TIP: You can point out that your historic request was higher than the amount you actually got, due to ATB cuts.

5. Who? You must fit CIHR’s ‘independent researcher’ definition & a) be the PI or co-PI on an open CIHR grant that ends between Oct 1, 2014 and Sept 30, 2015, or b) be an established investigator (>5 yr since 1stappt) who has never held a CIHR open grant as PI or co-PI, or c) be a new investigator (<5 yr since 1st appointment); leaves of absence aren’t included, and part time appointments will be pro-rated up to the maximum of 60 months post-appointment CIHR will verify your eligibility at registration.

6. What are defined as ‘open’ grants? Open Operating Grant Program (OOGP) Partnerships for Health System Improvement (PHSI) Knowledge Synthesis Grant (KRS) Knowledge to Action Grant (KAL) Proof of Principle Program Phase I and II (POP I and POP II) Industry-Partnered Collaborative Research (IPCR)

7. New names for PI and co-I Program Leader: individual responsible for setting and overseeing the intellectual and strategic direction of the research. Non-Canadian PLs must have Canadian co-Leader. Program Leader – Administrative Coordinator: must be a PL, at Canadian institution; assumes all responsibility for administrative matters related to the grant including submitting the application and coordinating any required reporting Program Expert: participates in or contributes to the program of research but does not direct the program; may or may not be involved for the entire length of the grant; may or may not be named on the grant at discretion of PL TIP: CVs of Experts are not included or assessed in Foundation applications, and they are not named in funding decisions. They can document their involvement in Foundation grants in their CVs, reports, etc., as agreed upon with the Program Leader(s), and they can apply for Project grants.

8. When? Registration – June 23, 2014 Stage 1 Application – Sept 15, 2014 Stage 1 Decision – Dec 1, 2014 Stage 2 Application – Feb 5, 2015 Stage 2 Decision – May 15, 2015 Stage 3 Decision – July 2, 2015 Funding Start – July 1, 2015 (yes, before the stage 3 decision…)

9. How? • Register via ResearchNet by June 23, 8 pm • If you pass the eligibility test, CIHR will send a link for the stage 1 application, due Sept 15, 2014, 8 pm • If unsuccessful at stage 1 (decision in Dec 2014), you can apply to the Spring 2015 Transitional OGP, registration deadline Jan 2015. • If successful at stage 1, you can apply to stage 2 OR to the Spring TOGP…not both. TIP: Depending on the comments at stage 1, and projected success rates at stages 2 and 3, you may want to consider whether you go forward, or bail out to the TOGP. If you go forward to stage 2 and are unsuccessful, you will have to wait until Fall 2015 to re-apply, with no money until July 2016 even if you succeed the second time.

10. 2014 2015 2016 NO (87%?) NO (87%?) chose to renew early choose to renew early choose to skip Fdn choose to skip Fdn Sept/16 Fdn Mar/15 transitional OOGP Sept/15 Fdn Pilot Sept/14 Fdn Pilot Mar/14 OOGP Mar/16 Project NO NO YES (13%?) YES YES (13%?) Sept/16 Project Sept/15 stage 1 (Dec/15) Sept/14 stage 1 (Dec/14) $ Oct/14 $ July/15 $ July/16 YES decided not to go forward to stage 2 decided not to go forward to stage 2 $ April/17 YES YES Feb/16 stage 2 (May/16) Feb/15 stage 2 (May/15) YES YES $ July/16 $ July/15 NO – REAPPLY MAYBE MAYBE May/15 stage 3 (July/15) May/16 stage 3 (July/16) NO – REAPPLY NO – REAPPLY YES YES NO – REAPPLY $ July/16 $ July/15

11. How do I register? http://www.cihr-irsc.gc.ca/e/48053.html You need a ResearchNet account, a CIHR PIN and a Common CV account.

12. the Common CV • fill out the CIHR version of the Common CV TIP: this can take significant time! • to register for the pilot, pick CIHR as the agency and ‘registration’ for the format • validate it and and click ‘submit’ (it doesn’t go anywhere) • the system will generate a confirmation number that needs to be entered into the registration application on ResearchNet • if you change anything, you need to repeat this process, the confirmation number will change

13. on ResearchNet… • Under ‘Apply for Funding’, pick ‘Foundation Scheme….’ and follow the instructions. • The Program Leader(s) must not change between registration and application. • You must provide a 5 (new investigators) or 7 yr budget estimate. You don’t need to justify at this point, and you can change it at stage 2. TIP: For new people, look at CIHR’s funding database for historic amounts handed out for the type of work you’re proposing to do • Suggest reviewers. This is critical, as these people will be recruited into the College of Reviewers. Exclude reviewers you don’t want.

14. What’s my competition like? • as of June 6, 43 registrations completed, 874 in progress • 305 are current grant holders; 278 have never held CIHR grants as PI (might be some overlap); 267 are new investigators and 67 are undefined • includes registrations opened on spec…but CIHR expects more at the last minute • CIHR is expecting ~1200 bona fide applicants, despite high numbers of registrants (since many will register but not submit). But, they are surprised at the new investigator numbers so far…

15. Stage 1 application (in development) Detailed instructions (soon) available on ResearchNet under ‘Opportunities’, ‘Foundation Scheme’. “To complete your Stage 1 Application, follow the instructions in the Foundation Scheme: 1st Live Pilot ResearchNet "Application" Phase Instructions. (link to come)”

16. Stage 1 application content • Program Leader(s) will need to generate a Foundation Scheme CV using the Common CV • SHORT stage 1 application has 3 sections: • Summary (1 page) • Caliber of the Applicant (1.5 pages, 75%) • -Leadership (1/2 page, 25%) • Significance of contributions (1/2 page, 25%) • Productivity (1/2 page, 25%) • Vision and Program Direction (1 page, 25%)

17. Program Leader’s CV http://www.cihr-irsc.gc.ca/e/48206.html See website for details of what is to be included. Only select contributions, with maximum numbers per section. e.g. Publications (Most relevant – up to 25) ‘Program Leaders will provide information regarding their publications in the form of journal articles, books, book chapters, reports, manuals, clinical care guidelines, and/or conference publications’.

18. Summary (1 page) • summarize research experience and vision of the research program: • overarching focus of research career, major questions or issues explored, and the impact of the research; • expertise and experience; • Important collaborations, within or outside of the research community; and • vision or direction of the program of research to advance knowledge and/or its application to health care, health systems, and/or health outcomes. TIP: the Summary is a MARKETING DOCUMENT – SELL YOURSELF!

19. Leadership (1/2 page) • highlight significant and effective leadership experience, reputation in your field and evidence that you’ve led major projects or research programs – point to examples in your CV • reviewers are asked if you are widely recognized, influential, community-building; if you have established, resourced, directed major efforts including training, infrastructure development, collaborations

20. Significance of Contributions (1/2 page) • highlight your contributions and their impact, tangible benefits or positive influences – give examples from your CV • reviewers are asked if you have significantly advanced knowledge and/or its translation into health care/systems/outcomes; and if you’ve engaged/trained/launched the careers of promising individuals in research or health-related areas TIP: although the sections may have overlap in terms of what you point to, CIHR thinks that true Leaders will have so many examples of their awesomeness, they won’t need to repeat any of them

21. Productivity (1/2 page) • highlight your productivity – quality and quantity of outputs, with examples from CV • reviewers are asked if you have demonstrated an outstanding level of research outputs based on prior work; if your previous work generated high quality research outputs TIP: if there is more than one Program Leader, their joint productivity will be considered

22. Vision and Program Direction (1 page) • articulate a compelling vision and direction for the program…highlighting the goal, overall objective(s), expected outputs/contribution(s), and significance of the proposed program. Specific details for each project in the program are not expected, because the grants are meant to be flexible, to allow innovation and new directions. • reviewers are asked if program is well-defined and well-articulated, logical career progression for the Program Leader(s)? forward-looking, creative, and appropriately ambitious? Does the vision aim to significantly advance knowledge and/or its translation?

23. Stage 2 application • get your stage 1 reviews and an invitation to go forward (or not) • the stage 2 application is 12 pages total plus ½ page of budget justification • will include: • summary (1 page) • quality of program – 40% • research concept (3 pages, 20%)…coherence, significance • research approach (2 pages, 20%)…appropriate, flexible • quality of expertise, experience, resources – 60% • expertise (3 pages, 20%)…who are your Experts? • mentorship and training (2 pages, 20%)…comprehensive plan • quality of support environment (1 page, 20%)…appropriate • budget justification (1/2 page)

24. Peer review of your stage 1 application This critical component remains under development. You will suggest (and exclude) potential reviewers upon registration. CIHR is shooting for 5 reviewers per application; some may be selected from existing peer review committees. Eventually, reviewers will be members of the College of Reviewers. Each reviewer will have up to 20 applications, and after scoring the criteria (and resolving discrepant scores via online discussion with dissenting reviewers), will rank the applications (no ties allowed). Each application will have a combined rank that is an average of 5 individual rankings, and the combined ranks are used to make decisions at stage 1. Right now, estimates are that the top ~350 may go forward to stage 2.

26. Pilot of the Structured Review Forms: Adjudication Scale Stage 1 Reviewers Stage 1 Reviewers stated that they used the full range of the adjudication scale; however, Stage 2 Reviewers disagreed Although the majority of Stage 1 Reviewers stated that the adjudication scale allowed the description of meaningful differences, their comments indicated that this was not true Limitation of Stage 2 Reviewer comments is that they only saw the top 39 (of 77) applications Stage 2 Reviewers

27. Distribution of Ratings: Letter Scale vs. Current Number System new: A+B 73.5% old: A+B 55% 25% *Scientific Merit and Potential Impact based on Spring 2013 Knowledge Synthesis competition consensus score Different scale (ABCDE is too close to letter grades in school)? Could also be a problem for international reviewers as the letters mean different things in different countries. Add granularity to the scale (e.g., A+, A, A-)? 5 letters is interpreted as a 20% difference between letter grades and this is not sensitive enough to capture subtle differences between applications;

28. Stage 1 - Preliminary Reviews • A little more than half of the Stage 1 Reviewers read other reviewer’s reviews • Reading other reviewer’s reviews did not often influence a reviewer’s assessment of the application • In future surveys, this question will be re-framed to: • Ask reviewers whether the ability to read other reviewers’ reviews was useful/helpful • Differentiate the responses between reviewers who had divergent ratings/comments and those who had similar ratings/comments

29. Stage 1 Asynchronous Online Discussion 75% of reviewers did not participate in online discussions… …and not because there was nothing to say. Almost half were not finished with their reviews in time. Reviewers felt their opinion was not very influential, but about half thought others could at least occasionally change their assessment. (Not clear if these were the 25% that did participate).

30. Stage 2: Face-to-Face Meeting Most thought the face-to-face meeting was necessary All of the applications were listed in rank order on a monitor (the top half from Stage 1) and a funding cutoff line was indicated, so people had an idea of which apps needed discussion Applications were grouped into A (green zone), B (grey zone) and C (red zone), and the discussion centered on group B apps.

31. Overall Satisfaction with Structured Review Process Applicant satisfaction with the structured review process correlated with the state of the application (funded, moved to Stage 2, unsuccessful after Stage 1) Most Stage 1 Reviewers were satisfied with review process Most Stage 2 Reviewers were satisfied with review process

32. Registration – June 23, 2014 Stage 1 Application – Sept 15, 2014 Stage 1 Decision – Dec 1, 2014 Stage 2 Application – Feb 5, 2015 Stage 2 Decision – May 15, 2015 Stage 3 Decision – July 2, 2015 TOGP registration – Jan 2015 TOGP application – Mar 2015 If you have questions, comments or concerns, email me! burrowl@mcmaster.ca