Arlington, VA March 31, 2004

1. Presentation for the Advisory Committee for Business & Operations Effective Practices Research Overview For Merit Review Arlington, VA March 31, 2004 This document is confidential and is intended solely for the use and information of the client to whom it is addressed. NSF BUSINESS ANALYSIS

2. Today, we will provide… • Brief update on the status of the NSF Business Analysis • Summary of the key findings from the Effective Practices research for Merit Review

3. NSF Business Analysis has recently competed the following activities and is currently moving into the design phase of the project Quick Snapshot of Project Status Completed as of 3/31/04 “Works in Progress” • Completed researching effective practices for the Merit Review and Award Management & Oversight processes • Completed workload study that assessed the amount and distribution of workload across NSF • Completed first full draft of the Target Enterprise Architecture for Applications, Data, Network and Security • Developing process scenarios and preliminary business cases for the Merit Review and Award Management & Oversight processes • Conducting an eJacket study to determine how the electronic-jacket technology tool impacts human capital issues • Developing an IT Technology Governance Framework and IT Implementation Plan that considers the 2-7 year transition plan necessary to move to the Target Enterprise Architecture

4. For the “effective” practices research, the team researched the practices of grant-making organizations in order to identify potential alternative practices Research Objectives: • To understand how other federal and non-federal organizations review proposals and manage awards, and the relative emphasis that is placed on either activity • To provide NSF with insights into how other organizations address challenges similar to those faced by the Foundation in the proposal review and award management processes • To identify practices or principles that could be leveraged or modified by NSF in the redesign of its core business processes

5. The team interviewed representatives of 14 federal and non-federal organizations Organizations Interviewed

6. Important to note is that few effective practices were identified in the areas of performance measurement, “working across the organization,” and technology usage • Few organizations apply performance metrics to their review or award management practices • Customer satisfaction surveys are performed ‘ad hoc’ • Processing times tend to be the same or longer than NSF • Few agencies have monitoring plans in place to address the degree of award oversight that is applied • Few organizations had a significant need for ‘working across the organization’ • Many agencies were either “developing” or “progressing” in their eBusiness capabilities with availability of resources cited as the greatest barrier to development efforts

7. Effective practices emerged in each of the following areas Merit Review Areas Description • Process for how an organization receives, processes, and streamlines activities associated with proposal intake, review, and decision-making Proposal Workflow Management Review Methodologies • Various practices an organization applies to review proposals and make funding decisions with limited resources Reviewer Community Management • Management practices that an organization applies to attract, recruit, and reward its reviewers, as well as how the organization maintains a relationship with its community

8. Many organizations employ different strategies to mitigate workload demands Proposal Workflow Management Review Methodologies Reviewer Community Management

9. Most organizations structure the review process to optimize the time of reviewers Proposal Workflow Management Review Methodologies Reviewer Community Management

10. Most organizations treat reviewers as “customers” throughout the process Proposal Workflow Management Review Methodologies Reviewer Community Management

11. Based on the findings in the effective practices research, NSF should consider the following in the scenario design • Broadening the use of Letters of Intent (LOIs) for determining program fit and begin planning and coordinating panels earlier in the process • More pro-actively coordinating program deadlines taking into account the needs of the research community • Developing piloting opportunities to broaden the use of temporary help during peak proposal times or outsourcing certain process functions • Incorporating process changes that reduce the degree of proposal budget rework that occurs throughout the Merit Review process • Compensating off-site reviews, broadening the concept of panel “terms of service”, and providing training for how best to perform a review • Developing strategies that “reach out” to a greater pool of reviewers (e.g., integrating databases for applicants, awardees, reviewers; enabling better remote access; accepting on-line reviewer applications, etc.)

12. Questions & Answers • For additional questions, please contact: • Abe Zwany, zwany_abe@bah.com, 703.902.5342 • Tim Koch, koch_tim@bah.com, 703.377.0389