1 / 20

Metrics Baseline Greg Hunolt, SGT ES-DSWG / MPAR-WG October 21-23, 2008

Metrics Baseline Greg Hunolt, SGT ES-DSWG / MPAR-WG October 21-23, 2008. Metrics Mission Statement.

ivie
Download Presentation

Metrics Baseline Greg Hunolt, SGT ES-DSWG / MPAR-WG October 21-23, 2008

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Metrics BaselineGreg Hunolt, SGTES-DSWG / MPAR-WGOctober 21-23, 2008

  2. Metrics Mission Statement To measure the success of each project in meeting its stated goals and objectives, to show the role and contribution of each project to the NASA science, application, and education programs, and to enable an overall assessment of the success of programs such as REASoN / ACCESS / MEaSUREs and their contribution to NASA’s goals.

  3. Metrics Overview • Programmatic Role Metrics: Characterize the role of the project within the NASA science, applications, and/or education programs by indicating program areas the project supports. • Products and Services Metrics: Measure the number of types of products and/or services provided by a project, with data volumes as applicable. • Common Metrics: Reported by most if not all projects, will be overall measures with sufficient cross-project commonality to allow assessment of the REASoN / ACCESS (etc.) program(s) as a whole, and will not be used as comparative measures of project performance. • Project-Specific Metrics: Projects may add one or more project-specific metrics, defined by each Project as best measures of its performance against its objectives. • Reporting: Both common metrics and project-specific metrics will be reported monthly unless otherwise agreed between a project and its study manager. • Impact Metrics: Specific success stories that provide an example (s) of how the project’s products / services have directly benefited a user , organization, or activity it supports.

  4. Common Metric #1, Distinct Users • Purpose:To measure the size of the activity’s user community, to be assessed in the context of its NASA program role. • Website Question:Please enter the count of individuals who, by any means, request and receive or in some other way use products, services and/or other information during the reporting period.

  5. Common Metric #2, Distinct Users by Class • Purpose:To measure the types of users served by the activity, to be assessed in the context of its NASA program role. • Website Question:Please enter the number of users who obtain products and services from your project by the following classes:

  6. Common Metric #4, Product Types • Purpose:The count of product types produced is a useful measure because of the effort by the activity required to develop and support each of its product types. A REASoN’s values for this metric are to be assessed in the context of its NASA program role. • Website Question:Please enter the number of product types made available to users during the reporting period.

  7. Common Metric #3, Products Delivered • Purpose: To measure, in conjunction with items 4, 5, and 6, the data and information produced and distributed by the activity, to be assessed in the context of its NASA program role. A particular set of values for these metrics might be much smaller for one activity than another activity, but in each case could represent excellent performance, given the particular role of each activity. The count of products delivered is a useful measure given the user oriented definition of a ‘product’ that is independent of how the product is constituted or how large it is. • Website Question:Please enter the number of products provided to users during the reporting period.

  8. Common Metric #12, Service Types • Purpose: The count of service types produced is a useful measure because of the effort by the activity required to develop and support each of its service types. A REASoN’s values for this metric are to be assessed in the context of its NASA program role. • Website Question:Please enter the number of service types made available to users during the reporting period. This metric is experimental, and may be altered by request from NASA HQ.

  9. Common Metric #11, Services Provided • Purpose: To measure, in conjunction with item 11the services performed by the activity, to be assessed in the context of its ESE role. A particular set of values for these metrics might be much smaller for one activity than another activity, but in each case could represent excellent performance, given the particular ESE role of each activity. • Website Question: Please enter the number of services provided to users during the reporting period. This metric is experimental, and may be altered by request from NASA HQ.

  10. Common Metric #5, Volume of Data Distributed • Purpose: The volume distributed is a useful output measure but one which depends heavily on the particular types of data an activity produces and distributes and must be assessed in the context of the activity’s role and data it works with. See note in metric 3. • Website Question: Please enter the volume of data and/or data and/or information provided as web downloads or hard media or otherwise distributed to users during the reporting period (in MB, GB or TB as appropriate, to three significant digits precision, e.g. "10.2 GB" as opposed to "10,186 MB"). If data is available but none was distributed then enter "0" in the entry box and explain in the comment box. If this Project has none available, then enter "na" in the entry box and 'not applicable' in the comment box.

  11. Common Metric #6, Data Volume Available • Purpose: The cumulative volume available for users provides a measure of the total resource for users that the activity creates. See note in metric 4. • Website Question: Please enter the total cumulative volume, as of the end of the reporting period, of data and held by the project and available to researchers and other users (MB, GB or TB to three significant digits). This number can include data that are not on-line but are available through other means. If paper products, enter as pages (pp) or publications (pb). If this Project has no data available, then enter "na" in the entry box and 'not applicable' in the comment box.

  12. Programmatic Metric #8, Support for Science Focus Areas • Baseline Definition:The REASoN projects will include a quantitative summary of the data products supporting one or more of NASA’s science focus areas, and report any changes at the next monthly metrics submission. The focus areas are: weather, climate change and variability, atmospheric composition, water and energy cycle, Earth surface and interior, and carbon cycle and ecosystems. • Purpose:To enable the ESE program office to determine which NASA Science Mission Directorate's Science Focus Areas are supported by the activity, and to assess how the data products provided by the activity relate to that support. • Website Question:Please list any products or services that support NASA Science Mission Directorate's Science Focus Areas and how many users were so provided within the last reporting period, and include which of the NASA categories they most correctly fit (may be multiple). This category is subject to definition between the REASoN management and their NASA program manager. • Categories from NASA Science Mission Directorate's Science Focus Areas: • 8.1 Weather, 8.2 Climate change and variability, 8.3 Atmospheric composition, 8.4 Water and energy cycle, 8.5 Earth surface and interior, 8.6 Carbon cycle and ecosystems, 8.7 Other- Please explain.

  13. Programmatic Metric #9, Support for Applications Areas • Baseline Definition:The REASoN projects will include a quantitative summary of the data products supporting one or more of NASA’s Applications, and report any changes at the next monthly metrics submission. The 12 applications areas are: agricultural efficiency, air quality, aviation safety, carbon management, coastal management, ecosystems, disaster preparedness, energy forecasting, homeland security, invasive species, public health, and water management. • Purpose:To enable the ESE program office to determine which NASA Science Mission Directorate's Applications of National Importance are supported by the activity, and to assess how the data products provided by the activity relate to that support. • Website Question:Please list any products or services that support NASA Science Mission Directorate's Applications of National Importance and how many users were so provided within the last reporting period, and include which of the NASA categories they most correctly fit (may be multiple). This category is subject to definition between the REASoN management and their NASA program manager. • Categories from NASA Science Mission Directorate's Application of National Importance: • 9.1 Agricultural efficiency, 9.2 Air quality, 9.3 Aviation safety, 9.4 Carbon management, 9.5 Coastal management, 9.6 Ecological forecasting, 9.7 Disaster management, 9.8 Energy management, 9.9 Homeland security, 9.10 Invasive species, 9.11 Public health, 9.12 Water management, 9.13 Other - Please explain.

  14. Programmatic Metric #10, Support for Education Initiatives • Baseline Definition:In partnership with the Study Manager the REASoN project will submit data pertaining to the adoption and use of educational products by noted audience categories (to be determined by project and study manager). These groups can include higher education, K-12, museums, informal education, and others as appropriate. • Purpose:To enable the ESE program office to assess support provided by the activity to NASA Science Mission Directorate's education initiatives, by indicating use by education user groups of the activity’s products and services. • Website Question:Please list any products or services provided to educators within the last reporting period, and include which of the NASA education categories they most correctly fit (may be multiple). This category is subject to definition between the REASoN management and their NASA program manager. • NASA Earth Science Education Plan: • 10.1 Elementary and Secondary Education, 10.2 Higher Education, 10.3 Underrepresented and Underserved, 10.4 e-Education, 10.5 Informal Education, 10.6 Other- Please explain.

  15. Impact Metrics – Success Stories Impact Metrics Include: • A one page narrative describing the success story, i.e. how the project’s products / services have directly benefited a user, organization, or activity the project supports. • The narrative might include: • The name of the product and/or service, in the title if possible. • The direct benefit of using the product or service, the impact, in a prominent manner; in the title if possible. • Any collaboration with local groups, NGO's, businesses, or federal agencies. • How working with NASA helped make this happen. • A JPEG image or graphic that provides a visual complement to the narrative description.

  16. Impact Metric: DISCOVER REASoN, PI: Frank Wentz May, 2008 Distributed Information Services: Climate/Ocean Products and Visualizations for Earth Research Provided daily near real-time data plots to Cirene international oceanographic cruise.

  17. Impact Metric: AMAPS ACCESS PI: Amy Braverman September, 2007 (w/ Large Picture) Aerosol Management and Processing System AMAPS provided MISR data to multi-agency AIRNow air quality project.

  18. Impact Metric: DIAL ACCESS PI: Bruce Carron January, 2007 w/ Large Picture Data and Information Layer: Enabling NASA Data-Rich User Software Application Development Provided graphics support to Science Education Resource Center (SERC) at Carleton College to demonstrate benefits of sharing graphic assets.

More Related