1 / 22

Overview

USDA Open Data / Big Data Presentation for the Xcelerate Solutions Federal Big Data Working Group Presented by the Office of the Chief Information Officer March 16, 2015. Overview. Open Data Implementation of OMB’s M-13-13- Open Data Policy USDA Last Quarter’s Accomplishments

vlove
Download Presentation

Overview

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. USDA Open Data / Big DataPresentationfor the Xcelerate Solutions Federal Big Data Working GroupPresented by the Office of the Chief Information OfficerMarch 16, 2015

  2. Overview • Open Data • Implementation of OMB’s M-13-13- Open Data Policy • USDA Last Quarter’s Accomplishments • Open Data Projects • Open Data Success Stories • Data Quality • Open Data Summary • Big Data • Intersection of Open and Big Data • Why Is It Important To Agriculture • USDA Big Data Projects • Food Resilience Theme of the Climate Data Toolkit Initiative • Using Administrative Data for Statistical Purposes • Big Data Summary

  3. Open Data 3

  4. USDA Implementation of M-13-13 • Developed 3-Step M-13-13 Implementation Strategy • Dataset Collection/Publishing • Created and maintained an Enterprise Data Inventory (EDI) • Submit EDI to OMB only (not to public) • Maintain a Public Data Listing (PDL) • Published datasets to PDL to USDA.gov/data and Data.gov • Developed and documented the USDA Data Publishing Process • Develop Dataset Prioritization, Privacy/Legal Review and IT Security Review Processes • Developed Customer Feedback and Engagement Plan • Conduct Internal/External Engagement Sessions Enterprise Data Inventory Public Data listing

  5. Last Quarter’s Accomplishments • November 2014 – Submitted 61 Datasets • February 2015 – Submitted Datasets • 14 New Datasets • 61 Updates • The Enterprise Data Inventory (EDI) consists of 651 datasets • 506 Release to the Public (Public Data Listing) • 145 Non-Public • December 2014 – Received Government/Industry Roundtable Final Results • Continue to support the USDA Science Council Committee on Statistics, we formed a cross Agency Administrative Data Categorization Working Group (ADCWG) • Determine ways to categorize USDA Data in a standardize manner • Determine how to collect, disseminate and access scientific statistical data for use within the Department

  6. Open Data Projects • Office of the Secretary (OSEC) Disaster Clearinghouse Project • Develop a one-stop application for disaster victims to find assistance before, during and after a disaster • Project sponsor - Dr. Ann Mills and Ms. Rebecca Shively of OSEC • Successfully completed requirements and preliminary design • User’s Test and Web-design will occur in March 2015 • Census Find it/Connect it Project • Develop a pilot using USDA, Census and NOAA data to determine the economic impact of a flood on Farmers, Ranchers, Producers and Consumers • Working with the National Institute Standards and Technology (NIST) to establish a common set of standards that can be shared across the federal government • Census One City, One Problem – Software Development Kit (SDK)Project • Unlocking the value of open data to promote economic growth, fuel innovation and entrepreneurship • Enabling cities and counties to use data to deliver tangible and transformative results for their communities • Select one city and one problem and solve it through the use of Census, USDA and NOAA’s open data • Census Center for Applied Technology (CAT) Lab • Real-Time Disaster Intelligence Map with Crowdsourcing • Pollinator Field Data Collection with Citizen Science • Farm Service Administration Common Land Unit (CLU) Hackathon for Open Application Programming Interface (APIs)

  7. . Open Data Success Stories • WhyHunger • The WhyHunger organization is using the Food Nutrition Service (FNS) data as one of the resources in providing information for the Summer Feeding Program for School Children, as well as healthy food sourcing in emergency food programs. Featuring program profiles, resources and strategies employed by food banks and agencies, including building relationships with farmers, creating voucher programs, teaching people how to grow their own food, and supporting an organizational culture that prioritizes nutrition.http://www.whyhunger.org/getInfo/showArticle?articleId=3674&utm_source=Clearinghouse+Connection+-+May+2014&utm_campaign=CCMay2014&utm_medium=email

  8. Open Data Success Stories . • Farmers Market Directory • The Open Data concept was used to develop applications such as the Farmers Market Directory, which gives developers and designers direct access to the wealth of farmer’s market information housed in the online database. • Rails-to-Trails • The Rails-to-Trails Project extensive mapping tool mixes data from the National Household Travel Survey and USDA’s Economic Research Service (ERS) to provide a better understand pedestrian and bicycle mode patterns in rural areas. 8

  9. Data Quality • When is the Data Good Enough for Release • Request for Forest Service for Trails Data • Should we consider a Disclaimer? • Data Quality Improvement (Data Cleansing) • Cleansing of Mata data – we’ve improved the quality of our datasets by 36% since our last submission. • Spelling • Broken links • Duplicate entries (etc) • Cleansing of the Actual Data to make it more open. We are adopting the five star maturity methodology • Stage 1 - PDF Files • Stage 2 – Excel Files • Stage 3 – CSV Files • Stage 4 – XML Files • Stage 5 - Data that can be link to other data • Data Management Policy • FY2015 – develop a policy to govern the data management of USDA data

  10. Open Data Summary • USDA Open Data Team has made tremendous progress • Data Collection • Process Development • Customer Feedback and Engagement • Challenges Facing the Open Data Team • Increasing Agencies and Administrative Offices participation in submitting datasets to be published (Internal and External) • Engaging the public to improve the quality and quantity of USDA datasets • Data Quality and Data Management • Dedicating resources to move the Open Data Initiative to the next level • Permanent Chief Data Officer • Data Scientist

  11. Big Data 11

  12. Intersection of Open and Big Data

  13. What Is Generating Big Data? Sensor Technology and Networks (measuring all kinds of data) Scientific Instruments (collecting all sorts of data) Mobile Devices (tracking all objects all the time)

  14. Value of AgBig Data ValueGeneratedbyAgBigData Making the link between • On-Farm Optimization • Data-based decision-making for many more decisions • Early problem identification for management response • Custom solutions to minimize inputs and maximize yields • Input Product Innovation for Unique Conditions • Biotech / seed research • Equipment R&D • Other input supplies • Market Linkage • Improves transparency and predictability of markets

  15. Big Data In Agriculture • Big data starts with precision ag, but is much larger than on-farm data alone • On-Farm Precision Ag (fertilizer, planting, crop protection) • 3rd Party Data (weather, satellite / aerial imagery, soil fertility, topographic data) • Research & Development (Academic / Land grant data, biotech and equipment company publications, software and analytics • Commodity Markets (local, regional, global)

  16. Importance of This Issue • Why is Big Data in agriculture emerging now? • Computer capacity has now expanded to process vast volumes of data • Why is Big Data an important issue? • Enables the promise of Precision Agriculture to be realized • Makes critical agronomic decisions on small areas within each field • What is the level of urgency? • Several major Ag companies have just launched or soon will launch Big Data

  17. Decision Support for Farmers EnablingFarmerstoMake40CriticalDecisions CriticalDecisionSets: Goal: Maximize Net Return Per Acre Productivity Tools: Pest-Control Factors • Focus on insect and weed control regimes Planting Factors • Focus on best field configuration, preparation and planting elements In-Season Decisions • Diseases, nutrient, etc. approaches based on in-field environment Seed Factors • Seed is lynchpin decision – key to establishing yield potential

  18. Farmer Profitability Strategy PioneerFarmer Profitability Strategy WholeFarm ProfitabilityServices CurrentServicesOffering Harvest / Evaluate • Field-by-fieldcropplans • Plantingmaps • Harvestmaps • Cropinsurance • Financialservices • Grainmarketing Plan Plant Monitor Tools todeliveradditional insights for advancedfarmer profitability.

  19. USDA Big Data Projects • Food Resilience of the Climate Data Toolkit Initiative • Led by the USDA OSEC and OSTP • Tool Kit Focuses on four types of Data; • Production: • How will crop yield and production of other food products be affected by changes in climate? • What crops and locations will be most vulnerable to changes in climate? • How will changes in climate affect the types of crops that can be successfully grown in a location? • Under what conditions (i.e., for what crops and in what locations) will indirect effects of climate change (e.g., pests, pathogens, invasive species, fire) overwhelm direct effects of increasing temperature and changes in precipitation? • Supply • What parts of the food supply chain will be most vulnerable to climate change? • How will processing, storage, and transportation need to be modified to continue to supply safe food products under a changing climate? • Nutrition • How will climate change impact food borne illness that includes chemical as well as microbial sources? • How will an increase in pesticide or veterinary drug use with climate change impact the contamination of food? • Security • How will climate change affect the nutrient content of staple foods? • How will the cost of a balanced diet be impacted under a changing climate? • What segment of the population will be most vulnerable to changes in food prices? • What countries are most vulnerable to climate change that affects food yield, quality, and availability? • Launch of the Food Resilience Theme – July 2014 19

  20. USDA Big Data Projects • Using Administrative Data for Statistical Purposes • Driven by Memo M-14-06 – Guidance For Providing and Using Administrative Data for Statistical Purposes • Led by USDA Science Council Committee on Statistics • Convenes monthly to discuss the progress on each Agency’s initiatives • Improving how agencies leverage existing and Federal data to facilitate agencies' programmatic work and enhance the value of those data to the American public • Facilitating the collection and publishing of USDA Agency administrative data to enable full, appropriate use of non-public administrative data for statistical purposes 20

  21. Big Data Summary • The Value of Investing in Open and Big Data: • Open and Big Data will create the next major technological “sea change” in agriculture • Will enable completely different ways to innovate and invent new business models • Enable the mass market to source, supply and consume it with better outcomes • Innovators and inventors can build stuff that matters and build stuff that works • Unlock more value in the existing data market and has the potential to create a new industry of businesses

  22. Questions Open Data Points-Of-Contact Joyce Hunter USDA Acting Chief Information Officer (CIO) and Deputy CIO for Policy and Planning 202-720-8833 joyce.hunter@ocio.usda.gov Bobby Jones USDA Senior Advisory to the Deputy CIO for Policy and Planning and Acting Chief Data Officer 202-690-4305 bobby.jones@ocio.usda.gov 22

More Related