Department of Commerce App Challenge : Big Data Dashboards

1. Department of Commerce App Challenge: Big Data Dashboards Dr. Brand Niemann Director and Senior Enterprise Architect – Data Scientist Semantic Community http://semanticommunity.info/ AOL Government Blogger http://gov.aol.com/bloggers/brand-niemann/ April 27, 2012. Update April 30, 2012. http://semanticommunity.info/AOL_Government/Department_of_Commerce_App_Challenge

2. Dr. Brand Niemann • Former Senior Enterprise Architect and Data Scientist, US Environmental Protection Agency (1980-2010). • Current Husband, Father, and Grandfather Enjoying the Golden Years!

3. Semantic Community • Our Mantra is: Data Science Precedes the Use of SOA, Cloud, and Semantic Technologies! We use data science to help marketing and business development efforts. • Our Mission is like Googles: Organize the world’s information and make it universally accessible and useful. • Our Method is like Be Informed 4: Architectural Diagrams and Questions and Answers are not enough, you need Dynamic Case Management! • Our Sound Byte: It is not just where you put your data (cloud), but how you put it there! • Our Work: Semantically enhancing your data and writing data science stories about it.

4. Introduction • I heard about this several months ago, but put it off until yesterday. I finished it today because I am a very good Data Scientist! • Well I almost finished it. I need the Patent data in a format that I can more readily work with and I am in communication with the USPTO about that. • I create Knowledge Bases about my Data Science work so others can follow what I do and even reproduce it themselves. My apps also work on mobile devices like iPads. • My goal was, and still is, to create a set of multiple interactive dashboards of DoC data like they have for Foreign Trade.

5. Data Science Knowledge Base http://semanticommunity.info/AOL_Government/Department_of_Commerce_App_Challenge

6. Data Science Spreadsheet http://semanticommunity.info/@api/deki/files/17946/=DoCApp.xlsx

7. Spotfire Dashboards • U.S. Census Bureau Geographic Names Information System • U.S. International Trade in Goods and Services • Data.Gov Data Catalog for US Department of Commerce • U.S. Bureau of Economic Analysis • U.S. Patent & Trademark Office

8. U.S. Census Bureau Geographic Names Information System Web Player

9. U.S. International Trade inGoods and Services Web Player

10. Data.Gov Data Catalog for US Department of Commerce Web Player

11. U.S. Bureau of Economic Analysis Web Player

12. U.S. Patent & Trademark Office • Methodology: • Overview: Apply Gall's Law and start with the end in mind (Mashups and Decision Support) and work out the details in a simple and small content example for my next AOL Government Story! Give everything a well-defined URL for a semantically enhanced index in a Dashboard (see next slide). • 1. Follow Gall's Law which says: "A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: a complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a simple system." - John Gall, systems theorist • 2. Copy to MindTouch and add structure to the Web Pages • See http://semanticommunity.info/AOL_Government/Department_of_Commerce_App_Challenge/DOC_USPTO_Apps_for_Innovation • 3. Look at one ZIP file under each section and subsection to see what it contains and how to use it in MindTouch (in process) • See http://semanticommunity.info/AOL_Government/Department_of_Commerce_App_Challenge/DOC_USPTO_Apps_for_Innovation/Electronic_Data_Products

13. U.S. Patent & Trademark Office Web Player

14. MindTouchDoC USPTO Apps for Innovation http://semanticommunity.info/AOL_Government/Department_of_Commerce_App_Challenge/DOC_USPTO_Apps_for_Innovation

15. MindTouchElectronic Data Products http://semanticommunity.info/AOL_Government/Department_of_Commerce_App_Challenge/DOC_USPTO_Apps_for_Innovation/Electronic_Data_Products

16. Work Plan in Process • Mash-Ups: • Combine USPTO applicant/inventor information with other USPTO datasets (e.g., with USPTO assignments (ownership) data): • Google or USPTO Daily and USPTO Retro • Combine USPTO patent grants and patent application publications with other DOC data (e.g., Census or Economic data) • Innovative Ideas: • Homogenize the patent grant bibliographic text data (i.e., make it all the same format). • Same for the patent application publication bibliographic data. • Capture patent grant bibliographic text data from 1790 to 1975 using the image data. • Build a text searchable database (updated weekly) that includes both of the datasets discussed in the Webinar. Search queries can be saved. Result sets can be saved/extracted/tailored. • Build a text searchable database (updated weekly) that includes subsets of both of the datasets discussed in the Webinar. (e.g., Green Technology related). • Same ideas as above, but use full-text (75 MB/104 MB per week) or full-text with embedded images (1.4 GB/1.5GB per week): http://www.google.com/googlebooks/uspto-patents.html Source: http://semanticommunity.info/AOL_Government/Department_of_Commerce_App_Challenge/DOC_USPTO_Apps_for_Innovation#Innovative_Ideas

17. More Questions For Todd Park About Big Data http://gov.aol.com/2012/04/25/more-questions-for-todd-park-about-big-data/

18. Conclusions and Recommendations • A Data Science approach to the App Challenge provided examples for improvements in data dissemination and visualization. • Most of the data sets are “big data” when it comes to the app developer community working on simple mobile apps using smaller data sets. • The Patent data dissemination offers the most challenge for improvement and opportunity for creative piloting using a Data Science approach. For details see: http://semanticommunity.info/AOL_Government/Department_of_Commerce_App_Challenge#Submission