Agenda

2. Agenda • Internet research overview • Set objectives • Determine a methodology • Identify sources • Collect data • Analyze data • Present data • Reach conclusions • Summary • Appendix: Data sources

3. Research Overview 3

4. Research Overview The Internet can be a valuable research tool, provided one has the skills to leverage its possibilities • Market research on the Internet is just like any other research project • The Internet offers the most amount of data in one place, but also gives us the most amount of useless data • Researchers must take care to “consider the source” and eliminate spurious data

5. Set Objectives 5

6. Set Objectives Since the Internet is so vast, it is imperative that research efforts have up-front, clear, limited objectives • Examples of bad objectives • Identify who is doing work at the IRS • In what capacity? • Over what duration? • Validate past performance for all bidders • How many citations? • How many bidders? • Review bidder qualifications • How many bidders? • In what capacity? • Determine commercial pricing • For what specifically? • Under what terms?

7. Set Objectives, cont. • Examples of good objectives • Assist management with strategic planning by providing changes, advances, and trends in technology and products • Help define requirements by determining whether commercial items are available • Identify standard commercial practices regarding warranties, financing, and maintenance • Evaluation objectives • Select and validate the best value contractor for a specific program • Reach a Determination of Responsibility on a prospective contractor • Conduct a risk assessment of future contractor performance • Establish and validate whether a proposed vendor has bid a fair and reasonable price • Evaluate competency of small business vendors • Review top five bidder qualifications, to include: verifying and analyzing financials, identifying merger and acquisition activity, identify recent wins

8. Determine Methodology 8

9. Determine Methodology Methodology describes the way in which a researcher intends to meet the stated objectives • Primary research is that which is solicited and conducted through primary means, e.g, telephone interviews or surveys • Difficult to obtain • Accurate • Secondary research is that which is obtained by seeking already-released data and analyzing it • Plenty of information already exists • Many risks • Lack of control over secondary data

10. Determine Methodology, cont. • Examples of primary data sources; POC information can be found on-line • Government customers • Company program managers or public relations staff • Reporters • Examples of secondary data sources • Government web sites • Government reports • Media • Company web sites • Trade associations • Financial reporting sites

11. Determine Methodology, cont. • What to ask primary sources • Quantitative data • Government customers: “On a scale of 1-5, how would you rate the vendor’s ability to stay on schedule?” • Company staff: “What is your forecasted revenue for this year?” • Qualitative data • Government customers: “How happy in general are you with Vendor X to date?” • Company staff: “How does your solution for seat management compare to other vendors’ solutions?” • Reporters: “When you wrote the recent article on the use of videoconferencing in government, what were your most valuable data sources?”

12. Determine Methodology, cont. • What to get from secondary sources • Quantitative data • Company financial information • Procurement data • Government budget figures • Qualitative data • Trends • Drivers • Barriers • Company strengths and weaknesses • Company contract histories

13. Identify Sources 13

14. Identify Sources On-line sources* exist for multiple dimensions of research. Known sources are supplemented with search engines. • Government market data • Office of Management and Budget • GSA’s Federal Procurement Data Center • Commerce Business Daily (CBD) • GSA and GWAC contracts • Most agencies publish annual strategic plans • General Accounting Office and Inspectors General • Trade publications • Library of Congress “THOMAS” legislative data * A list of recommended data sources appears in the Appendix

15. Identify Sources • Private sector market data - events and reports • FSI Outlook conference and networking breakfasts • AFCEA conferences • Electronic Industries Alliance (EIA) forecasts • Information Technology Association of America (ITAA) presentations • Financial data • Securities and Exchange Commission “EDGAR” filing data (10-Q and/or 10-K files) • Company annual reports • Mergers and acquisitions data

16. Identify Sources • Company past performance • Dun & Bradstreet Past Performance Evaluation (PPE) services • Supplier Evaluation Reports • Supplier Analysis Reports • Supplier Performance Review • Press releases announcing contract wins (company and government) • In general, past performance data are difficult to determine through secondary means • There are occasional press articles about poor contract performance • Sometimes we identify references that cite contract terminations

17. Identify Sources, cont. • Company strengths and weaknesses • Current job listings on contracts • Few delivery orders issued on a multiple award, indefinite delivery contract • Citations of won recompetitions • Congressional testimony • Memberships in professional organizations or associations • Bidders lists put out by procurement organizations • General Accounting Office oversees and maintains historical data on protests • Small Business Administration • Small and Disadvantaged Business Utilization (SADBU) offices

18. Identify Sources, cont. • Product and Services Information - government • GSA’s Federal Procurement Data Center reports contract spending actions by Product Service Code (PSC) and Standard Industrial Classification (SIC) code • PSCs and SICs cover products and services • Product and Services Information - contractor • GSA Schedule and some IDIQ contracts are online and provide downloadable contract documentation, including • Contractor teammates • Contractor services and products pricing

19. Collect Data 19

20. Collect Data Collecting data represents the true “research phase”. Do not become overly concerned with relevance during this phase. • Contact appropriate persons for primary research • Based on objectives, develop a list of relevant questions • Begin primary research early; call backs take time • Download relevant reports • Be careful of report versions • Pay attention to publication dates

21. Collect Data • View other data according to your research style • Cut and paste text into one electronic document and then edit that document online • Copy down all relevant URLs (http://www.xyz.gov) in one place and save all online research for a single session • Copy URLs into a word processing document • Save URLs as bookmarks • Cut and paste information directly into your research report all at once • Print out all pages in hard copy if that is the best method for you

22. Collect Data • Learn how and when to use search engines • Search engines can result in overwhelming amounts of data • Each search engine has a unique “language” that impacts accuracy • Know exactly what you’re looking for before you start. Narrower searches result in fewer hits and more relevant hits • Search engines are “dumb”; remember that you may have to restate something five times to cover all relevant citations • Mergers and acquisitions also affect your search results

23. Analyze Data 23

24. Analyze Data Because the Internet is infinite, the data we collect can seem infinite at first. Learn to “separate the wheat from the chaff” • Prioritize collected data • Ask yourself how significant each data set is to the overall research objectives and address them in rank order • “Official” documents and primary research are generally higher priority than documents acquired through other sources • Hierarchy varies from project-to-project, but typically will resemble: • Primary research • Government reports • Company literature or financials • Press releases • Published articles

25. Analyze Data, cont. • Dismiss sketchy information or conflicting data • If there is no solid way to validate the source, dismiss the data • Distinguish between objective and subjective information • It is usually necessary to caveat subjective data rather than state it as fact • Dismiss extraneous data (“fluff”), or that which does not match objectives • Stated objectives are important for the reader, but also help keep the researcher focused • “Fluff” contributes to overall length of document, but adds no value

26. Present Data 26

27. Present Data The look and feel of a presentation should vary according to its intended audience • Consider the reader and tailor the document to them • Full text report or executive summary only? • Management-level highlights supplemented with full text backup? • What can the researcher communicate about the data? • Trends and observations • Graphs and charts • Double-check that charts and graphs do not contradict each other

28. Reach Conclusions 28

29. Reach Conclusions The researcher needs to sum up findings, and determine if they are adequate and appropriate • Did the research accomplish all the stated objectives? What did the research show in the process? • Restate the report • A few paragraphs (written format) • 10-15 bullets (slide format) • Illustrate how the research benefits the recipient • Educates them on the topic • Validates their current thoughts or ideas • State the bottom line - what does it all mean?

30. Summary 30

31. Summary The Internet can be a valuable research tool, provided one has the skills to leverage its possibilities • Market research on the Internet is performed just like any other research project • Setting good, viable objectives will help keep the research focused • A methodology establishes a roadmap by which objectives can be met • Many sources exist; identify and prioritize the most valuable sources • Collect data in a logical manner • Analyze data that has been collected; identify holes • Present data so that the reader can see its value • Reach conclusions that restate the “bottom line” and validate that the research met stated objectives

32. Appendix: Data Sources 32

33. Data Sources • Search engines (comparison chart on following slide) • Altavista www.altavista.com • Excite www.excite.com • Google www.google.com • Goto www.goto.com • Hotbot www.hotbot.com • Northern Light www.northernlight.com • Yahoo www.yahoo.com

34. Data Sources, cont. Search engine feature comparison • Use your favorite search engine, but be aware of its query language and limitations • Supplement one engine with another if you don’t find what you are seeking

35. Data Sources, cont. • Government sites • Office of Management and Budget www.whitehouse.gov/OMB/ • Federal Procurement Data Center fpds.gsa.gov/Fpds/customer.htm • General Accounting Officewww.gao.gov • Agency Inspectors General www.ignet.gov • Library of Congress “THOMAS” thomas.loc.gov • GSA Office of Governmentwide Policywww.itpolicy.gsa.gov/ • National Partnership for Reinventing Governmentwww.npr.gov/ • Chief Information Officers Councilwww.cio.gov • SEC Edgarwww.sec.gov/edgarhp.htm • Small Business Administration www.sba.gov • Commerce Business Daily cbdnet.access.gpo.gov

36. Data Sources, cont. • Company past performance • Dun & Bradstreet www.dnb.com/purchase/pqualify.htm • Press releases various company sites • Private sector sources • FSI conferences and breakfasts www.fedsources.com • Federal Computer Weekwww.fcw.com • Government Computer Newswww.gcn.com • Washington Technologywww.washtech.com • Government Executivewww.govexec.com • Government Technologywww.govtech.net • AFCEA conference data www.afcea.org • EIA forecasts www.eia.org • ITAA presentations www.itaa.org/indexgov.htm • IMART www.imart.org/

37. Data Sources, cont. • GSA Contracts • Searchable GSA Schedules • eagu6.fss.gsa.gov/NetDynamics/NetDynamics40/ndNSAPI.nd/SchedELib2/pgSearch • GSA Advantage! Online Ordering • www.fss.gsa.gov/cgi-bin/advwel • Example: GSA Advantage! List of PC vendors • www.fss.gsa.gov/advvend.html#70 • Selected GWAC contracts • DOD DEIS II www.disa.mil/D4/diioss/deisiichar.html • HHS GWAC contracts nitaac.nih.gov/ • NASA ODIN www.odin.nasa.gov/homepage.html • Navy IT Umbrella Contracts www.it-umbrella.navy.mil/ • Transportation’s ITOP itop.dot.gov

38. Data Sources, cont. • Government procurement sites • Department of Agriculture • Procurement www.usda.gov/da/procure.html • Air Force • Acquisition Home Page www.safaq.hq.af.mil • Army • CECOM Acquisition Center www.monmouth.army.mil/cecom/ac/ac.html • Commerce • IT Acquisition www.itao.sao.noaa.gov • Defense • Defense Procurement www.acq.osd.mil/dp/ • DefenseLink www.defenselink.mil • DTIC Acquisition Information www.dtic.mil/hovlane/ • Defense Procurement Gateway progate.daps.mil/home/ • Education • Grants and Contracts gcs.ed.gov/

39. Data Sources, cont. • Government procurement sites, cont. • Energy • Office of Procurement www.pr.doe.gov • Environmental Protection Agency • Doing Business with EPA www.epa.gov/oam/ • Grants es.epa.gov/ncerqa/ • General Services Administration • Business/Finance www.gsa.gov/commerce.htm • Doing Business with GSA www.gsa.gov/bizwgsa • Health and Human Services • Acquisition Management www.os.dhhs.gov:80/progorg/oam/ • Housing and Urban Development • Contracting www.hud.gov/cts/ctshome.html • Interior • Electronic Acquisition www.ios.doi.gov/pam/ideahome.html

40. Data Sources, cont. • Government procurement sites, cont. • Justice • Justice Acquisition www.usdoj.gov/07business/index.html • Department of Labor • Grants and Contracts www.dol.gov/dol/oasam/public/grants/main.htm • National Aeronautics and Space Administration • Acquisition Internet Service procurement.nasa.gov/ • Navy • NavyOnLine www.ncts.navy.mil • ITEC Direct itec-direct.navy.mil/ • Postal Service • Acquisition www.usps.gov/business • Social Security Administration • Acquisition www.ssa.gov/oag/oag1.htm

41. Data Sources, cont. • Government procurement sites, cont. • Department of State • Procurement Executive www.statebuy.gov/home.htm • Department of Transportation • Procurement Information Office www.dot.gov/dotinfo/ost/m60/index.html • Agency Procurement Offices www.dot.gov/business/dotlinks.htm • Department of Treasury • Bureau of Engraving and Printing www.bep.treas.gov/procurement/ • Financial Management Service www.fms.treas.gov/procure/index.html • IRS procurement.irs.gov/ • Secret Service www.treas.gov/usss/proc/ • Customs www.customs.treas.gov/career/con_opps.htm • Department of Veterans Affairs • Acquisition and Materiel Mgmt www.va.gov/oa&mm/busopp.htm