Quality Assurance and Data Quality for Stormwater Programs

1. Quality Assurance and Data Quality for Stormwater Programs Z z z z z z ... 8th Annual EPA Region 6 MS4 Stormwater Conference Richard Smith INCOG June 27, 2006

2. Data Quality and Quality Assurance • “QA” is an essential component of creating and maintaining the usefulness of data and information. • QA addresses: • “Monitoring” data (e.g. field and lab measurement data) • Observation data (e.g. visual inspections) • Records of conversations, meetings, emails, letters, etc. • Data summarization and analyses (e.g. statistics, estimates) • Reports and data transfers (e.g. spreadsheets, letters) • In-house processes for communication and record-keeping • Storage, calibration and use of equipment and supplies “QA = CYA”

3. How Will Your Data Be Used ? • Permitting Authority – (Annual Report / DMRs / Phase II audits / P.A. pollution & compliance enforcement). • Other State and Federal Agencies – to fulfill their own data collection efforts for projects / databases. • City Councils / County Commissions – program updates, budgets, justify additional resources or changes to ordinances, etc. • Internal Staff Requests – information flow within city. • Citizens / Local Parties – public education, data requests. • Local Enforcement Actions – building your case. • Local Media – radio, newspaper, television. The Data Needs will dictate how you collect, store and report data.

4. Phase II Data Streaming DATA IN-FLOWS DATA OUT-FLOWS Lab & Field Data Information to Media Who, what, where, when, why, how Data for Court Enforcement Data Visual Observations Reports to Councils Data Management System Photographs Management Reports Reports to City Staff BMP Quantities Inspection Interviews Reports to Citizens Map Data Data to Agencies Data from Agencies ODEQ Annual Report Training Records Program Audits Complaint Tracking Budget items

5. Stormwater Program Manager CITY MANAGER PUBLIC WORKS DIR. CITY COUNCIL Many cities and counties struggle to define a management structure. CONSULTANTS LOCAL CITIZENS & BUSINESSES STORMWATER PROGRAM MANAGER AGENCIES & SUBSTATES VENDORS MS4 / FACILITY INSPECTIONS EDUCATION & TRAINING CONSTRUCTION INSPECTIONS DATA MANAGER & REPORTING

6. Most Phase IIs Need Help With... • Map data. . . . . . . . . . . .GIS, GPS, AutoCAD, paper. • QA methods. . . . . . .What’s required, most efficient. • Electronic data sets. . . . . . Tier II, RCRA, TRI, etc. • Field test kits. . . . . . . . . .D.O. pH, Cond., etc. • Other field gear. . . . . . . .Samplers, bottles, etc. • Data Management: • Software for supervision, records, reports • Forms for inspections, contact records. etc. • Data and monitoring Quality Assurance (QA) • Sampling and Lab services

7. Data Management Options • Consider Your Ability to: • Purchase • Learn • Use • Maintain

8. Why is QA Important ? • Many circumstances require verification of information: • Measurement data for court cases or enforcement. • Reports to City Councils, Commissions, citizens, contractors, consultants, agencies and in-house use. • Regulatory decision-making (e.g. violations of WQS). • Political fallout if your data is discredited. • Formal documentation of QA procedures is wise. • Don’t expect to have a fully developed QA program quickly – give yourself time to learn and experiment with what works best for your local needs.

9. To What Activities Should QA Apply ? • Technically, all activities pertaining to any type of data collection, analysis, manipulation and reporting.... • Determine your highest priorities first. For example: • Visual inspections / lite-monitoring of MS4 • Source tracking inspections • Construction site inspections • If private consultants and/or labs are used, be sure that they employ appropriate and adequate QA to their activities. • Make this part of your contract specifications, especially if their work is needed for enforcement or court evidence.

10. Even Simple Test Kits Need Attention • Expired chemicals - field test kits have chemicals that expire – keep records of refresh purchases. • Some kits require “calibration” – use fresh calibration standards and keep a log of calibration activities. • Some kits can measure “check samples” (of known analyte concentration) – perform checks frequently and keep records of all tests. • SOPs help ensure continuous proper use, and aid in training new employees. • Maintenance includes verifying equipment function and replacing defective parts before re-use of the kit. Detailed maintenance records should be kept.

11. Types of Field Instruments Comparator Test Kits These types of kits rely upon chemical reactions to indicate the presence and/or concentration of one specific parameter. Test Strips Titrations An SOP should be written for each type of kit. Colorimeters Multi-Test Kits No endorsement of products is intended.

12. Types of Field Instruments Test Pens These types of kits measure directly the concentration of one or more parameters. Single Parameter Meters Multi-Parameter Meters An SOP should be written for each device and procedure. Field Fluorometer Turbidimeter No endorsement of products is intended. Flow Meter

13. QA Concepts Precision is a measure of how close multiple readings are of the same sample. Accuracy is a measure of how close a reading is to the known value. Both are important concepts in assessing quality of data. Both are normally reported with laboratory data.

14. More QA Concepts Representativeness Does the sample represent what you intend? Completeness Do you have enough samples? Comparability Are sampling methods useable to others? Chain of Custody Legal documentation of “ownership” of samples during all phases of transport. Adequate information on each sample bottle to show ownership, collection information and intended analyses. Sample Labels

15. Types of QA Documentation to Consider • QAPP – QA Project Plan - how you conduct data gathering, analysis and reporting for your Phase II program. • DQOs – Data Quality Objectives – deciding in the beginning what types of data you will be collecting and, most important, how it will be used. • QMP – Quality Management Plan – down the road, you may consider this – describes your overall data management and communication system. • SOPs – Standard Operating Procedures – detailed instructions for how you perform specific data collection activities, including use, storage and care of measurement equipment.

16. What Is Absolutely Required ? • Each State’s General Permit should provide specific requirements. • In Oklahoma, OKR04 Part V lists a number of requirements for “Monitoring, Record Keeping and Reporting” • “Samples and measurements...shall be representative...” • “If lab analysis is conducted it must [use] 40 CFR Part 136” [methods] • “Monitoring records must include...[names, dates, times, place, methods, etc.]” • “Retain records of all monitoring information” (e.g. copies of reports, instrument charts, etc.)

17. Are Formal QA Documents Required ? • Your State’s General Permit will state which, if any, QA documents will be required (e.g. QAPP, DQOs, QMP and/or SOPs). • In Oklahoma, OKR04 does not specifically require any of these documents. • However, one must consider the “CYA” aspect of QA as well as what the minimum permit requirements are. • Particularly if your data are to be used in court, for local enforcement or for making regulatory decisions... • Or for that matter, just giving yourself credibility to your manager, peers, mayor and citizens.

18. How to Start a Formal QA Program • Managers, Mayors and Councilmen must learn the importance of good QA’d data, and how it will help them defend their programs. • Don’t try to do it all at once – expect changes as you go. • Decide what level of QA you need to apply to each type of data and activity. • Start with SOPs and a basic QAPP. • Seek guidance from experienced resources. • Be comfortable with what you create – stick to basics. • Amend as your Phase II program grows – QA is dynamic.

19. A Good Beginning QA Program • Start applying QA to “hard data” first – e.g., chemical data from field measurements, field test kits and lab data as well as visual observation data from visual inspections. • DQOs - Decide upon the level of data quality you want for the main types of data you will be collecting. Use of data. • SOPs – identify all routinely repeatable actions and write detailed step by step instructions and list supplies needed. • QAPP – prepare an umbrella QA plan for all activities requiring formal QA attention. • Lab and Service contracts – include in your contract (and QAPP) what technical requirements you want for lab services and from vendors and consultants that will be performing technical services (e.g., inspections).

20. EPA’s QA Resources • EPA guidance documents: • http://www.epa.gov/earth1r6/6wq/ecopro/watershd/monitrng/qappsprt/ • DQO Guidance = QA/G-4 • QAPP Guidance = QA/R-5 • QMP Guidance = QA/R-2 • SOP Guidance = QA/G-6 • These are not approvable by EPA unless they pertain to projects funded by EPA water quality grants. • EPA periodically offers QA training courses. • Contact your Permitting Authority and substates for QA advice.

21. Thank you for your infinite patience. Any Questions ?