GIS Mapping of the New York City Sewer System

1. GIS Mapping of the New York City Sewer System • Diagram • Geodatabase Design • Topic Outline Entering spatial/non-spatial data   -  Brief Overview of Baker's process (scanning, digitizing and scrubbing)  Linking spatial and non-spatial data  -  Indexing Application, Sewer Database design (database structure, Feature Class, table attribute, relationship) Data validation -  QA process (Automated and detailed feature inspection) . 

2. Entering spatial data  • Types of DEP source Document • As - built • Contract • Interceptor Plans • Index Maps • Emergency Contracts Drawings • Infiltration and Inflow Maps (I/I maps) • Connection Book • Field Survey • Types of Non - DEP source document • Department of Transportation • MTA • Bridge and Port Authority • NYC Orthophotos • DCPLION • NYC LANDBASE • GEOSUPPORT • Sanbon - Property Maps

3. Methods of Data Inputs • Scanning The process of translating hardcopy media into digital form that can be recognized by a computer. Create permanent records. • Digitizing The method of data capture that involves the conversion of data in hardcopy or raster forms into Digital vector formats. Digitizing produces features (pipes and manholes) used in a GIS. * Features are digitized relative to the NYCMAP with 1 foot horizontal accuracy * Precision : Offset value (distance btw feature and the curb line). Pipe Length Scanning differs from digitizing in that the entire pages of data or map sheet are captured all at once • Geo-referencing The process of defining the position of a geographic object relative to a standard reference. Images with spatial data (I/I maps) were geo-referenced. • Field Survey

4. Entering non-spatial data • ScrubbingThe act of cleaning a surface by rubbing it with a brush and soap and water  A meticulous process of reviewing and highlighting relevant attribute information for inclusion in the database. * Cleaned for inconsistency * Information retrieved are used to populate and link to their respective feature. * Standardization of domain and ranges (spatial and attribute)

5. Linking spatial and non-spatial data  • Indexing Creating a database of information for easy search and retrieval Indexing aid in filtering through a huge amount of information. * Document Indexing A procedure that associates a source document with its respective street centerline * Feature Indexing A procedure that associates a source document with its respective feature or attribute Where multiply source have been linked the most recent document acts as the default Source. Document are stored using a directories or folders and subfolders by their borough location. Relationships are established within the database. Relationship require ID.

6. Quality Assurance Procedures QA Workflow The QA process consists of several phases that are defined in the procedures below: • Check of digital data delivery and hard copy supporting documents • Automated database checks using QA software • Context check of plots and Engineering Review within the delivery area. • Feature inspection based on a randomly selected sample set using ANSI standards

7. 4. Conduct Sample Feature 2. Conduct Automated Checks 1. Check Data Delivery Package Workflow 3. Conduct Context Check & Engineering Review YES YES Accept? Accept? NO NO YES Delivery Complete? Accept? YES NO NO 5. Accept delivery 6. Reject delivery 7. Conversion Issue Resolution and Communication Figure 1: QA Flow Chart

8. Quality Assurance Data Validation Techniques The QA processensuresthat database design specifications are adhered to during conversion. The automated routine runs checks for data completeness, spatial inaccuracies, context and attribution errors, scheme and data structure and connectivity error. The process is divided into three broad groupings – Automated Checks, Context Checks/ Engineering Review and Feature Inspections Checks. The compliance level for automated checks is 100% while the compliance level for feature inspection is 95%. Automated Checks Automated checks consists of three types of data evaluations—object design, geometry, and connectivity checks. Table 1 summarizes the Automated Validation Checks. The Automated application runs in batch mode with pre-configured tables controlling the parameters. Most feature characteristics and attributes can be checked through automated routines.

9. Object Design • Review all features to ensure that all attribute values match domain tables • Review all features to ensure that all mandatory values are populated and only permissible null values are populated in the database • All unique values such as Facility ID are in fact unique both in a single delivery, as well as in the entire database • Compare the attributes with the various area polygons such as Grid_no, etc., to determine if the attributes are correctly populated • Check facilities with assigned linked annotation to assure they have at least one annotation feature linked to them. Geometry test • Review all junction features with elevation to ensure that the To and From attribution are correctly populated per pipeline • Review all field-surveyed points in the Geodatabase located in the x,y coordinates captured in the field • Review all features to identify any features that occupy the same x, y coordinate (i.e. duplication of geometry). Connectivity Test. The connectivity tests will: • Validate that all features meet the topology business rules (i.e., identify all floating junction features and/or edge features that are not terminated with a coded junction feature). • Conduct an audit and generate a report of all junction features and list features that have 0, 1, 2, and >2 edges connected to them. • Per each junction type, generate a list of features that have 0, 1, 2, or > 2 edges connected providing the ability to individually review each feature.

10. These tests are critical for sewer dataset because invalid values can cause symbology and analysis errors..

12. Context Checks • Check Plots • Reviews sewer configuration by comparing the geo-referenced I/I maps and digitized sewers. This check is to make certain that no significant segments of the sewer system are missing. Connectivity and flow tracing will support this step. Other step include sewer length verification. • Engineering Plots • Baker Delivery Plots (Engineering Review) - performed by a team of DEP Engineers/Staff. The check results in identification of configuration, attribute, and annotation errors. Valid errors are entered in the QA application along with other recorded errors Infiltration and Inflow Maps (I/I Maps): Infiltration and Inflow maps (I/I) were originally produced from as-built drawings.  They cover 14 drainage areas within New York City .  The I/I maps show the following sewer information:  sewer type, size, shape, invert elevations, flow direction, and year of as-built.  Because they have not been maintained, they are considered incomplete, and are generally inaccurate.  They should be used for reference only as a secondary source to be used for an understanding of the sewer system within an area.

13. Sample Feature Inspection This process involves a detailed inspection on a selected set of feature. The selection is chosen randomly and based on statistical principals that establish an overall confidence on the quality of the data delivered. The ANSI test consists of four components: • Calculate batch size, • Select general inspection level, • Calculate sample size and make a random selection from the batch size • Summing the errors for acceptance/rejection of the batch. Table 2 shows how the ANSI standard (American National Standard Institute) is used in the Sewer Project.

14. Table 2: ANSI Sample Size Code Letters14

15. Table 3. Single Sampling Plans for Normal Inspection

16. Table 4: ANSI Interpolated Number of Errors Allowed per Sample Size for 95% Acceptance and Rejection Criteria

17. Table 5:Feature Count and Sample Size

18. Results of Random Sample Check

19. A B C D E F Attributes Number of Average Attribute to be features in Total attributes weight checked batch attributes per feature (1/average) B * C D/C 1/E Sewer/Force Main ss_pressurizedmain 18 4 72 ss_gravitymain 21 3456 72576 Group G 3460 72648 21.00 0.05 Manholes ss_manhole 16 2785 44560 Group M 2785 44560 16.00 0.06 Other Point Features generic_node 2 292 584 pump_station 10 3 30 ss_detention_system 9 1 9 ss_dischargepoint 17 12 204 ss_sewer_structure 8 38 304 Group P 346 1131 3.27 0.31 Table 6: Computation of Attribute Weights

20. Data validation -  QA process Context Check Baker Delivery Plots Feature Inspection Customized Tool for inspection Feature Dialog Attribute Dialog Sample Tools