Spatial Analysis in Government Lecture 2- Data & Information

1. Spatial Analysis in GovernmentLecture 2- Data & Information By Guy Thigpen, M.Phil., MUSA

2. Purpose of Lecture Review the Data and Information contained in GEODB2 Explore an example (ULRS) of the information available in the data warehouse Examine the data structure of a parcel layer

3. Frameworks of Understanding Geography Systems Thinking Spatial Analysis Cartography

4. Philadelphia’s Data Warehouse Data and Information 282 feature files (tables, points, etc) 20 departments provide information 26 geodata folders with additional feature files

5. GIS DATA • METADATA • Documentation of GIS datasets. • Contains information such as, • the coordinate system, • when the data was created, • contact of who created it • definitions for any of the • code attribute data.

6. ArcGIS Services ESRI Arc GIS Server 10.1 • CachingOne of the main characteristics that makes the server very capable of publishing maps over web is Caching. Additionally, we have more control over how the cache gets built and updated. Caching particular areas on the map and On Demand caching are the other main features. • SecurityManagement of services with ArcGIS Server's role based security has been a major advantage. Also, the product supports token based security for web services and applications. • ImageryArcGIS Server 10.1's Imagery capabilities enable us to handle large amount of GIS data and provides access to it enterprise wide. It also allows processing and analysis of images on the client side. Some of the services that we provide include, City Aerial Photography, City Hybrid Map, Parcel Layer, Building Footprint Layer, Septa Lines etc. All these services can be overlaid upon each other according to the user's choice in any web application or desktop around the city. 

7. Pictometry Pictometry is the leading provider in digital, geo-referenced, oblique aerial imagery and innovative geospatial technologies. ORTHO Shot straight down and oriented North-up. OBLIQUE shot at a 40~45 degree angle. Individual image is available from North, South, East or West.

8. SPECIFICATION: RESOLUTION:Pictometry offers two levels of resolution, “community” level and “neighborhood” level. The average resolution of oblique imagery at community level (shot from around 4,500 ft) is 1-foot per pixel, and at neighborhood level (shot from around 3,500 ft) is 4-inch per pixel. DATE/ TIME STAMP : Each image is date and time stamped with the time of capture as reported by the time information transmitted from the Global Positioning System. CAPTURE AREA: The oblique imagery is captured for the City of Philadelphia, including • center city area, • the entire Philadelphia International Airport, which is located in Delaware County, and • the Wyndmoor portion of Montgomery County. With Pictometry Electronic Field Study (EFS) software, licensed users are allowed to view and work with the aerial images contained in the Pictometry Image Library

9. How to make a connection to GEODB2? • In ArcCatalog: The easiest way to create a connection to the central data warehouse, is by using ArcCatalog and connect to the ArcSDE application server (aka. Geodb2). Geodb2 will broker all your operations to the database. 2) Connecting with Oracle Client: The central warehouse is in essence an Oracle database, and for power users out there who perform more complex operations and queries, you can connect to the warehouse directly using any Oracle Client . GIS Spatial Data Sharing and Distribution • In general the staff and departments who have been in the forefront of the development of the City’s spatial data support the sharing of this data within the City and to the public where appropriate and where the City’s costs to provide this data access can be reimbursed.  • The interns at the Philadelphia redevelopment Authority are also provided access to the database.

10. Publically Available Data from GEODB2@http://maps11.phila.gov/EnterpriseGIS/Downloads.aspx

11. Philly GIS Forum • Bring together the GIS users in all the departments of City of Philadelphia. • The topics discussed here are expected to get attention from the concerned users who may make posts pertaining to the discussion. • Threads can serve as future reference for various issues should the need arise. Group structure with restrictions: • Threads in the forum pertaining to certain specific topics will be having access limitations; consequently, the forum users will be divided into groups according to the topics that are discussed and these groups will be having access those threads. One user can be present in multiple groups depending on the topics he is engaged in. Another one will be the read only restriction for certain threads and for certain users. • Solutions can be found soon without numerous email tags and phone call exchanges as the problems are viewed by the community. • Betterment in speed, resource utilization and problem-solving of city-wide GIS analysis and development.

12. Data and Information Process Regulation Distribution Medium Compliance Standard Data Processing Resources Committed Primary Data Capture

13. Data and Information The foundation of an analysis include DATA and INFORMATION. The City of Philadelphia has solid foundation of data and information. W = Wisdom U = Understanding K = Knowledge I = Information(URLS parcel layer and batch geocoder) D = Data(various shapefiles geospatially located)

14. S I P O C Inputs Processes Outputs Suppliers Customers DATAPROCESSING: Data is uploaded, stored and collated to be available as data or information Public Clients City Agencies DATA: Programs and projects produce data PLATFORM: Open Data Philly PASDA phila.gov/map COMPLIANCESTANDARD: Programs/projects are required or encouraged by federal, state, and local sources to collect data carried out by contractors and city agencies

15. Data and Information Relates to Analysis Wisdom Understanding Parcel Polygons Knowledge Information Basic Analysis Data

16. ULRS: Information in GEODB2 A simple task – displaying an address spatially What do we know about an address? • What are the sources for that address? • (process of standardization) • What parcel from the Department of Records does this address belongs too? • (process of geocoding)

17. Address standardization Incoming addresses are scrubbed to one or more ADDRESS_IDs • An ADDRESS_ID is any unique combination of • STREET_NUMBER • STREET_NUMBER_SUFFIX • STREET_PREFIX • STREET_NAME • STREET_TYPE • STREET_SUFFIX • (~ similar to LOC field in REAL_ESTATE table) • STREET_CENTERLINE + any other street known by OPA informs the “valid” streets • !!! ADDRESS_ID is an artifact of the ULRS !!!

18. Parcel geocoding For each ADDRESS_ID, assigned to active DOR parcel • 2 methods • by direct reference to DOR: if the address is in the parcel layer • by estimation: if the address is not in the parcel layer, ULRS will perform “educated guess” !! Not every ADDRESS_ID can be associated with a parcel !!

19. Relations in ULRS Process of geocoding Process of address standardization SPATIAL NON-SPATIAL

21. ULRS access? Exposed through various means • Webservices + their clients • Ready-made applications: AddressExtractor, AddressExplorer • bulk data reports in the “GIS_ULRS_DATA” schema

22. Web Services • = developer technology • Two flavors • CityMaps Webservices • “generic GIS” functionality: • relies on ULRS for only two pieces: the address cleanup logic & parcel-based geocoding • ULRS WebServices • Main window in the ULRS. • Allows you to extract any information that is stored in the ULRS • Provides simple read access to other db systems.

23. Web Services • Who uses the webservices? • Around 25 clients from various departments: lni, police, revenue, dot, brt, ohcd, dhs, water, sherrif’s office,… • 2,000,000+ lookups against the web services per month • Supports web sites + batch processing • Majority of ULRS clients do not rely on “geocoding logic” • mainly to resolve address input • switchboard to retrieve account information based on address (!!! after migration to ArcGIS Server, some functionality was lost!!! )

24. Existing Applications • Address Explorer • Replaces the ULRS Explorer – which was discontinued after June 30, 2009. • Why the rename? • address is central – not the parcel, not the BRT#,… • also, underline the fact that ULRS does not store any other information other than relationships between addresses & parcels on the one hand, and addresses & accounts on the other hand. (nothing more, nothing less) Mimic three basic ways of retrieving information from the web services - by address - by parcel # - by account # (topic key) ~corresponds to 3 ways of organizing the data

25. Existing Applications • Batch Geocoder/Address Extractor • Allows you to process address based data • Standardize; clean up an address(= most basic operation) • Validate: clean up an address & check if imported by ULRS • Geocode: clean up an address + find location • Extract land records (~~~ topic pane from Address Explorer) • Extract DOR parcel info: retrieve parcel identifiers by address

26. Data Reports • bulk delivery of ulrs data under GIS_ULRS_DATA on geodb2 (on monthly basis) • Replaces ulrs publisher: low usage + data not always meaningful • GOALS • organize data in meaningful way • secure core applications • flexible schema • data added upon request: make sure that every report actually has a user • not recommended for application use – but not enforced either. • SO FAR: • ALIAS_ADDRESS • MASTER_ADDRESS • MASTER_ADDRESS_ALL • MASTER_ADDRESS_BRT_PICT • PARCEL_ULRS • BRTREALESTATE_DORPARCEL

27. Exploring DATA: how it is generated This Square

28. Legal Description and Survey Plan Consistency - some legal descriptions do not indicate streets and just use bearings where others do indicate the streets with the bearings.

29. Understanding Bearings Example- South 03 degrees 42 minutes 30 seconds East S03º42’30”E • 1: First it gives the rotation of the line with respect to the geographical north in degrees, minutes and seconds. It is in the middle part between the directions. • 2: Second it gives the direction of the line from the starting point to the end point in one of the four predefined ways - South to East, South to West or North to East, North to West. These directions are at the start & end.

30. Reading Bearings From a Legal Description

31. Bearing Definition NE NW 15º 55º 60º 20º SW SE

32. How to Define Bearings N LO =N 55º43’59” W OL =S 55º43’59” E bL =? Lb =? gL =? Lg =? b O 55º43’59” 15º21’45” L W E 20º46’12” g 60º10’25” W S

33. How to Check Bearing of Parcel Each Segment should have bearing Angles in Bearings are never greater than 90º Bearings are referenced from North or South Four predefined ways - South to East, South to West North to East, North to West A Parcel Should have Bearings either in a clockwise or Anticlockwise Direction Minutes & Seconds should not be greater than 60º bL = S 15º21’45” W Lb = N 15º21’45” E gL = N 60º10’25” W Lg = S 60º10’25” E

34. Example: Parcel 61B Missing Bearing Either – 74.417’ Or – 74’5”

35. Two Bearing Segments Missing from Legal Description

36. Missing Segments Assumed At Right Angle Bearing To Be Verified

37. 2nd Deed

38. 2nd Shape in identified parcel

39. Final Remainder This Square

40. Remainder Legal Description • ALL THAT CERTAIN lot or piece of ground Situate in the 6th Ward of the City of Philadelphia and described as follows: • Beginning at a point of intersection of the Southeasterly side of 46th Street ( ) and the Southeasterly side of Fairmount Avenue (60 feet wide); • Thence extending N86°17'30"E, along the Southeasterly side of Fairmount Avenue a distance of 243.417 feet; • Thence extending S03°42'30"E, along a line at right angles to said Fairmount Avenue a distance of 249.000 feet; • Thence extending S86°17'30"W, along a line parallel with said Fairmount Avenue, a distance of 243.417 feet to a point on the Southeasterly side of said 46th Street; • Thence extending N03°42'30"W, along the said Southeasterly side of said 46th Street, a distance of 249.000 feet to the point of beginning. • Containing 60,610.83 square feet or 1.3914 acres, more or less. • BEING KNOWN AS:?

41. The resulting shape in GIS layer This Square

42. Next week’s Topic: The Front Door We will discuss the challenges and analysis that led to the implementation of the ‘Front Door’