GIS System Architecture and Design Some Assembly Required!

1. GISSystem Architecture and DesignSome Assembly Required! Parts of this lecture draw upon: Dave Peters, System Design and Configuration Strategies, ESRI User Conference 2002 For latest version, go to: Peters, D. System Design Strategies, ESRI White paper August 2005 @ http://www.esri.com/library/whitepapers/pdfs/sysdesig.pdf

2. The Cop Shop Your Service Environment Citizen Sam & Samantha at home Freda and Fernando in the field

3. ArcInfo ArcEditor ArcView ArcExplorer Browser Internet Your GIS Components Consistent interface Increasing capability Clients c:\ ArcGIS Workstation $ ArcMap ArcCatalog ArcToolbox ArcMap ArcCatalog ArcToolbox ArcMap ArcCatalog ArcToolbox ArcServer Services ArcEngine/ ArcObjects Application Development & Customization ArcIMS Services ArcSDE Services Source: ESRI with mods.

4. Your Info System Components LANs Wireless Firewalls Portables, Tablets, Palm/Pockets (Laptops/Tablets/PDA/Phones)

5. System architecture and design involves bringing these three componets together in a package which: minimizes cost maximizes performance and works reliably and securely “Assembly Required!”

6. GIS Characteristics • Detailed, high resolution graphic displays • Large data files • Intensive data analysis/computation Consequence? Need for: • Powerful workstations • Powerful servers for data retrieval and batch processing • High capacity networks These requirements differ significantly from standard IT environments with which your IT folks are most familiar.

7. Information System Components • Computer hardware: physical machinery, boxes, cables, connectors • software: instructions which make hardware perform as desired by user • Operating System to control the basic functions of the hardware and networks • Applications to provide users with desired results • Data and databasesto store information required by users • Networks to distribute information between different computers and users • orgware (Costis Toregas): people, procedures & organizational structures to make all of the above function • 85% of cost of IT? • $12,000 per annum to keep a PC user functioning. • like the iceberg under the water

8. Information System Organization • Functional: access to information - users view • Physical: computers as nodes on networks -IS view Access all the information required to carry out my job

9. The Computer: Hardware Components • central processor: • microprocessor (control unit and arithmetic/logic unit) • primary storage -RAM (main memory or simply memory) • secondary storage or simply storage • disks • magnetic • optical • tapes • Input/Output Devices (I/O) • Scanners, printers • communications devices/ network connections Storage -disks -tapes Central cpu Input devices Output devices (hardcopy) Main Memory Processor Networks

10. Issues: Performance bottlenecks • CPU performance • Memory (RAM) • I/O: to disk and/or to network • Network performance: speed and/or load Balanced system critical Issues: Capacity Planning • peak load versus standard load Issues: CPU Selection & configuration (see next slides) • Servers: database, geoprocessing, web, terminal • Client: Workstation, terminal client, browser client • Mobile: portable, palm, pocket (laptop, PDA, phone)

11. GP GP GP ArcSDE DB ArcSDE DB ArcSDE DB Web Web Web Terminal Server Terminal Server Terminal Server Types of servers Example GIS Server Configurations GP GP Geoprocessing ArcSDE Data DB Web Web Web Terminal (possible) Terminal Server Terminal Server One server, plus TS Two servers, plus TS Dedicated Servers Multiple Servers -load balancing -redundancy GP: geoprocessing DB: database

12. Example ArcIMS configurations relative to security Least secure One of many intermediate solutions Most secure

13. Client Configurations Web server Terminal Server Geoproc. Server Database Server Servers GIS Software runs on terminal server Can hand-off some work to GP server Data stays on servers Image to Client Low end PC with special TS software Low network bandwidth Full GIS functionality (Citrix is primary vendor) GISWeb Software runs on web server Can hand-off some work to GP server Data stays on servers Image to Client Low end PC with standard web browser Low network bandwidth Limited GIS functionality (depends on GIS Web software) GIS Software runs on PC Data to PC High end PC High network bandwidth Full GIS functionality LAN link Terminal Client PC Workstation Web Browser LAN link Or VPN* Web Link *VPN Virtual Private Network

14. Mobile Client Considerations Device type: Portable, tablet, or palm • How much capability is required • In-field editing, or info look up only? • Is Connectivity required? • Wireless data connections slow and expensive, relative to in-house • How keep databases in synch.?

15. Software Layers HARDWARE • Operating System: manages basic operations of the hardware (disk writing, communication links, etc) • Windows, UNIX, legacy (old mainframes still around) • Utilities: add additional functionality to OS • e.g. security and backup are critical functions • Data Base Software: handles and stores information to be processed . • Oracle, SQL-Server, IBM’s DB2 • Application Software: software to perform specific functions desired by user • E.g. ArcGIS, but it exists within environment of multiple other applications USER

16. finance/accounting: track dollars general ledger accounts payable/receivable purchasing budgeting funds management human resources : track people employee records payroll benefits job classification/career path physical assets (space/property): track things space inventory capital assets materials and supplies industry specific (private sector/university/city) sales/student recruiting/econ dev. order processing/ registration/utility billing manufacturing automation/ instruc. support/road maint. sys Types of Application Systems: Operational:Transaction Processing Systems The software environment into which GIS must integrate.

17. ArcGIS--ARC/INFO (ESRI, Inc., Redlands, CA) privately owned by Jack Dangermond, a legend in the field and former Harvard prof. originated commercial GIS and still clear market leader with about a third of the market Strong in gov., education, utilities and business logistics MapInfo Trying very hard to move from Desktop/Present. to Enterprise/Industrial with newer MapX, MapXtreme and MapInsight products Strong presence in telecom and business, especially site selection & marketing Intergraph (Huntsville, AL) origins in proprietary CAD hardware/software strong in design and FM (facilities management), and running hard to match ESRI in GIS UNIX-based MGE (Modular GIS Environment) and FRAMME evolved from CAD after split with Bentley, courageously committed to NT and object technology in 1995 and first released “new generation” GeoMedia product in March 1997 Serious threat to ESRI until ArcInfo 8 released. Bentley Systems (Exton, PA) Bentley brothers (Barry & Ray) originally developed the PC-based Micro-Station product under contract with Intergraph, but split in 1995 have very successfully continued to develop and sell MicroStation GeoGraphics since then. Strong in engineering; advertises itself as “geoengineering” Autodesk’s AutoCAD Map dominant CAD supplier and world’s 4th largest software company fully topological AutoCAD Map introduced in 1996 (but no raster) High initial expectation (huge installed AutoCad base) but hasn’t lived up to earlier expectations Primarily small business/small city customer base Software for GIS: The Main Vector Players The only two “pure GIS” companies.

18. ERDAS/Imagine long established leader, purchased by Leica Systems of Switzerland in 2001 ER MAPPERveryagressive newcomer originating in Australia Envi, another relative newcomer, radar specialization, now (2000) Kodak owned PCI long-term Canadian player CARIS another newer Canadian entry Genasys II older and fading US player Software for GIS: Remote Sensing/Raster • origins in remote sensing thus raster oriented • Some now include vector support, but has proven insufficient for them to really compete with vector-based GIS • Convergence of raster and vector GIS software has not occurred to the degree expected 5 years ago • Need one of these products if you are heavy into remote sensing, image analysis, or orthophotography production • new satellite data products enhance their utility • Will never compete in the enterprise/management market For links to web sites of all these vendors, go to: http://www.utdallas.edu/~briggs/other_gis.html

19. Other Commercial Players Manifold (CDA International Corp)one of the best deals around Maptitude (Caliper Corp, MA): another low cost one CadCorp SIS (Spatial Information System) recent entry from UK WinGIS (Progis Corp, Austria): European entry Smallworld Systems(Englewood, CO) first to use OO (in early 1990s) but never broke out of its FM utilities base and bought by GE in 2000 Blasts from the Past Atlas/GIS from Strategic Mapping, Santa Clara, CA originator of PC-based mapping in the 1980s Failed to “make-it” in GIS purchased by ESRI in 1996 Discontinued 2001 GRASS (Rutgers Univ.) Classic old-timer raster oriented, but some vector routines originally developed by US Army Construction Engineering Research Lab(CERL) army ended dev. & support in 1996 but assumed by Baylor University. IDRSI (Clark Univ) one of the pioneering, university developed packages, limited commercial penetration but still trying raster orientation with some vector capabilities GDS (Graphic Data Systems, Englewood CO) originated as McDonnel Douglas in-house system Purchased by Convergent Group and EDS closure in 1997 left customers in the lurch (e.g. City of Irving) Software for GIS: other professional/analytical

20. Software for GIS: Niche products(examples) Business Targeted • BusinessMAP (from ESRI): $99 choropleth and pin mapping (originally Maplinx) • Calaritas data supplier; iXPRESS integrates demographic and geographic software • Tactician:specialist product for market analysis, site selection, etc. Other Industry Targeted • TransCAD (from Caliper):specialized GIS for transportation GIS specialized functions • Surfer (Golden Software):contour creation & spatial interpolation • Blue Marble Geographic Calculator: projections, conversions, etc. Extensions to standard GIS Products • Add-ons to standard GIS packages to meet niche needs • Remote Sensing, Surveying, Business, Hydrology, Civil Engineering etc. extensions for ArcGIS, MAPINFO and other standard GIS vendors

21. Total Revenue Estimates 2001 1,073 M 2004 2, 020 M Estimated Market Shares 2003 Sector share Utilities 44% Public sector 29% Private sector 24% 2003 Type share Software 64% Services 24% Data 8% Hardware 4%

22. ESRI Product Line-up: ArcGISclient products ArcReader (“adobe acrobat” for maps) & ArcExplorer (spatial data viewer) • Free viewers for geographic data. ArcGIS 9.x Desktop: two primary modules (MS NT/2000/XP only) • ArcMap: for data display, map production, spatial analysis, data editing • ArcCatalog: for data management and preview ArcToolbox, for specialized data conversions and analyses, available as a window in both Available capabilities within these modules are “tiered” • ArcView: viewing, map production, spatial analysis, basic editing • ArcEditor: ArcView, plus specialized editing • ArcInfo: ArcView & ArcEditor plus special analyses and conversions Extensions: for special apps.: Spatial Analyst, 3D Analyst, Geostatistics, Business Analyst,etc. ArcObjects: build specialized capabilities within ArcMap or ArcCatalog using VB for Applications ArcGIS Workstation (for UNIX and MS NT/2000/XP) • the old command line ArcInfo 7.1 ArcGIS Engine (MS NT/2000/XP) • Set of embeddable GIS components (ArcObjects software objects) for building custom applications • Runs under Windows, Unix and Linux, with support for Java, C++, COM and .NET • Replaces MapObjects which were based upon a previous generation of GIS objects Notes: ArcGIS 8 released 2000 to integrate two previous standalone products: ArcView and ArcInfo ArcGIS 9 released 2004 providing the full capability that should have been in ArcGIS 8!!! --full support for all data types (coverages, shapefiles, geodatabases) --full support for all previous geoprocessing analyses --Modelbuilder for scripting and repetitive processing --ArcEngine for building custom applications ArcView 3.3 (the predecessor to ArcGIS 8.x) the only GUI option for UNIX.

23. ESRI Product Line-up: ArcGISserver products SDE (Spatial Database Engine) • middleware to support spatial data storage in standard DBMS • Supports all major industry databases: • Oracle, SQL-Server, IBM DB2, Ingres ArcGIS Server • Permits the creation of server-based GIS services using any ArcGIS capability • Provides GIS capabilities to a user without a desktop GIS system: • inward focus—user goes to server ArcIMS • Software to develop Internet server-based mapping and basic analysis • Provides maps to the user without a desktop GIS system : • outward focus—gives user a map ArcGIS Services • Server based applications built and operated by ESRI or its partners and made available on the Internet for subscription • Normally charged on a “per transaction” basis, but can be flat fee • presumably built using ArcGIS Server

24. Decisions/Actions Requiredvis-à-vis Software • Operating systems for clients and servers • UNIX or Windows for servers • Windows for clients • Interfacing with existing non-GIS application systems or databases • Is it required? • How accomplish? • Selection of GIS software appropriate for task required

25. Council, get off my back! Planner for Plano. Need faster, easier, jazzier way of answering council member queries. Need professional analysis package use ArcView 9, since city has extensive ArcInfo shapefiles & coverages; Uses ModelBuilder and Python scripts to automate most common queries Appraisal District needs to accurately maintain parcel files Replaces current CAD system with ArcEditor for accurate topological editing and easier interface with GIS files Emergency Preparedness Manager for coastal county needs application to track potential spread of oil spills Uses ArcObjects and VB for Applications within ArcInfo version of ArcGIS to develop customized model City Planner wants easier way for citizens to know location of city facilities, property zoning, roads, etc.. Developes ArcIMS application on city server which includes layers for roads, zoning, parcels,schools, other city facilities, etc.. which citizens can access from their home PCs using any browser GIS Software Selection:Choices, Choices, Choices!! Contd on next slide

26. GIS Software Selection:Choices, Choices, Choices!! • Where is my bus? DART service manager. Want application to display real time location of buses on touch-screen terminal to passengers waiting at transit centers • Uses ArcServer to develop central application to track buses • Uses internet browser at transit station to query ArcServer • Developer of software to support operations within a real estate office, including payroll, client tracking, billing, etc.., requires way to map location of houses sold by office • Uses ArcEngine to incorporate mapping capabilities within his software system • Keep the Taxes Coming In. Director of IS for Big City. Must tie all my data together--land ownership, tax rolls, utility lines, roads, 911 calls etc. • Needs enterprise solution. • ArcSDE with ORACLE data base, accessed with ArcExplorer (free), ArcView 9 (for read only & analysis), ArcEditor 9 (for data maintenance) & ArcInfo 9 (for specialized analyses) depending on a particular employees needs

27. Database Choices ArcSDE DB Workstation Middleware ArcSDE supports • Oracle • Microsoft SQL Server • IBM’s DBII • Informix Selection often a function of what already exists in house for business applications Some database vendors have ability to support spatial data directly without ArcSDE (e.g. Oracle Spatial) • may conform to ISO standard • Better security • May cost less than ArcSDE • More limited capability (e.g. no “geodatabase”)

28. RAID Options for Backup and Performance • JBOD—just a bunch of disks: • Lowest cost, no protection • RAID 0 (stripping) • Maximum performance, no protection • RAID 1 (mirroring) • Maximum protection but 50% storage loss • RAID 3 & 4 (parity disk) • Protection, 90% storage, but performance problem ‘cos of contention on parity disk • RAID 1,0 (mirroring & striping) • Protection and performance, but 50% loss • RAID 5 (parity and striping) • Maximum space with protection and performance Composite solutions (RAID = redundant array of independent disks)

29. Configurations differ depending on database. Remember: Does not protect against disastrous destruction! (9/11, tornado, etc.)

30. 4,000-fold increase 12,000-fold increase Networking and TelecommunicationsWhat Changed, beginning in the 1980s? • speed and capacity dramatically increased • UTD to Austin • 1984: 9,600 baud (9 dot 6: 9.6 kbps) • 1996: 44,736,000 bps (T3--44.7 Mbps) • transatlantic phone cable • 1960: 125 simultaneous calls • 1990: 1.5 million • Revolutionary new capabilities arrived • The Internet • and then Wireless

31. What are the implications? • permitted dramatically different ways of doing things, not just more of the same. • centralized or dispersed geographic databases • Accessed by police & utility workers in the field • Thus, dramatically increases options available to managers • necessitates careful analysis of alternatives • makes decision making more difficult

32. The Signal Flip • wireless for mass broadcast, PAST cable for individual voice/data wireless if it moves Present/FUTURE cable if it stays in one place Reason: electromagnetic spectrum is a scarce resource. one fiber can carry the entire spectrum (TV, radio, satellites, etc.) regulated by the FCC!

33. Types of Networks(wired) • “Last Mile to the house” • 300 baud dial-up early ‘80s • 56 kbps dial-up (since mid 1990s) • ISDN 64-128 kbps (digital) • DSL: 1.5Mb down/64Kb up (6Mb/640K in future) • cable modem: 2 Mbps (load dependent) • LANs (local area network) within building or campus • 300 baud RS232 (1970s) • 10MB Ethernet (1980s) • 100MB Ethernet (mid 1990s) • 1GB Ethernet (late 1990s) • WANs (wide area network) connects LANs • shared, public facilities (Internet) • Dedicated Telco leased lines (fixed bandwidth) • T1 (24 x 64 Kbps channels:1.54MBps) • T3 (28 T1 circuits: 45MBps; requires fiber) • Dedicated fiber (usually SONET protocol) • OC1 (base rate) 50 mbps OC12 = 600 Mbps • OC3 = 150 Mbps OC48 = 2.4Gbps • ‘virtual circuit’ protocols (max. guaranteed bandwidth) POTS: Plain Old Telephone System ISDN: Integrated Sevices Digital Network (or Innovations Subscribers Don’t Need) DSL: Digital Subscriber Line Telco: Telecommunications provider (e.g AT&T) baud: analog measure of speed bps: (bits per second) digital measure of speed baud ~= bps unless compression involved;

34. twisted pair wire: (now up to 1GB) distance/speed trade-off 150m. max. for highest speed high speed: closet to desktop (using Level 5 cable) low speed: home to telco office Maximum leased line over copper is T1 (1.4Mbps) coaxial cable (up to 200Mbps): longer distances (up to 20 miles) was fading but cable TV and Internet brought revival for “last mile” applications fiber optic (exceeds 100Gbps & doubling every 9 months--twice Moore’s Law) single mode for distances over 1 or 2 miles Multi mode within buildings install 62.50/125 micron multimode with ST connectors to closets Types of Communications Channels • wireless • Cellular data: 3G services • Wireless LANs (WiFi 802.11b/g) • radio--mobile data networks (police) • satellite low orbit --individual links • satellite geostationary (22,000 miles up)--’group’ links • infrared (1 mile) • microwave (35 miles line of site)

35. LAN/WAN Network Configurations ISP: Internet Services Providers e.g. Netcom, onramp Isolated LAN Internet “cloud” Dial-up MCI Internet Exchange Carriers The “Last Mile” problem: high speed communications into homes from nearest ‘telco’ office. SONET over fiber CIX Sprint T1 from long distance provider (e.g MCI T1 leased line from local telephone company (over fiber or copper) T1s from local telco T1 Dial-up • LAN • LAN LAN in Another City Typically, campus or building is wired with fiber (multi-mode) between building and to communications closets, then twisted pair from closet to office.

36. hub hub hub hub Configuring an Ethernet LAN Cisco predominates Router (for campus) to WAN (T1 leased line?) PPP protocol dial-up router Ethernet Switch (in each building? ) copper (<150m) or fiber in closet copper cable Level 3-5 150m

37. Network Challenges • architecture and standards • ensuring everything will work together • capacity suitability & availability • ensuring demand can be met • does each link have appropriate capacity? • reliability and redundancy • ensuring continuous availability • security: providing for • authentification (who really sent it?) • confidentiality (is message private?) • verification (has message been changed?) • confirmation (has message been received?)

38. Putting it Together

39. The Cop Shop Freda and Fernando in the field ? Network Configuration Citizen Sam & Samantha at home Switch on each floor Hub in each closet Router Infrared Fiber T1 Terminal Servers Or replicated data base (synch. nightly) with local Workstations Or ArcIms (for view only)

40. Network Loads Note: --limited capability of WAN --greater efficiency of database (SDE) Source: Dave Peters, System Design and Configuration Strategies, ESRI User Conference 2002

41. Server Configuration: User Loads Source: Dave Peters, System Design and Configuration Strategies, ESRI User Conference 2002

42. Server Selection for ArcSDE An Intel Xeon 4 CPU 3GHz server with 6GB memory running 2004 ArcGIS software will support 100 users at SPECrate of 60 Source: Dave Peters, System Design and Configuration Strategies, 2004 p. 7-27