1 / 47

ArcGIS for Server: Administrative Scripting and Automation

ArcGIS for Server: Administrative Scripting and Automation. Shreyas Shinde Ranjit Iyer. Agenda. Introduction to server administration Command line tools ArcGIS Server Manager Overview of the administrative API Automation Popular scripting workflows Publishing of services

neron
Download Presentation

ArcGIS for Server: Administrative Scripting and Automation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ArcGIS for Server: Administrative Scripting and Automation Shreyas Shinde Ranjit Iyer

  2. Agenda • Introduction to server administration • Command line tools • ArcGIS Server Manager • Overview of the administrative API • Automation • Popular scripting workflows • Publishing of services • Setting up permissions • Extracting popular extents • Monitoring services and databases • Scaling and statistics ArcGIS for Server: Administrative Scripting and Automation

  3. Introduction to server administration Select a deployment pattern Provision hardware ArcGIS Server Installation Media ArcGIS Server Setup Software Authorization Keep it running… ArcGIS for Server: Administrative Scripting and Automation

  4. The administrator’s toolbox ArcGIS Server Administrator Directory ArcGIS Server Manager Console tools ArcGIS for Server: Administrative Scripting and Automation

  5. Command line tools • Located under /arcgis/server/tools • A collection of most popular tools • Categories • Administrative credentials • For resetting administrative accounts • SSL/certificate management • Backup and restore • Managing cache • Lock the tools down using OS file permissions! ArcGIS for Server: Administrative Scripting and Automation

  6. Demo Command line tools ArcGIS for Server: Administrative Scripting and Automation

  7. ArcGIS Server Manager • Browser based interactive application • Located at: http://server.domain.com:6080/arcgis/manager • Simple, easy-to-use • Ability to manage your entire server site • Even though you are accessing Manager on a specific server node • Great for common workflows! ArcGIS for Server: Administrative Scripting and Automation

  8. Demo le Tour de Manager ArcGIS for Server: Administrative Scripting and Automation

  9. ArcGIS Server Manager – under the hood Simple HTTP requests to the administrative API ArcGIS for Server: Administrative Scripting and Automation

  10. Overview of the administrative API • Entire server administration is an API • If ArcGIS Server Manager can use it, so can you! • RESTful • GET and POST of JSON data • Well organized and well documented • Lots of samples • Scriptable • Designed for automation! ArcGIS for Server: Administrative Scripting and Automation

  11. The administrative API Create and manage your services. Start or stop them as wanted Services Add new nodes to grow capacity. Remove them to shrink. Organize machines into clusters. System environment Machines + Clusters Manage system properties, web adaptors, server directories. Get fine grained statistics on your GIS services. Statistics (new at 10.3) Admin API Security Databases Manage all your database connections through a single interface. Logs Configure GIS tier or web tier authentication. Manage accounts, roles and permissions on services. Configure SSL. Harvest logs for auditing, understanding usage patterns and troubleshooting. ArcGIS for Server: Administrative Scripting and Automation

  12. Why automate? • Common repeated workflows • Plot popular extents for the data • Scheduled actions • Restarting services when data is updated • Build map caches during low demand by moving machines to caching cluster • Reactive actions • Adding capacity (server nodes) to a cluster when demand increases • Send email notifications when errors are logged • Works with all programming/scripting languages that support HTTP! ArcGIS for Server: Administrative Scripting and Automation

  13. Where should you begin? ArcGIS for Server: Administrative Scripting and Automation

  14. Demo Publish Services ArcGIS for Server: Administrative Scripting and Automation

  15. Motivation • Publishing many services • Moving services between Dev, Staging, Production environments • Backup/restore is an option but sometimes you just want to republish services • Can publish from Desktop and Manager • Tedious if you have lots of services • Automation is your friend! ArcGIS for Server: Administrative Scripting and Automation

  16. Task : Publish a list of services listed in a file • Help system has a script that uses ArcPy • ArcPy is an python library that complements the Admin REST API • Provides many convenience functions ArcGIS for Server: Administrative Scripting and Automation

  17. Caveats • Works great, just need a few lines of code • However, you need ArcPy on the machine • Script publishes service definitions sequentially • For every SD • Upload • Publish • With many SDs to publish this can be a very slow process • Publishing ~100 services took hours ArcGIS for Server: Administrative Scripting and Automation

  18. We can do better • Run the script from any machine • No ArcPy dependency • Exploit concurrency • Parallel uploads • Parallel publishing • If your script is amenable to concurrency, exploit it by all means ArcGIS for Server: Administrative Scripting and Automation

  19. Publishing a service with an SD • Two step process • Upload the SD to the Server • QUICK • Invoke the publishing tool • INTENSIVE ArcGIS for Server: Administrative Scripting and Automation

  20. Script design Upload Publish Upload SD SD SD SD UpldID UpldID UpldID Publish Upload ArcGIS for Server: Administrative Scripting and Automation

  21. Comparison results Speed up of ~3 ArcPy publishing script DIY multi-threaded publishing script ArcGIS for Server: Administrative Scripting and Automation

  22. Demo Starting a service ArcGIS for Server: Administrative Scripting and Automation

  23. Options • Manager • Admin • Script Start Geometry Service ArcGIS for Server: Administrative Scripting and Automation

  24. Anatomy of an Admin script • Main function • Drives the logic • Internal utility functions • Get Token • Make HTTP requests • Parse JSON responses ArcGIS for Server: Administrative Scripting and Automation

  25. Demo Apply Permissions ArcGIS for Server: Administrative Scripting and Automation

  26. Task : Apply permissions to services • Permissions are the way to secure your service • Do it in Manager • Tedious for lots of services. Must automate! ArcGIS for Server: Administrative Scripting and Automation

  27. Demo • Show them previously published services in Manager • Show them roles • Run the script • Discuss the script ArcGIS for Server: Administrative Scripting and Automation

  28. Demo Health check for services ArcGIS for Server: Administrative Scripting and Automation

  29. Get all the folders Print report Using the report API, get all the service instances. Using Geo-services REST API to test availability. ArcGIS for Server: Administrative Scripting and Automation

  30. Demo Health check for databases ArcGIS for Server: Administrative Scripting and Automation

  31. Use findItems to search for all enterprise geodatabases Use validateDataItem to test database connectivity. Print output ArcGIS for Server: Administrative Scripting and Automation

  32. Demo Popular extents ArcGIS for Server: Administrative Scripting and Automation

  33. ArcGIS for Server: Administrative Scripting and Automation

  34. Demo Elastic scaling ArcGIS for Server: Administrative Scripting and Automation

  35. Task : Dynamically add machines to your site • Add machines to reduce latency and increase throughput • Dynamically add them when a latency threshold is breached • De-allocate machines when traffic is less ArcGIS for Server: Administrative Scripting and Automation

  36. Demo • Start with a single node site • Make service requests and watch latency (we’ll use JMeter) • Run a script that adds a machine to a cluster • Watch the latency drop ArcGIS for Server: Administrative Scripting and Automation

  37. Statistics in 10.3 (Sneak peak) ArcGIS for Server: Administrative Scripting and Automation

  38. Statistics at 10.3 • Server is always collecting statistics • Avg response • Number of requests • Number of timeouts • . . . • Report defines the resource, metric and interval • Previous day, last 7 days, last 30 days • Custom time window • Server chooses intelligent data point intervals in the reports • REST API to harvest all this information ArcGIS for Server: Administrative Scripting and Automation

  39. Python is great!! ArcGIS for Server: Administrative Scripting and Automation

  40. Short detour… • Most modern languages (like Python) support Threads • Some languages provide concurrency as a language feature • Go • Erlang • Clojure • Scala • Suited for concurrent programming ArcGIS for Server: Administrative Scripting and Automation

  41. Go • Invented at Google • About concurrency • Lightweight threads (goroutines) • Typed thread-safe communication and synchronization (channels) • Readable concurrent code ArcGIS for Server: Administrative Scripting and Automation

  42. Erlang • Invented at Ericsson • Functional language • Light weight ‘processes’ • Can spawn 100s of thousands of them • Concurrency via message passing • Actors model ArcGIS for Server: Administrative Scripting and Automation

  43. Python is great!! Don’t get us in trouble, we don’t want to start a rebellion  ArcGIS for Server: Administrative Scripting and Automation

  44. Thank you… Please fill out the session survey: First Offering ID: 1146 Second Offering ID: 1344 Online – www.esri.com/ucsessionsurveys Paper – pick up and put in drop box ArcGIS for Server: Administrative Scripting and Automation

  45. Questions? ArcGIS for Server: Administrative Scripting and Automation

  46. Resources ArcGIS for Server: Administrative Scripting and Automation

  47. ArcGIS for Server: Administrative Scripting and Automation

More Related