Python & ModelBuilder

1. Python & ModelBuilder

2. Overview • Python/ModelBuilder Concepts • The Geoprocessor • Checking some environment variables • Providing feedback from your model/script • Starting a script with a model • Bringing a completed script into ArcToolbox • Exercises

3. Why model or write scripts? • Automate workflows • Test hypotheses • Reproducibility • Extend Functionality • Simplification • Quality Control

4. The Geoprocessing Object • Also known as the "geoprocessor" • Has many methods • Manages layers and selections • Provides access to all of ArcGIS's tools • Provides status of ArcGIS licensing • Handles errors and warning messages

5. The Geoprocessing Object gp.Clip_analysis("Roads_2007", "Counties", "Mercer_Roads.shp")

6. Check Data Existence • gp.exists() returns a boolean depending on whether or not the feature class exists • Check most user-submitted data for existence • if gp.exists(data): # continue workingelse: # warn user about missing data • ArcToolbox will check, but scripts run from the command line will not

7. Defining a Workspace • When the geoprocessor has a workspace defined, all processing results will be stored in that folder, geodatabase, or SDE gdb • gp.workspace = "C:\\Student\\” • Backslash is an escape character; ie a “tab” is “\t”, new line is “\n”For just a simple backslash, you need to escape the escape character, which is why it’s doubled. • Once defined, we do not need to specify absolute paths to datasets or outputs located in that workspace

8. Describe Method • gp.describe returns an object with numerous properties based on the type of data passed • Feature Classes (shapefiles, geodatabase layers) • Feature Datasets • Workspaces • Raster Data • gp.describe provides you with GIS-specific information about your data

9. Using the Describe Method • Use Describe to ensure that data passed to your model fits certain criteria • Prevent line or point features being used as Clip Features • Add an area and perimeter field to shapefiles if not present • Convert rasters to 8-bit unsigned if not already • Use Python to get a list of .shp files, then process only the polygon shapefiles

10. Check License Availability • Models and scripts might be dependent on the level of ArcGIS available • The gp.CheckProduct("level") returns "Available" or "NotLicensed" depending on the availability of the product level passed • The gp.SetProduct("level") tells the geoprocessor to use that specific level

11. Handling Input • ArcGIS passes information to your Python script through sys.argv • For example, a clip script would need to listen for three inputs • Input Feature / sys.argv[1] • Clip Feature / sys.argv[2] • Output Feature / sys.argv[3] • Gets tricky when we ask for multiple inputs

12. Providing Feedback • Geoprocessor methods to provide feedback • gp.AddMessage(message string) • gp.AddWarning(message string) • gp.AddError(message string) • Adds messages to the message list • Message list is simply that – stores all the errors, warnings and messages generated by the script

13. Providing Feedback • gp.GetMessage() and gp.MessageCount • GetMessage(x) returns message number x in the list • print gp.GetMessage(gp.MessageCount - 1) • prints the last message generated • Print messages at the end of script or when an error is caught • Print before and after a lengthy processing step

14. Defining a Function • A Python Function allows you to write a block of code that accepts arguments and returns results • Functions allow you to write something once and reuse it as needed • Helps debugging • Keeps code clear and concise • Easier to program

15. Using Functions • We use the def statement to start a block of code that will only be run when our function is called • We want our functions to return values we can use in our geoprocessing • The last statement in our function should be return.

16. Basic Function • A simple function that adds 1 to value passed • def addone(x): return x + 1 • print addone(2)>>> 3 • var = addone(3)print var>>> 4

17. Multiple Clips • Like Exercise #2 from last week, we will provide two lists to a script • List of Input Features • List of Clip Features • We also need to tell the script where to place the output, using gp.workspace()

18. Start in ModelBuilder

19. Export to Python

20. Our First Function • We need a custom function that will take a string of text, split it into pieces and return those pieces as a list • SplitMultiInputs will clean up a line of code and return it as a list

21. Adding SplitMultiFeatures • Add the function and split the input string and the clippers string passed by the Toolbox to our script

22. Two Loops • We will need another loop for each of the inputs • The "inputs" loop contains the "clippers" loop • The loops are structured like so: for input in inputs: (runs m times) for clip in clippers: (runs n times) • The script will then perform mn clips

23. Completed Script

24. Back to the Toolbox • Once our script is complete, we can test it in IDLE and on the command line • What we'd really want is to run our script like any other tool in the Toolbox • We can add the script back to the Toolbox • We need to specify how to talk to the script

25. Add Your Script • Open ArcToolbox • Right click on the Toolbox you want your script to be located • Add > Script…

26. Add Your Script • Our model takes two input parameters • Input Feature Classes • Clipping Feature Classes • Output Workspace • Note that parameters can be set for multiple values • MultiValue = Yes

27. County Clipper • We’ll take a small ModelBuilder model that clips one input using the statewide Counties layer • Using Python, we’ll enhance the model to clip the input 21 times for each of the separate counties • Add functionality to clip multiple inputs by each county

28. Setting Up Our Model • We need a County layer as our clip feature, and a user-defined layer as our input feature • Make Feature Layer • Select By Attributes • Clip (of course) • User-defined Output Workspace

29. Basic Model to Python Script loop just a portion of the model

30. Exported Python Script

31. Adding County Tuple • We need a static list of County names to pass as part of the Select by Attributes “where” clause • Put all the County names into a tuple • Use a for loop to loop over the Select by Attributes and Clip steps with different Counties each time • for county in counties

32. Looping Our Model loop just a portion of the model

33. Looping Our Model • For each county in the model • prepare a SQL statement to select County • run the selection to create our clipping features • use selected features to clip • create 21 output feature classes • each output layer named input_county

34. SQL Statement • SQL statements use single and double quotes to delimit field and value names • Python also uses quotes to delimit strings • Keep track of which quotes are being used • Escape quotes using backslash (\') or (\") • SQL: "COUNTY" = 'MERCER'Python: "\"COUNTY\" = 'MERCER'"

35. Looping Over SQL Expressions • Each time the loop is run, we want to change the SQL statement to select the current county in the loop • We can concatenate the parts of the SQL statement with the county name • Example:

36. Changing the Output Feature • Each output feature name needs to be unique • Append the current County name • Wetlands.shp becomes • Wetlands_Atlantic.shp • Wetlands_Bergen.shp • … • Some string methods are needed

37. Changing the Output Feature

38. Finalizing Our Loop Input and Output County Layer Our For Loop Modified SQL Statement Create Output Path Geoprocessing Tasks

39. USGS Quad Boundaries Clipped

40. Remember Multiple Inputs • How will we modify the script to accept multiple inputs? • We'll need another for loop in the script • Set the Input parameter to accept multiple values • MultiValue = Yes

41. Clip To Counties Tool • If the parameters are set correctly, the dialog box will look similar to the one at right • Our CountyClipper now ready to handle multiple layers, clipping each by county