Wednesday, August 9, 2017

GIS Programming Module 11: Sharing Tools

This week was focused on different ways of sharing custom script tools for ArcGIS, and for the assignment we were provided with a script tool and asked to make some updates and then embed the script in the tool. This allows the toolbox to be shared without the separate script file, and also allows the creator to password-protect the script so that other users can't copy or edit it.

The script tool was already partially functional, but needed some hard-coded variables to be replaced with coding that accepts user input from the tool interface. Then we needed to add descriptions for each of the tool parameters, and finally embed the script in the tool and set a password.

This screenshot shows both the tool interface and the output of an example from the lab using 50 points and distances of 10000 meters:

This is the last week of class, and I've had a great time and learned a ton. The most exciting and immediately useful thing I learned during this course is how to write geometries using a Python script, which I used in my final project to solve a real-life problem. I also found the lessons on turning a model into a script and turning a script into a custom tool to be very interesting, and I already have plans to apply those concepts outside of class as well. I also think it’s very cool that you can use Python scripts to manipulate GIS files and even map documents without ever even opening ArcMap.

Wednesday, August 2, 2017

GIS Programming Module 10: Custom Tools

 This week we learned how to convert a standalone Python script into a tool that can be run like any other ArcGIS tool, which makes scripts more versatile and shareable.

Assuming you've already written a script, the steps to turn it into a customized tool are:
create a new toolbox in ArcGIS > add the script to the toolbox and give it a name/description/etc. > set the tool's parameters > modify the script to allow it to accept user input for the parameters > change any print statements to arcpy.AddMessage (this makes them print to the tool progress window in ArcMap)

Here is the tool window for the script tool from this week's assignment (don't worry about the red Xs; ArcMap is just warning me that those default file paths don't exist on my local computer):

And here is the results window after running the tool successfully:

This was an easy lesson, but an incredibly helpful thing to know, since tools are so much easier to use and share, and better integrated into ArcGIS, than standalone scripts.

Wednesday, July 19, 2017

GIS Programming Module 8: Working With Geometries

This week we learned how to use Python to access geometric elements of features, like their shape type, coordinates, and length/area. For the assignment, we were tasked with writing a script that would iterate over all parts of all features in a shapefile and write the coordinates of each vertex, along with ID numbers and the name of the feature, to a separate text file (created within the script).

The first step is to create a search cursor to find the necessary data > then create a writable text file > then loop over each feature > and for each part of each feature, print the vertex coordinates and other data to both the text file and the interactive window.

Here's the pseudocode I wrote to show how the script works:

Cursor = OID, Shape, Name in rivers.shp
Text = rivers.txt
For row in cursor:
     Vertex = 0
    For point in row
        Vertex + 1
        Write to text OID, Vertex, X and Y coordinates, and Name
        Print OID Vertex, X and Y coordinates, and Name

And here's part of the text file my script produced:

For me, this was definitely the hardest assignment yet, but I was really looking forward to this lesson and I'm glad to be starting to understand how manipulating geometries works.

Wednesday, July 12, 2017

GIS Programming Module 7: Manipulating Spatial Data

This week we tackled some new data structures and the use of cursors to find and edit data in tables. The script for this week's assignment had a number of steps:

create a new geodatabase > create a variable containing a list of all the feature classes in the module data folder > copy each shapefile from the module data folder to the database > create a search cursor that retrieves name, feature, and population for only the county seats from the "cities" shapefile > create an empty dictionary > add the name and population of each city to the dictionary > print the dictionary

It also prints messages at each step to let the user know what's happening and whether the script is running correctly, as you can see below in a screenshot of part of my script's output:

The hardest part for me this week was writing the search cursor correctly and then the right code for adding data to the dictionary. At one point after sorting out the latter, I was still getting mysterious error messages (I was running lines of code in the interactive window as I went so I could see if they worked before writing them into the standalone script) and it turned out I still had a problem with the search cursor. It's always syntax issues for me!

Wednesday, June 28, 2017

GIS Programming Module 6

This week we began working with arcpy and learning to code using geoprocessing functions. The lab assignment was to write a short script that calls a series of geoprocessing tools and prints the resulting messages (the same ones that would appear in the geoprocessing results window of ArcMap).

Specifically, it: adds XY coordinate data to the "hospitals" shapefile (Add XY tool) > prints messages > creates a 1000 meter buffer around features in the "hospitals" shapefile (Buffer tool) > prints messages > dissolves the individual buffers into a single feature (Dissolve tool) > prints messages

Here are the results in PythonWin when the script is run:

Tuesday, June 20, 2017

GIS Programming Module 5: Geoprocessing

This week we looked at geoprocessing using ArcGIS tools, as well as ArcGIS's Model Builder. I've encountered Model Builder before, but learning that you can use a model as the base for a standalone Python script (and then, a Python script as the base for a tool in ArcGIS) is a game-changer. I've already thought of a real-world application to make my life easier outside of class.

Anyway, for this week's lab assignment, we were given several datasets and asked to make a model and then a Python script. I liked doing a model first instead of diving right into scripting, because the models are set up like flow charts, and it's a good way to visualize all the steps you need to go through to get the desired result. From there, it's easy to finalize the model, then export it to a script and finalize that so it can run independently.

For this assignment, it went something like this: all map layers as variables > clip each variable to "basin" layer shape > from clip output for "soils" layer, select all that are not prime farmland > use selection result to erase areas of basin layer

So the final output, of both the model and the Python script, is a shapefile of the basin layer minus parts that aren't prime farmland, like this:

Wednesday, June 14, 2017

GIS Programming: Peer Review 1

Article Citation:
Cerrillo-Cuenca, E. (2017). An approach to the automatic surveying of prehistoric barrows through LiDAR. Quaternary International, 435, 135-145. Retrieved June 14, 2017, from

The author of this paper proposes a methodology for automated identification of prehistoric barrows (burial mounds) in western Europe using Python code to systematically analyze publicly available LiDAR data. Essentially, he has invented a method of creating a predictive model to help identify unknown barrow sites. 
Cerillo-Cuenca argues for the significance of his methodology on the basis that understanding the spatial patterning of barrows, particularly their relationship to settlement sites, is essential for understanding other aspects of prehistoric societies. Remote sensing and systematic survey are an efficient way to gain a better understanding of large areas, especially understudied ones, but, according to Cerillo-Cuenca, other methods used to assess LiDAR data rely on visual interpretation, which has many limitations.  
In the method outlined in this paper, the imagery is first processed to create a bare-earth topography model, which is then analyzed by a series of algorithms to identify locations that match the characteristic topography and shape of barrows, and return coordinates of those locations so that they can be confirmed, either on the ground or from aerial imagery.
I appreciated that Cerillo-Cuenca tested his method in a controlled way by comparing predicted locations to previously recorded sites, and it does appear that the process is fairly successful in identifying well-preserved barrows. It does not do well with poorly-preserved barrows, which is to be expected, but it also seems to consistently miss barrows that are smaller than average. The author acknowledges this shortcoming and suggests the use of higher-resolution imagery, with the possible trade-off of getting more false identifications—of which there are already quite a few based on this article. I would like to read more about how both of these issues might be mitigated, although, to be fair, it sounds as if the methodology is still being refined at this time. I’m also curious as to how the author would make the case that this is a significant improvement over other LiDAR survey methods (it’s likely faster, although I don’t think he explicitly says so), given that it also has clear limitations and still requires review and confirmation of predicted locations.   
On a more immediately practical level, the paper could also benefit from a clearer (and dare I say simplified?) explanation of the process used to identify potential sites, which was a bit over my head as someone with only a rudimentary understanding of this kind of analysis.  
Overall, though, this is an innovative approach that reflects the trend in archaeology towards greater use of technologies like remote sensing and digital modeling, and any method—particularly one that can be automated—for identifying new sites has big implications not only for research but for risk assessment and preservation. No predictive model can be perfect, and this one does seem to need more work, but it sounds very promising.