GIS Programming Module 6

This week we began working with arcpy and learning to code using geoprocessing functions. The lab assignment was to write a short script that calls a series of geoprocessing tools and prints the resulting messages (the same ones that would appear in the geoprocessing results window of ArcMap).

Specifically, it: adds XY coordinate data to the "hospitals" shapefile (Add XY tool) > prints messages > creates a 1000 meter buffer around features in the "hospitals" shapefile (Buffer tool) > prints messages > dissolves the individual buffers into a single feature (Dissolve tool) > prints messages

Here are the results in PythonWin when the script is run:

GIS Programming Module 5: Geoprocessing

This week we looked at geoprocessing using ArcGIS tools, as well as ArcGIS's Model Builder. I've encountered Model Builder before, but learning that you can use a model as the base for a standalone Python script (and then, a Python script as the base for a tool in ArcGIS) is a game-changer. I've already thought of a real-world application to make my life easier outside of class.

Anyway, for this week's lab assignment, we were given several datasets and asked to make a model and then a Python script. I liked doing a model first instead of diving right into scripting, because the models are set up like flow charts, and it's a good way to visualize all the steps you need to go through to get the desired result. From there, it's easy to finalize the model, then export it to a script and finalize that so it can run independently.

For this assignment, it went something like this: all map layers as variables > clip each variable to "basin" layer shape > from clip output for "soils" layer, select all that are not prime farmland > use selection result to erase areas of basin layer

So the final output, of both the model and the Python script, is a shapefile of the basin layer minus parts that aren't prime farmland, like this:

GIS Programming: Peer Review 1

Article Citation:

Cerrillo-Cuenca, E. (2017). An approach to the automatic surveying of prehistoric barrows through LiDAR. Quaternary International, 435, 135-145. Retrieved June 14, 2017, from http://www.sciencedirect.com/science/article/pii/S1040618216002159

The author of this paper proposes a methodology for automated identification of prehistoric barrows (burial mounds) in western Europe using Python code to systematically analyze publicly available LiDAR data. Essentially, he has invented a method of creating a predictive model to help identify unknown barrow sites. 

Cerillo-Cuenca argues for the significance of his methodology on the basis that understanding the spatial patterning of barrows, particularly their relationship to settlement sites, is essential for understanding other aspects of prehistoric societies. Remote sensing and systematic survey are an efficient way to gain a better understanding of large areas, especially understudied ones, but, according to Cerillo-Cuenca, other methods used to assess LiDAR data rely on visual interpretation, which has many limitations.  

In the method outlined in this paper, the imagery is first processed to create a bare-earth topography model, which is then analyzed by a series of algorithms to identify locations that match the characteristic topography and shape of barrows, and return coordinates of those locations so that they can be confirmed, either on the ground or from aerial imagery.

I appreciated that Cerillo-Cuenca tested his method in a controlled way by comparing predicted locations to previously recorded sites, and it does appear that the process is fairly successful in identifying well-preserved barrows. It does not do well with poorly-preserved barrows, which is to be expected, but it also seems to consistently miss barrows that are smaller than average. The author acknowledges this shortcoming and suggests the use of higher-resolution imagery, with the possible trade-off of getting more false identifications—of which there are already quite a few based on this article. I would like to read more about how both of these issues might be mitigated, although, to be fair, it sounds as if the methodology is still being refined at this time. I’m also curious as to how the author would make the case that this is a significant improvement over other LiDAR survey methods (it’s likely faster, although I don’t think he explicitly says so), given that it also has clear limitations and still requires review and confirmation of predicted locations.   

On a more immediately practical level, the paper could also benefit from a clearer (and dare I say simplified?) explanation of the process used to identify potential sites, which was a bit over my head as someone with only a rudimentary understanding of this kind of analysis.  

Overall, though, this is an innovative approach that reflects the trend in archaeology towards greater use of technologies like remote sensing and digital modeling, and any method—particularly one that can be automated—for identifying new sites has big implications not only for research but for risk assessment and preservation. No predictive model can be perfect, and this one does seem to need more work, but it sounds very promising.

GIS Programming Module 4: Debugging and Error Handling

This week's programming topic was debugging, or finding and fixing errors in code, and error handling, which means writing code that assumes certain kinds of errors may occur and "traps" them, allowing the code to complete anyway.

For the assignment, we were given three pre-written Python scripts containing errors, two of which we had to fix so that the code would run correctly, and the last of which was an exercise in error handling. All of the scripts this week use ArcGIS geoprocessing functions, so this was also my first official exposure to how Python and ArcGIS interact.

The corrected first script takes a shapefile and prints a list of the field names in the attribute table. It works something like this: import ArcGIS modules > assign workspace (map document) > assign feature class > make list of fields > for each item in fields list, print field name

Here are the results:

The second script is a bit more complex and was harder to fix, with more errors and more complicated ones. (I also ran into some added complications on this script and the third one because I'm working from my local computer rather than the UWF GIS remote desktop and thus have to change the file paths in the scripts, which introduces opportunities for new errors!) I'm not sure about everything this script is doing, because we haven't learned much about the geoprocessing functions yet, but in the end, it prints a list of the layers within a specified data frame:

I was nervous about the error handling part of the assignment, but it turned out to be very easy--a generic try-except statement rather than trying to deal with specific kinds of errors separately. The third script has two parts, the first of which is supposed to turn on the visibility and labels for a layer within an ArcMap document, but is written so that it raises an exception. I enclosed most of the code for this section of the script in the try-except statement, which means that the script now prints "Error:" and the error message for this first part, before continuing the run the second part, which prints the name, spatial reference, and scale for the specified data frame:

Another issue I had this week is that I was using IDLE as my Python editor instead of the recommended PythonWin, and I have two versions of it on my computer, a standalone copy and the one that installs along with ArcGIS. The standalone version doesn't recognize arcpy, the module containing the GIS-specific functions, but it is the one my scripts open in by default, so I kept getting error messages about arcpy that weren't part of the assignment. So frustrating! But at least I figured it out, and now know that if I want to use IDLE, I have to open the right version. Or maybe I should just stick to PythonWin.

GIS Programming Module 3: Python Fundamentals, Again

We're still going over the basics of using Python, and this week's topic was conditional statements and loops, which are some of the building blocks of more sophisticated code. For the lab assignment, we were asked to complete a partial script. The finished script prints out the results of a dice-throwing game. Then it generates a list of random numbers, removes any instances of a specific number from that list, and prints the final result. Here's the output:

The trickiest things for me this week were remembering the names of the functions and methods I want to use (I have a problem with knowing what Python can do but not how to do it) and remembering basic details like converting integers to strings in a print statement and adding spaces between things that are going to be printed. I also briefly forgot that the remove method only takes out the first instance, not all of them, and had to look that up to figure out the right code to finish the script. But overall, so far so good.