Contents

Assignment #03

Today’s exercises have to be done with help of the python standard library alone! No external module can be used (except for the --plot option where you can use the helper packages).

Exercise #03-01: automated data download

SRTM is a digital elevation model (DEM) at ~90 m resolution covering almost the entire globe (up to \(\pm\) 60° latitude). The data is organized in 5°x5° tiles. To see a map of the tiles have a look at this download page. This tool is nice to use if you know which tile you want, but not very useful if you want more than one tile at different places of the globe.

Fortunately, the entire dataset is available on this server: srtm.csi.cgiar.org/wp-content/uploads/files/srtm_5x5/TIFF/

In order to protect from disruptive “download all” behavior, the files stored here cannot be listed (i.e. automated data scraping won’t work). Fortunately for us, the file naming convention is very simple:

Here are some examples of locations and their associated tile:

  • (-179, 59) -> ‘srtm_01_01.zip’

  • (-179, 51) -> ‘srtm_01_02.zip’

  • (-174, 54) -> ‘srtm_02_02.zip’

  • (-180, 60) -> ‘srtm_01_01.zip’ (upper-left corner case)

And so forth.

A. Write a script which, given a longitude and a latitude as arguments, downloads the corresponding file in the current directory. The tool should print an error message when the given location is not valid and exit the script.

Hint A1: define “valid” locations first: some are easy to catch, some cannot be caught automatically. Do we really have to deal with those?

Hint A2: unlike last week where we asked for user input, here I’m asking for a script with command line arguments. The command %run download_srtm.py 9 44 (in an ipython interpreter) should download the file at the location 9°W, 44°N.

Hint A3: although I agree with the python documentation that argparse is more robust (and much better) than parsing sys.argv yourself, for the purpose of learning I would like you to use sys.argv today and handle the command line arguments yourselves.

B. Extend this script to be a bit more clever: download the data file only if the file isn’t already available in the current directory. This is particularly useful because the server is not very fast.

C. Extend this script to be even more clever: given a range of longitudes and latitudes, it should download all the files covering this area. For example, the range 9°W to 18°W and 44°N to 47°N would download 6 files. In order to avoid manual mistakes, warn the user by announcing the number of files that the command is going to download, and ask for confirmation before going on (by using the input() function).

Hint C1: the command should now be %run download_srtm.py 9 44 18 47.

D. Let’s add even more functionality to our script:

D1. print a message to the user announcing if the file was downloaded or was already on disk

D2. print the size of the file after download - choose an appropriate unit for the text

D3. let’s assume that the user actually wants to download the files in a specific folder instead of the current directory (i.e. a folder where she stores all her DEMs). Let’s give this opportunity to her by adding two options to the script:

  • --output-dir /path/to/a/dir should write the file (and check for its existence) in the directory given as argument

  • --default-dir should do following: it checks for a hidden file in the user’s $HOME directory called .demdownrc (a text file), which if present should be opened to read one single line of text: the path to the default directory. If this file does not exist, ask the user to provide the default directory as input in the command line, and write this information in the .demdownrc file on disk (your script has to create this file)

  • in both cases, if the target directory does not exist, print a message to the screen, indicating that you can’t continue and exit the script

  • you can decide if --default-dir can or cannot be at any location in the argument list, i.e. %run download_srtm.py 9 44 --default-dir 18 47 could be a valid command or not. However, %run download_srtm.py 9 44 18 47 --default-dir should be a valid command.

D4. (optional: rasterio package installation and windows issues likely!) if the user asks for a plot, write a plot to disk (alongside the DEM file) with the file name srtm_39_04.png (if the tile 39, 4 was asked for). The user may ask for a plot by adding a --plot to the list of arguments. I.e. %run download_srtm.py 9 44 18 47 --default-dir --plot would write 6 plots in the user-chosen default directory.

Here is a template for your plotting function:

import matplotlib.pyplot as plt
import xarray as xr

# This is some real magic: we open the file from within the zip
f = 'zip://srtm_39_04.zip!srtm_39_04.tif'

# Read the data and plot
with xr.open_rasterio(f) as da:
    da.sel(band=1).plot.imshow(cmap='terrain', vmin=0)
    plt.ylabel('Lat (°)'); plt.xlabel('Lon (°)'); plt.title('srtm_39_04'); 
plt.savefig('srtm_39_04.png')
plt.close()

No need to add more features to the plot than that: we haven’t talked about these libraries yet! Just make sure that the filename is set to the correct value.

Note: you’ll need the matplotlib, xarray and rasterio libraries installed for this to work (on your laptop: conda install matplotlib xarray rasterio). Although, I’m not sure if this is going to work on Windows.

03 - Good practices, programming style and conventions 01 - Introduction to numpy: why does numpy exist?