Project #1: the ClimVis package

This is the main project of this semester, and will count for 30% of the final grade. The submission is going to happen in three stages:

  • First submission deadline: Sunday 08.12.2019 (one .zip file per group containing their version of the climvis package)
  • Code review by me and 30 min feedback round per group (week from December 09 to December 13)
  • Second submission deadline: Sunday 12.01.2020

Here are the points I will take into account for the grading:

  • code quality: documentation, comments, tests, compliance to pep8
  • traceability of group work: who did what, are the author(s) name(s) provided in the function's docstrings
  • level and complexity of the implemented tools (i.e. amount of programming work)
  • originality of the implemented tools
  • reaction to the code reviews and updates

I will try and test the code (using pytest) on my computer - any new data or library that is necessary should be documented in the code, or in the command line utility documentation!

The climvis package

I've written a small package called climvis. Its design is based on the template package, but this time it really does something.

Download the zipped package from here and extract it. Read the README and the package requirements first. You will have to install the motionless package for it to work: it is not available on conda, but you can install it with: pip install motionless

Now install climvis in development mode. If you are working on your own computer run:

pip install -e .

from the root directory. If you are working on the university computer, run:

pip install --user -e .

instead.

Try the command line interface (cruvis -h) from a terminal. Now explore setup.py: can you identify what makes the command available from the terminal? Where is the code executing the command cruvis?

Familiarize yourself with the tool. Can you understand what the role of each function is?

Can you run the tests successfully? Probably not. Somewhere in cfg.py, a hard coded path is pointing to a non-existing directory.

If you are working on your own computer, change the path to a local folder of yours and download the necessary data:

Careful! The total decompressed output is about 6GB large.

If you are working on the university computer, a folder containing the data is available at:

/project/c7071047/data-scipro/

The scratch folder contains only a subset of the data (Central Europe): the tool works for these locations, but not globally.

Make sure you are able to run the command succesfully before going on. For example, cruvis -l 12 47 should work fine and display a page in your browser.

Guided exercise: make the tool more robust

Here are a couple of smaller tasks to get you started:

  • The hard coded path in cfg.py is very ugly. At the same time, we really need the CRU data for the tool to run! Add a safety check in the code in order to make sure the files are available. If the CRU files are not available, print the following message and exit the program:

    The CRU files are not available on this system. For cruvis to work properly, please create a file called ".cruvis" in your HOME directory, and indicate the path to the CRU directory in it.

  • Add a small function in cfg.py which parses this new .cruvis file and uses the given directory in place of the hard-coded one. Tip: you could make good use of os.path.expanduser for this function.

  • Selecting a location over the oceans will raise a very uninformative python error message. Make the write_html function more robust: if the data contains NaNs, print a warning and exit the program.

Project: make the tool better

Now you should be ready to contribute to this great package! Add at least N simple functionalities to it, where N is your group size (2, 3, or 4). This functionality can be anything you want, as long as it makes you write some code.

Here are some ideas of functionalities that you can adapt at wish:

  • you could add a plot of the 1901-2018 timeseries for this grid point, and make a trend analysis for the period.
  • you could add other data from CRU to the plot.
  • you could add a new command to the tool called uibkvis, which instead plots the data from the UIBK weather stations. For example, you could plot the temperature for all stations, or even better: give choices to the user as to what they want to plot. You'll have to document its command line options of course. You can get inspiration from last year's excercise, which we didn't have time to do this year.
  • you could redo the plots from the old ACINN webpage, but better?
  • you could use a python windrose package to plot a windrose from the uibk data, and maybe extend assignment 06 (wind statistics) to produce a webpage (instead of the sentence it produces now)
  • you could add an option to cruvis, which accepts a city name instead of the lon and lat of the point. You could use the file shipped with the package for this (world_cities.csv).
  • you could add a new command line tool to compare the climate at two locations.
  • you could parse any other data you'd find interesting from the web and plot it! For example, upper air soundings?
  • you could use an interactive plotting library like Bokeh (possibly by making use of higher level tools like holoviews) to make reactive plots in your HTML page.
  • any idea you find interesting! You can discuss some ideas with me beforehand if you are unsure.

Some tasks are more difficult than others. I don't mind if you take an easy one, as long as you write something on your own, and write good code. Try to split the tasks within the group to get something coherent, ideally not only with "easy" tasks! Don't forget to write tests.

Back to the table of contents