Term project: the ClimVis package

This is the main project of this semester, and will count for 30% of the final grade. The submission is going to happen in three stages:

Careful!!! New deadlines after the student vote on November 30.

• First submission deadline: Monday 09.01.2023 at 00H00 (midnight on Sunday): one .zip file per group containing their version of the climvis package

• Code review by me and ~30 min feedback round per group: (online) meeting between January 10th and 13th, to be scheduled individually

• Second submission deadline: Tuesday 31.01.2023 at 00H00 (midnight on Monday). Same project with reviews addressed (reviews are small requested changes to the existing project, not a big change)

• Bonus: on Tuesday 31.01, I will ask all the groups to shortly “showcase” their project, i.e. run their app for the class and show what it can be used for. This is not graded, and is more for fun and exchange than anything else!

Here is an indicative table I’ll use for the grading:

Group grade (10 points):

• Documentation: 3 (project documentation + narrative report + individual contributions’ docstrings, i.e. no individual grade here)

• Integration tests: 1 (tool / project level tests)

• pep8: 1 (compliance to pep8)

• Coherence: 2 (how do the individual contributions fit together, i.e. was there an effort of documentation at least)

• Traceability: 1 (who did what, are the author(s) name(s) provided in the function’s docstrings)

• Response to reviews: 2 (reaction to the code reviews and updates: an inadequate or failure to respond to the review can lead to a refusal)

Individual grade (10 points):

• Narrative self-report: 2 (quality and honesty of the self-report)

• Complexity: 3 (level and complexity of the implemented tools, i.e. amount of programming work)

• Originality: 3 (did you do something more than following the list of suggestions)

• Unit tests: 2 (individual, function tests)

Important

I will try and test the code (using pytest) on my computer - any new data or library that is necessary should be documented in the code, or in the command line utility documentation!

The climvis package

I’ve written a small package called climvis. Its design is based on the template package, but this time it really does something.

Download the zipped package from here and extract it. Read the README and the package requirements first. You will have to install the motionless package (mamba install motionless) for it to work.

Now install climvis in development mode. If you are working on your own computer run:

pip install -e .

from the root directory. If you are working on a computer on which you have no admin rights, run:

pip install --user -e .

instead.

Try the command line interface (cruvis -h) from a terminal. Now explore setup.py: can you identify what makes the command available from the terminal? Where is the code executing the command cruvis?

Familiarize yourself with the tool. Can you understand what the role of each function is?

Can you run the tests successfully? Probably not. Somewhere in cfg.py, a hard coded path is pointing to a non-existing directory.

If you are working on your own computer, change the path to a local folder of yours and download the necessary data:

• temperature

• precipitation

• elevation

Careful! The total decompressed output is about 6GB large.

Make sure you are able to run the command succesfully before going on. For example, cruvis -l 12 47 should work fine and display a page in your browser.

Guided exercise: make the tool more robust

Here are a couple of smaller tasks to get you started:

• The hard coded path in cfg.py is very ugly. At the same time, we really need the CRU data for the tool to run! Add a safety check in the code in order to make sure the files are available. If the CRU files are not available, print the following message and exit the program:

  The CRU files are not available on this system. For cruvis (part of the climvis package) to work properly, please create a file called ".climvis" in your HOME directory, and indicate the path to the CRU directory in it.

• Add a small function in cfg.py which parses this new .climvis file and uses the given directory in place of the hard-coded one. Tip: you could make good use of os.path.expanduser for this function. Optional: you can even write a small command line program (e.g. cruvis_setup) which asks the user for the path of the CRU folder, checks if the data is available and if yes, write the file for the user.

• Selecting a location over the oceans will raise a very uninformative python error message. Make the write_html function more robust: if the data contains NaNs, print a warning and exit the program.

Project: make the tool better

Now you should be ready to contribute to this great package! Add at least N+1 simple functionalities to it, where N is your group size. This functionality can be anything you want, as long as it makes you write some code.

Here are some ideas of functionalities that you can adapt at wish.

• you could add a plot of the 1901-2018 timeseries for this grid point, and make a robust trend analysis for the period.

• you could add other data from CRU to the plot.

• you could add a new command to the tool called uibkvis, which instead plots the data from the UIBK weather stations. For example, you could plot the temperature for all stations, or even better: give choices to the user as to what they want to plot. You’ll have to document its command line options of course. You can get inspiration from the wind statistics we did together, but with more variables.

• you could use a python windrose package to plot a windrose from the uibk data, and add the plot to a webpage

• you could add an option to cruvis, which accepts a city name instead of the lon and lat of the point. You could use the file shipped with the package for this (world_cities.csv: careful - it is quite a badly maintained file - i.e. duplicates and missing cities - and you might want to find another one).

• you could add a new command line tool to compare the climate at two locations in one webpage.

• you could parse any other data you’d find interesting from the web and plot it! For example, upper air soundings?

• you could use an interactive plotting library like Bokeh (possibly by making use of higher level tools like holoviews) to make reactive plots in your HTML page. For example to visualize the ACINN data interactively?

• any idea you find interesting! You can discuss some ideas with me beforehand if you are unsure if your idea is not enough or too much. Examples from previous years include:

  • a “fly safe” info webpage, which scrapped data from the web specifically for flying

  • an interactive bokeh app to visualize CRU climate trends in various cities

  • an El Niño advisory page - warning people when El Niño is likely to occur

  • a bokeh tool to visualize radio soundings

  • and more…

Expectations

New in WS22/23: each group member and the entire group needs to write a short narrative self-report (a few lines per point) addressing the following point:

• what was your programming objective when you started? (in the case of a true group work: what task was assigned to you when you started?)

• what changed during the implementation? I.e., what is the outcome of the project?

• what was the biggest challenge you faced in the implementation?

• group question: what was the biggest challenge when merging everyone’s work into one package?

The narrative report should be written in the README in markdown format, and will count towards grading in the elements “Documentation” and “Coherence” for the group grade, and “self-report” in the individual grade.

Some tasks in the list above are significantly simpler than others. Some of you might pick a project which is too ambitious and then reduce their expectations with time. Others might pick one of the simpler task and be finished very quickly.

If you struggle to know what is expected of you, please just be honest with yourselves and with me. I am reluctant to impose thresholds on the number of lines of code, because I find this measure silly (many lines of code is NOT a measure of quality). My goal is that you all get better at programming after the project then when you started, regardless of your starting level.

The list of possible topics should help you to decide on what to code, but otherwise I can provide a vague rule of thumb: if you coded 10 lines of code (without comments), that’s probably not enough (unless they are very clever lines of code). If you coded 1000 lines of code, that’s probably too much.

I don’t mind if you take an easy task, as long as you write something on your own, and write good code. The difficulty of the task will decide on the base grade, and the quality of the code / tests will add points on top. If you pick an “easy” task, you may pick two or three tasks instead of just one. Try to split the tasks within the group to get a coherent product!

You all have very different backgrounds and interest. Try to find a project you like, and convince me that you pushed your own programming limit a little bit and that you learned something in the process.

Ah! And don’t forget to write tests.