Assignment 05#
Due date: 03.05.2023
This week’s assignment has to be returned in the form a jupyter notebook. Use jupyter notebooks for what they excel at: combining text, code, and figures. Use the markdown format to structure your assignments, and write text when asked to.
As explained in the video, I recommend to download this notebook in order to answer the questions directly in it.
Don’t forget the instructions!
Introduction: live meteorological data#
Warning
Online access to ACINN data does not work anymore. For a similar excercise I recommend to have a look at GeoSphere Austria’s API. I may update the resources here in the future.
The institute website provides raw data from several stations around Innsbruck using a live feed at the following addresses:
https://acinn-data.uibk.ac.at/innsbruck/3 for the three days data
https://acinn-data.uibk.ac.at/innsbruck/7 for the seven days data
The datasets for other stations are available, per analogy:
two other stations are available but we’ll have a look at them next time
The data is shared by ACINN under a Creative Commons Attribution-ShareAlike 4.0 International License.
The data is provided in the json format, often used for web applications. Fortunately, this is very easy to read in python. To make your life easier, I have prepared a reading function below.
It is not important that you understand how it works exactly, but you should understand how to call the function as explained in the examples below.
import numpy as np
def acinn_meteo_data(station="innsbruck", ndays=3):
"""Parse live meteorological data from the ACINN servers.
Requires an internet connection!
Parameters
----------
station : str
one of "innsbruck", "obergurgl"
ndays : int
either 3 or 7 days
Returns
-------
(time, air_temp, wind_speed, wind_dir) :
the meteorological data (python lists)
Examples
--------
>>> time, air_temp, wind_speed, wind_dir = acinn_meteo_data()
>>> type(time)
<class 'numpy.ndarray'>
>>> len(time) > 0
True
"""
from urllib.request import Request, urlopen
from datetime import datetime, timedelta
import json
url = f'https://acinn-data.uibk.ac.at/{station}/{ndays}'
req = urlopen(Request(url)).read()
# Read the data
data = json.loads(req.decode('utf-8'))
# Convert the time
time = np.array([datetime(1970, 1, 1) + timedelta(milliseconds=ds) for ds in data['datumsec']])
# Read the data
wind_speed = np.array(data['ff'])
wind_dir = np.array(data['dd'])
air_temp = np.array(data['tl'])
# This is for missing data filtering - more on this in the next class
air_temp[air_temp < -99] = np.NaN
is_finite = np.isfinite(air_temp) & np.isfinite(wind_speed) & np.isfinite(wind_dir)
if np.any(~is_finite):
time = time[is_finite]
air_temp = air_temp[is_finite]
wind_speed = wind_speed[is_finite]
wind_dir = wind_dir[is_finite]
# Return the data
return time, air_temp, wind_speed, wind_dir
# Testing
import doctest
doctest.testmod()
**********************************************************************
File "__main__", line 22, in __main__.acinn_meteo_data
Failed example:
time, air_temp, wind_speed, wind_dir = acinn_meteo_data()
Exception raised:
Traceback (most recent call last):
File "/Users/uu23343/.mambaforge/envs/oggm_env/lib/python3.12/doctest.py", line 1368, in __run
exec(compile(example.source, filename, "single",
File "<doctest __main__.acinn_meteo_data[0]>", line 1, in <module>
time, air_temp, wind_speed, wind_dir = acinn_meteo_data()
^^^^^^^^^^^^^^^^^^
File "/var/folders/29/f4w56zjd1c1b_gdh_b7bjs3c0000gq/T/ipykernel_55317/3280285223.py", line 33, in acinn_meteo_data
req = urlopen(Request(url)).read()
^^^^^^^^^^^^^^^^^^^^^
File "/Users/uu23343/.mambaforge/envs/oggm_env/lib/python3.12/urllib/request.py", line 215, in urlopen
return opener.open(url, data, timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/uu23343/.mambaforge/envs/oggm_env/lib/python3.12/urllib/request.py", line 521, in open
response = meth(req, response)
^^^^^^^^^^^^^^^^^^^
File "/Users/uu23343/.mambaforge/envs/oggm_env/lib/python3.12/urllib/request.py", line 630, in http_response
response = self.parent.error(
^^^^^^^^^^^^^^^^^^
File "/Users/uu23343/.mambaforge/envs/oggm_env/lib/python3.12/urllib/request.py", line 559, in error
return self._call_chain(*args)
^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/uu23343/.mambaforge/envs/oggm_env/lib/python3.12/urllib/request.py", line 492, in _call_chain
result = func(*args)
^^^^^^^^^^^
File "/Users/uu23343/.mambaforge/envs/oggm_env/lib/python3.12/urllib/request.py", line 639, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
**********************************************************************
File "__main__", line 23, in __main__.acinn_meteo_data
Failed example:
type(time)
Exception raised:
Traceback (most recent call last):
File "/Users/uu23343/.mambaforge/envs/oggm_env/lib/python3.12/doctest.py", line 1368, in __run
exec(compile(example.source, filename, "single",
File "<doctest __main__.acinn_meteo_data[1]>", line 1, in <module>
type(time)
^^^^
NameError: name 'time' is not defined. Did you forget to import 'time'
**********************************************************************
File "__main__", line 25, in __main__.acinn_meteo_data
Failed example:
len(time) > 0
Exception raised:
Traceback (most recent call last):
File "/Users/uu23343/.mambaforge/envs/oggm_env/lib/python3.12/doctest.py", line 1368, in __run
exec(compile(example.source, filename, "single",
File "<doctest __main__.acinn_meteo_data[2]>", line 1, in <module>
len(time) > 0
^^^^
NameError: name 'time' is not defined. Did you forget to import 'time'
**********************************************************************
1 items had failures:
3 of 3 in __main__.acinn_meteo_data
***Test Failed*** 3 failures.
TestResults(failed=3, attempted=3)
The function can be called as follows:
time, air_temp, wind_speed, wind_dir = acinn_meteo_data(station='innsbruck', ndays=7)
---------------------------------------------------------------------------
HTTPError Traceback (most recent call last)
Cell In[2], line 1
----> 1 time, air_temp, wind_speed, wind_dir = acinn_meteo_data(station='innsbruck', ndays=7)
Cell In[1], line 33, in acinn_meteo_data(station, ndays)
30 import json
32 url = f'https://acinn-data.uibk.ac.at/{station}/{ndays}'
---> 33 req = urlopen(Request(url)).read()
34 # Read the data
35 data = json.loads(req.decode('utf-8'))
File ~/.mambaforge/envs/oggm_env/lib/python3.12/urllib/request.py:215, in urlopen(url, data, timeout, cafile, capath, cadefault, context)
213 else:
214 opener = _opener
--> 215 return opener.open(url, data, timeout)
File ~/.mambaforge/envs/oggm_env/lib/python3.12/urllib/request.py:521, in OpenerDirector.open(self, fullurl, data, timeout)
519 for processor in self.process_response.get(protocol, []):
520 meth = getattr(processor, meth_name)
--> 521 response = meth(req, response)
523 return response
File ~/.mambaforge/envs/oggm_env/lib/python3.12/urllib/request.py:630, in HTTPErrorProcessor.http_response(self, request, response)
627 # According to RFC 2616, "2xx" code indicates that the client's
628 # request was successfully received, understood, and accepted.
629 if not (200 <= code < 300):
--> 630 response = self.parent.error(
631 'http', request, response, code, msg, hdrs)
633 return response
File ~/.mambaforge/envs/oggm_env/lib/python3.12/urllib/request.py:559, in OpenerDirector.error(self, proto, *args)
557 if http_err:
558 args = (dict, 'default', 'http_error_default') + orig_args
--> 559 return self._call_chain(*args)
File ~/.mambaforge/envs/oggm_env/lib/python3.12/urllib/request.py:492, in OpenerDirector._call_chain(self, chain, kind, meth_name, *args)
490 for handler in handlers:
491 func = getattr(handler, meth_name)
--> 492 result = func(*args)
493 if result is not None:
494 return result
File ~/.mambaforge/envs/oggm_env/lib/python3.12/urllib/request.py:639, in HTTPDefaultErrorHandler.http_error_default(self, req, fp, code, msg, hdrs)
638 def http_error_default(self, req, fp, code, msg, hdrs):
--> 639 raise HTTPError(req.full_url, code, msg, hdrs, fp)
HTTPError: HTTP Error 404: Not Found
Exercise 1: data exploration#
Please answer the questions one by one in the cells below.
What is the dtype of the data stored in the air_temp, wind_speed, wind_dir arrays? And their size? And their shape?
# your answer here
What is the dtype of the data stored in the time array? Try to ask the following questions:
print(time.dtype)
print(type(time[0]))
# your answer here
What is the time resolution of the data? To answer this question, you can print the two first elements of the time array.
# your answer here
Exercise 2: plotting#
Create a graph of the temperature and wind in Innsbruck over the last 3 days. Start by noticing that matplotlib understands datetime objects very well, and create a graph which looks approximately like this: example image (note that your plot can be different with respect to dates, colors, etc. But be careful with the axis labels and legends).
# your answer here
Note: date formatting on axes is something which is very difficult to do automatically. If you feel like it you can try things like shown on the example below (optional).
# Define the date format
from matplotlib.dates import DateFormatter, HourLocator
ax.xaxis.set_major_locator(HourLocator(byhour=[0, 12]))
ax.xaxis.set_minor_locator(HourLocator())
ax.xaxis.set_major_formatter(DateFormatter("%b %d, %HH"))
fig.autofmt_xdate()
Now create a graph of the temperature at Innsbruck and Obergurgl over the last 7 days. The graph should look approximately like this: example image (note that your plot can be different with respect to dates, colors, etc. But be careful with the axis labels and legends).
# your answer here
Exercise 3: statistics#
Compute the average, maximum and minimum temperature in Innsbruck over the past 7 days.
# your answer here
Compute the average, maximum and minimum temperature in Innsbruck over the past 24H. For this, assume that the data has no missing timestamp and that the time resolution is constant (this is sometimes wrong but we don’t care here).
# your answer here
Compute the number of times that the 10m wind speed in Innsbruck was above 5 m/s over the past 7 days. Now convert this number to a percentage over the past 7 days.
# your answer here
Compute the number of times that the air temperature in Innsbruck was between 0°C and 10°C over the past 7 days. Now convert this number to a percentage over the past 7 days.
# your answer here
Bonus: average wind direction#
Look at the plot you made about wind direction. Can you think of what problem would occur if you try to compute the average of the wind direction?
Next week, we will address this problem together - so if you are bored you can start to think about how to solve it already ;-)