Errors, Exceptions and Warnings#
At this stage of the lecture, you probably have encountered several (many?) error messages when executing your code. Error messages are not nice, but they are frequent (no matter your programming skills) and simply belong to the programming workflow. In this chapter you will learn to recognize the different type of errors, and you will learn how to deal with them.
Copyright notice: parts of this chapter are taken from the official python tutorial
Syntax errors#
Syntax errors are perhaps the most common kind of complaint you get while you are still learning the language:
t = range(12
Cell In[1], line 1
t = range(12
^
SyntaxError: incomplete input
The parser repeats the offending line and displays a little ‘arrow’ pointing at the earliest point in the line where the error was detected. The error is caused by (or at least detected at) the token preceding the arrow. File name and line number are printed so you know where to look in case the input came from a script.
Tip: avoiding syntax errors. If you chose to work with a good IDE, it will let you know about syntax errors way before you even execute the code (this is called a linter). But sometimes it is still difficult to find out what’s wrong in a line. The syntax errors which are hardest to track down are missing parentheses:
# bad coding: where's the parenthesis missing?
very_useful_formula = ((10 + 2)**3 - (4 - 5)**2)*10/(5/6) + (6/5)**2)
Cell In[2], line 2
very_useful_formula = ((10 + 2)**3 - (4 - 5)**2)*10/(5/6) + (6/5)**2)
^
SyntaxError: unmatched ')'
Here, it helps to separate your code into simpler, multiple-lines statements:
# better coding
numerator = (10 + 2)**3 - (4 - 5)**2
numerator *= 10 # do you know this one?
denominator = (5/6) + (6/5)**2
useful_number = numerator / denominator
Exceptions#
Even if a statement or expression is syntactically correct, it may cause an error when an attempt is made to execute it. Errors detected during execution are called exceptions and are not unconditionally fatal: you will soon learn how to handle them. Most exceptions are not handled by programs, however, and result in error messages as shown here:
10 * (1/0)
---------------------------------------------------------------------------
ZeroDivisionError Traceback (most recent call last)
Cell In[4], line 1
----> 1 10 * (1/0)
ZeroDivisionError: division by zero
4 + spam*3
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
Cell In[5], line 1
----> 1 4 + spam*3
NameError: name 'spam' is not defined
'2' + 2
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In[6], line 1
----> 1 '2' + 2
TypeError: can only concatenate str (not "int") to str
The last line of the error message indicates what happened. Exceptions come in different types, and the type is printed as part of the message: the types in the examples are ZeroDivisionError, NameError and TypeError. The string printed as the exception type is the name of the built-in exception that occurred.
The rest of the line provides detail based on the type of exception and what caused it.
The preceding part of the error message shows the context where the exception happened, in the form of a stack traceback. In general it contains a stack traceback listing source lines; however, it will not display lines read from the interactive interpreter.
Raising Exceptions#
The raise statement#
When writing programs you will sometimes need to raise exceptions yourself. This can be done with the raise statement:
def useful_addition(a, b):
if a > 12 or b > 12:
raise ValueError('Adding numbers larger than 12 is not possible')
return a + b
A ValueError is often used by programmers to tell the user that they are trying to use non-valid arguments in a function:
useful_addition(4, 22)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[8], line 1
----> 1 useful_addition(4, 22)
Cell In[7], line 3, in useful_addition(a, b)
1 def useful_addition(a, b):
2 if a > 12 or b > 12:
----> 3 raise ValueError('Adding numbers larger than 12 is not possible')
4 return a + b
ValueError: Adding numbers larger than 12 is not possible
Raise the correct exception#
Here, I had to take two decisions: which exception should I raise, and which message should I send to the function’s caller. I could have taken another, much less informative path:
def bad_addition(a, b):
if a > 12 or b > 12:
# Not recommended
raise RuntimeError('An error ocurred.')
return a + b
It is your job to raise more helpful exceptions than that. The built-in exceptions page lists the built-in exceptions and their meanings. Scroll through the list of possible standard errors: you will see that many of them have meaningful, informative names stating what the error is about. You should learn to use some of them for your own purposes.
When to raise an exception?#
The type of exception you might be tempted to use most often is ValueError or TypeError. For example, a well intentioned programmer might want to be nice to the user and write the following:
def an_addition(a, b):
# Check input
if type(a) == str or type(b) == str:
raise TypeError('We can only add numbers, not strings!')
if type(a) != type(b):
raise TypeError('We can only add numbers of the same type!')
# OK, go
return a + b
While it is sometimes useful to check the input of a function, this kind of boilerplate code is considered bad practice in python. The reason is that, very often, the error messages sent by python itself are informative enough. See the following examples
def simple_addition(a, b):
return a + b
simple_addition('1', 2)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In[12], line 1
----> 1 simple_addition('1', 2)
Cell In[11], line 2, in simple_addition(a, b)
1 def simple_addition(a, b):
----> 2 return a + b
TypeError: can only concatenate str (not "int") to str
Here the message is informative enough: so why should we bother adding our own message on top of that? As a rule of thumb: raise your own exception when you think that the default error message is not informative enough.
A good example comes from the assignment where we had to download SRTM files from a server. Two situations could lead to a bad error message:
SRTM data are available only from 60°S to 60°N: attempting to download a file outside this range would result in an HTTPError
There are no topography files for ocean tiles: attempting to download a file in the ocean would also result in an HTTPError
HTTPErrors are not very informative. In both cases it is recommended to send a better message than that. Here is my implementation of the srtm_zone function:
def srtm_zone(lon, lat):
"""Returns the code of the SRTM zone corresponding to the given location.
Parameters
----------
lon : float
The longitude
lat : float
The latitude
Returns
-------
The SRTM zone (e.g. '02_10')
"""
# Check input
if abs(lat) > 60 or abs(lon) > 180:
raise ValueError('The given coordinates ({}, {}) '.format(lon, lat) +
'do not fit to the available data range [60S;60N].')
[code deleted]
This informs the user in an informative way that the provided input is not valid. For the “ocean” case the problem is not trivial: how to know beforehand if a tile exists or not? For this task we will use another trick, described below.
Handling exceptions#
If you expect parts of a program to raise exceptions in some cases, it might be useful to handle them and avoid the program to stop or to send a cryptic error message. Look at the following example, which asks the user for input until a valid integer has been entered, but allows the user to interrupt the program (using Control-C or whatever the operating system supports):
while True:
try:
x = int(input("Please enter a number: "))
break
except ValueError:
print("Oops! That was no valid number. Try again...")
The try statement works as follows.
First, the try clause (the statement(s) between the try and except keywords) is executed.
If no exception occurs, the except clause is skipped and execution of the try statement is finished.
If an exception occurs during execution of the try clause, the rest of the clause is skipped. Then if its type matches the exception named after the except keyword, the except clause is executed, and then execution continues after the try statement.
If an exception occurs which does not match the exception named in the except clause, it is passed on to outer try statements; if no handler is found, it is an unhandled exception and execution stops with a message as shown above.
A try statement may have more than one except clause, to specify handlers for different exceptions:
try:
a = b + c
except ValueError:
print('Ha!)
except NameError:
print('Ho?)
An except clause may name multiple exceptions as a parenthesized tuple, for example:
except (RuntimeError, TypeError, NameError):
print('All these are ok, we pass...')
But when will we need to catch errors? As explained earlier, it might be useful to replace a bad error message with a more informative one. Here is my implementation of the download_from_server function I used for the assignment:
from urllib.request import urlretrieve, HTTPError
def download_from_server(zone):
"""Downloads an SRTM file for the given zone.
If the file is already available, skips.
Parameters
----------
zone : str
The file identifier
"""
server_url = 'http://srtm.csi.cgiar.org/wp-content/uploads/files/srtm_5x5/TIFF/'
file_name = 'srtm_{}.zip'.format(zone)
if os.path.exists(file_name):
print('File already available! Skipping...')
return
# Download the file
try:
print('Downloading {}...'.format(file_name))
urlretrieve(server_url+file_name, file_name)
print('Done!')
except HTTPError:
print('The file does not seem to be available on the server. '
'This might be due to the fact that no topography is '
'available at this location (e.g. oceans), or that '
'there was a connection problem. In this case, check your '
'internet connection and try again.')
Note: connection errors are tricky to handle and this will not work in all the cases. This function will need some tuning for more robust solutions, but we leave it like this for now.
Warnings#
Warning messages are typically issued in situations where it is useful to alert the user of some condition in a program, where that condition (normally) doesn’t warrant raising an exception and terminating the program. For example, a library will issue a warning when a program uses an obsolete module. Numpy issues warnings when mathematical computations lead to non-finite results.
Warnings are useful because they do not terminate the program:
import numpy as np
a = np.arange(10)
a / a # issues a warning
/var/folders/29/f4w56zjd1c1b_gdh_b7bjs3c0000gq/T/ipykernel_52354/2968769965.py:4: RuntimeWarning: invalid value encountered in divide
a / a # issues a warning
array([nan, 1., 1., 1., 1., 1., 1., 1., 1., 1.])
This is a nice feature because this invalid division at one location does not imply that the rest of the computations are useless. NaNs (Not A Number, to be explained in the next lesson) are an indicator for missing data, and most scientific libraries can deal with them.
Depending on your use case, you might want to disable warnings or turn them into errors:
Silencing warnings#
It is possible to temporarily disable warnings with the following syntax:
import warnings
with warnings.catch_warnings():
warnings.simplefilter("ignore")
b = a / a
b
array([nan, 1., 1., 1., 1., 1., 1., 1., 1., 1.])
Warnings as exceptions#
In a similar way, you can turn warnings into exceptions:
import warnings
with warnings.catch_warnings():
warnings.simplefilter("error")
b = a / a
---------------------------------------------------------------------------
RuntimeWarning Traceback (most recent call last)
Cell In[15], line 5
3 with warnings.catch_warnings():
4 warnings.simplefilter("error")
----> 5 b = a / a
RuntimeWarning: invalid value encountered in divide
The with statement in the examples above is called a context manager. As the name indicates, it serves to temporarily change the context of the code block that follows. In this case, it defines a part of the code where the warnings are silenced or errored. We will get back to context managers later in the lecture.
You can also disable warnings for an entire script or interpreter session simply by filtering them without using the context manager:
warnings.simplefilter("ignore")
This is not recommended, as it might hide important and unexpected warnings.
Filter warnings by type or message#
warnings.filterwarnings gives more control to the filter in order to suppress predefined warnings only:
np.array([2.**100])**100
/var/folders/29/f4w56zjd1c1b_gdh_b7bjs3c0000gq/T/ipykernel_52354/3776166511.py:1: RuntimeWarning: overflow encountered in power
np.array([2.**100])**100
array([inf])
with warnings.catch_warnings():
warnings.filterwarnings("ignore", "invalid value encountered", RuntimeWarning)
# Divide by zero is ignored
b = a / a
# But overflow is not
np.array([2.**100])**100
/var/folders/29/f4w56zjd1c1b_gdh_b7bjs3c0000gq/T/ipykernel_52354/3074648172.py:6: RuntimeWarning: overflow encountered in power
np.array([2.**100])**100
Take home points#
exceptions are not something to be afraid of: they are helping us to find problems in our code
there are different types of exceptions, which is useful to understand what is causing the problem. Often, an additional error message is printed too, further explaining the problem
you can raise exceptions yourself in your code.
the
try ... exceptstatements help to catch expected errors and do something about it. This is particularly useful in software which need to run whatever happens.