Errors, Exceptions and Warnings#

At this stage of the lecture, you probably have encountered several (many?) error messages when executing your code. Error messages are not nice, but they are frequent (no matter your programming skills) and simply belong to the programming workflow. In this chapter you will learn to recognize the different type of errors, and you will learn how to deal with them.

Copyright notice: parts of this chapter are taken from the official python tutorial

Syntax errors#

Syntax errors are perhaps the most common kind of complaint you get while you are still learning the language:

t = range(12

  File "/tmp/ipykernel_33985/2729650419.py", line 1
    t = range(12
                ^
SyntaxError: incomplete input

The parser repeats the offending line and displays a little ‘arrow’ pointing at the earliest point in the line where the error was detected. The error is caused by (or at least detected at) the token preceding the arrow. File name and line number are printed so you know where to look in case the input came from a script.

Tip: avoiding syntax errors. If you chose to work with a good IDE, it will let you know about syntax errors way before you even execute the code (this is called a linter). But sometimes it is still difficult to find out what’s wrong in a line. The syntax errors which are hardest to track down are missing parentheses:

# bad coding: where's the parenthesis missing?
very_useful_formula = ((10 + 2)**3 - (4 - 5)**2)*10/(5/6) + (6/5)**2)

  File "/tmp/ipykernel_33985/2797068931.py", line 2
    very_useful_formula = ((10 + 2)**3 - (4 - 5)**2)*10/(5/6) + (6/5)**2)
                                                                        ^
SyntaxError: unmatched ')'

Here, it helps to separate your code into simpler, multiple-lines statements:

# better coding
numerator = (10 + 2)**3 - (4 - 5)**2
numerator *= 10  # do you know this one?
denominator = (5/6) + (6/5)**2
useful_number = numerator / denominator

Exceptions#

Even if a statement or expression is syntactically correct, it may cause an error when an attempt is made to execute it. Errors detected during execution are called exceptions and are not unconditionally fatal: you will soon learn how to handle them. Most exceptions are not handled by programs, however, and result in error messages as shown here:

10 * (1/0)

---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
/tmp/ipykernel_33985/734848216.py in <cell line: 1>()
----> 1 10 * (1/0)

ZeroDivisionError: division by zero

4 + spam*3

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
/tmp/ipykernel_33985/2481180859.py in <cell line: 1>()
----> 1 4 + spam*3

NameError: name 'spam' is not defined

'2' + 2

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/tmp/ipykernel_33985/209420021.py in <cell line: 1>()
----> 1 '2' + 2

TypeError: can only concatenate str (not "int") to str

The last line of the error message indicates what happened. Exceptions come in different types, and the type is printed as part of the message: the types in the examples are ZeroDivisionError, NameError and TypeError. The string printed as the exception type is the name of the built-in exception that occurred.

The rest of the line provides detail based on the type of exception and what caused it.

The preceding part of the error message shows the context where the exception happened, in the form of a stack traceback. In general it contains a stack traceback listing source lines; however, it will not display lines read from the interactive interpreter.

Raising Exceptions#

The raise statement#

When writing programs you will sometimes need to raise exceptions yourself. This can be done with the raise statement:

def useful_addition(a, b):
    if a > 12 or b > 12:
        raise ValueError('Adding numbers larger than 12 is not possible')
    return a + b

A ValueError is often used by programmers to tell the user that they are trying to use non-valid arguments in a function:

useful_addition(4, 22)

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/tmp/ipykernel_33985/391301362.py in <cell line: 1>()
----> 1 useful_addition(4, 22)

/tmp/ipykernel_33985/3711269571.py in useful_addition(a, b)
      1 def useful_addition(a, b):
      2     if a > 12 or b > 12:
----> 3         raise ValueError('Adding numbers larger than 12 is not possible')
      4     return a + b

ValueError: Adding numbers larger than 12 is not possible

Raise the correct exception#

Here, I had to take two decisions: which exception should I raise, and which message should I send to the function’s caller. I could have taken another, much less informative path:

def bad_addition(a, b):
    if a > 12 or b > 12:
        # Not recommended
        raise RuntimeError('An error ocurred.')
    return a + b

It is your job to raise more helpful exceptions than that. The built-in exceptions page lists the built-in exceptions and their meanings. Scroll through the list of possible standard errors: you will see that many of them have meaningful, informative names stating what the error is about. You should learn to use some of them for your own purposes.

When to raise an exception?#

The type of exception you might be tempted to use most often is ValueError or TypeError. For example, a well intentioned programmer might want to be nice to the user and write the following:

def an_addition(a, b):
    # Check input
    if type(a) == str or type(b) == str:
        raise TypeError('We can only add numbers, not strings!')
    if type(a) != type(b):
        raise TypeError('We can only add numbers of the same type!')
    # OK, go
    return a + b

While it is sometimes useful to check the input of a function, this kind of boilerplate code is considered bad practice in python. The reason is that, very often, the error messages sent by python itself are informative enough. See the following examples

def simple_addition(a, b):
    return a + b

simple_addition('1', 2)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/tmp/ipykernel_33985/840712476.py in <cell line: 1>()
----> 1 simple_addition('1', 2)

/tmp/ipykernel_33985/3819827575.py in simple_addition(a, b)
      1 def simple_addition(a, b):
----> 2     return a + b

TypeError: can only concatenate str (not "int") to str

Here the message is informative enough: so why should we bother adding our own message on top of that? As a rule of thumb: raise your own exception when you think that the default error message is not informative enough.

A good example comes from the assignment where we had to download SRTM files from a server. Two situations could lead to a bad error message:

SRTM data are available only from 60°S to 60°N: attempting to download a file outside this range would result in an HTTPError
There are no topography files for ocean tiles: attempting to download a file in the ocean would also result in an HTTPError

HTTPErrors are not very informative. In both cases it is recommended to send a better message than that. Here is my implementation of the srtm_zone function:

def srtm_zone(lon, lat):
    """Returns the code of the SRTM zone corresponding to the given location.
    
    Parameters
    ----------
    lon : float
        The longitude
    lat : float
        The latitude
        
    Returns
    -------
    The SRTM zone (e.g. '02_10')
    """

    # Check input
    if abs(lat) > 60 or abs(lon) > 180:
        raise ValueError('The given coordinates ({}, {}) '.format(lon, lat) + 
                         'do not fit to the available data range [60S;60N].')    
    [code deleted]

This informs the user in an informative way that the provided input is not valid. For the “ocean” case the problem is not trivial: how to know beforehand if a tile exists or not? For this task we will use another trick, described below.

Handling exceptions#

If you expect parts of a program to raise exceptions in some cases, it might be useful to handle them and avoid the program to stop or to send a cryptic error message. Look at the following example, which asks the user for input until a valid integer has been entered, but allows the user to interrupt the program (using Control-C or whatever the operating system supports):

while True:
    try:
        x = int(input("Please enter a number: "))
        break
    except ValueError:
        print("Oops! That was no valid number. Try again...")

The try statement works as follows.

First, the try clause (the statement(s) between the try and except keywords) is executed.
If no exception occurs, the except clause is skipped and execution of the try statement is finished.
If an exception occurs during execution of the try clause, the rest of the clause is skipped. Then if its type matches the exception named after the except keyword, the except clause is executed, and then execution continues after the try statement.
If an exception occurs which does not match the exception named in the except clause, it is passed on to outer try statements; if no handler is found, it is an unhandled exception and execution stops with a message as shown above.

A try statement may have more than one except clause, to specify handlers for different exceptions:

try:
    a = b + c
except ValueError:
    print('Ha!)
except NameError:
    print('Ho?)

An except clause may name multiple exceptions as a parenthesized tuple, for example:

except (RuntimeError, TypeError, NameError):
    print('All these are ok, we pass...')

But when will we need to catch errors? As explained earlier, it might be useful to replace a bad error message with a more informative one. Here is my implementation of the download_from_server function I used for the assignment:

from urllib.request import urlretrieve, HTTPError

def download_from_server(zone):
    """Downloads an SRTM file for the given zone.
    
    If the file is already available, skips.
    
    Parameters
    ----------
    zone : str
        The file identifier
    """
    
    server_url = 'http://srtm.csi.cgiar.org/wp-content/uploads/files/srtm_5x5/TIFF/'
    file_name = 'srtm_{}.zip'.format(zone)
    
    if os.path.exists(file_name):
        print('File already available! Skipping...')
        return
    
    # Download the file 
    try:
        print('Downloading {}...'.format(file_name))
        urlretrieve(server_url+file_name, file_name)
        print('Done!')
    except HTTPError:
        print('The file does not seem to be available on the server. '
              'This might be due to the fact that no topography is '
              'available at this location (e.g. oceans), or that '
              'there was a connection problem. In this case, check your '
              'internet connection and try again.')

Note: connection errors are tricky to handle and this will not work in all the cases. This function will need some tuning for more robust solutions, but we leave it like this for now.

Warnings#

Warning messages are typically issued in situations where it is useful to alert the user of some condition in a program, where that condition (normally) doesn’t warrant raising an exception and terminating the program. For example, a library will issue a warning when a program uses an obsolete module. Numpy issues warnings when mathematical computations lead to non-finite results.

Warnings are useful because they do not terminate the program:

import numpy as np

a = np.arange(10)
a / a  # issues a warning

/tmp/ipykernel_33985/2968769965.py:4: RuntimeWarning: invalid value encountered in divide
  a / a  # issues a warning

array([nan,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.])

This is a nice feature because this invalid division at one location does not imply that the rest of the computations are useless. NaNs (Not A Number, to be explained in the next lesson) are an indicator for missing data, and most scientific libraries can deal with them.

Depending on your use case, you might want to disable warnings or turn them into errors:

Silencing warnings#

It is possible to temporarily disable warnings with the following syntax:

import warnings

with warnings.catch_warnings():
    warnings.simplefilter("ignore")
    b = a / a
b

array([nan,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.])

Warnings as exceptions#

In a similar way, you can turn warnings into exceptions:

import warnings

with warnings.catch_warnings():
    warnings.simplefilter("error")
    b = a / a

---------------------------------------------------------------------------
RuntimeWarning                            Traceback (most recent call last)
/tmp/ipykernel_33985/1532216124.py in <cell line: 3>()
      3 with warnings.catch_warnings():
      4     warnings.simplefilter("error")
----> 5     b = a / a

RuntimeWarning: invalid value encountered in divide

The with statement in the examples above is called a context manager. As the name indicates, it serves to temporarily change the context of the code block that follows. In this case, it defines a part of the code where the warnings are silenced or errored. We will get back to context managers later in the lecture.

You can also disable warnings for an entire script or interpreter session simply by filtering them without using the context manager:

warnings.simplefilter("ignore")

This is not recommended, as it might hide important and unexpected warnings.

Filter warnings by type or message#

warnings.filterwarnings gives more control to the filter in order to suppress predefined warnings only:

np.array([2.**100])**100

/tmp/ipykernel_33985/3776166511.py:1: RuntimeWarning: overflow encountered in power
  np.array([2.**100])**100

array([inf])

with warnings.catch_warnings():
    warnings.filterwarnings("ignore", "invalid value encountered", RuntimeWarning)
    # Divide by zero is ignored
    b = a / a
    # But overflow is not
    np.array([2.**100])**100

/tmp/ipykernel_33985/3074648172.py:6: RuntimeWarning: overflow encountered in power
  np.array([2.**100])**100

Take home points#

exceptions are not something to be afraid of: they are helping us to find problems in our code
there are different types of exceptions, which is useful to understand what is causing the problem. Often, an additional error message is printed too, further explaining the problem
you can raise exceptions yourself in your code.
the try ... except statements help to catch expected errors and do something about it. This is particularly useful in software which need to run whatever happens.

Scientific Programming

Errors, Exceptions and Warnings

Contents