Exceptions — handling what breaks

The problem we are solving

The last question of the previous chapter had a program process three lines correctly and stop on the fourth. Smaller, it looks like this:

numbers = ["10", "twenty", "30"]
total = 0

for text in numbers:
    total += int(text)

print(total)

ValueError: invalid literal for int() with base 10: 'twenty'

10 was added correctly. 30 was never added at all. And print(total) never ran.

One bad line stopped the whole job. Until now we have had only two options — either every piece of data is perfect, or the program stops.

numbers = ["10", "twenty", "30"]
total = 0

for text in numbers:
    try:
        total += int(text)
    except ValueError:
        print("skipping", text)

print(total)

skipping twenty
40

There is a third option: let the problem happen, and then answer it.

By the end of this chapter you can

• Catch one specific error with try / except
• Say why a bare except: or except Exception: pass is nearly always wrong
• Take hold of the error with as err and read its message
• Say what else and finally do
• raise errors of your own, and judge when to
• Recognise the family of errors, and catch one of the right size

Prerequisites: Reading and writing files.

An error is an object too

Writing as puts the error in your hands:

try:
    int("twenty")
except ValueError as err:
    print(type(err))
    print(err)

<class 'ValueError'>
invalid literal for int() with base 10: 'twenty'

The red text you have been seeing in the terminal all this time is these objects, and printing one gives its message.

There is a practical consequence: the error carries the best explanation there is. Printing err is almost always more useful than writing "something went wrong" in your own words.

Catch the right error

try:
    int("twenty")
except TypeError:
    print("caught")

ValueError: invalid literal for int() with base 10: 'twenty'

An except catches only the kind it names. This one names TypeError, and what happened was a ValueError — so the except block may as well not be there.

That can feel obstructive, and it is the useful part. The except line states which mistake you were prepared for — and whatever you were not prepared for still stops the program, which is correct.

Which is why you should not write this:

numbers = ["10", "twenty", "30"]
total = 0

for text in numbers:
    try:
        total += int(text)
    except Exception:
        pass

print(total)

40

The answer is right and the code is still bad. except Exception: pass swallows everything — the failures you thought about and the ones you did not. When numbers accidentally becomes a dictionary tomorrow, or a name inside is misspelled, this program will quietly give a wrong answer.

The rule: catch exactly the error you have a plan for. And having caught it, do something — at minimum, say what happened.

More than one kind

def get(data, key):
    try:
        return data[key]
    except KeyError:
        return "no such key"
    except IndexError:
        return "no such position"


print(get({"pen": 15}, "pen"))
print(get({"pen": 15}, "bag"))
print(get([1, 2, 3], 9))

15
no such key
no such position

Several except clauses can follow one another, and the first one that matches runs — the rest are not even looked at.

When the answer is the same they can be written together:

for value in ["10", None, "x"]:
    try:
        print(int(value))
    except (ValueError, TypeError) as err:
        print(type(err).__name__, "-", err)

10
TypeError - int() argument must be a string, a bytes-like object or a real number, not 'NoneType'
ValueError - invalid literal for int() with base 10: 'x'

type(err).__name__ gives the name of the kind of error — useful when writing logs.

Errors come in families

print(issubclass(FileNotFoundError, OSError))
print(issubclass(ValueError, Exception))
print(issubclass(KeyError, LookupError))
print(issubclass(IndexError, LookupError))

True
True
True
True

Errors are arranged as a tree, and catching a parent catches its children:

try:
    raise FileNotFoundError("gone")
except OSError as err:
    print("caught as OSError:", err)

caught as OSError: gone

A few relationships worth knowing: FileNotFoundError and PermissionError are both OSError; KeyError and IndexError are both LookupError; and nearly everything is an Exception.

From which the rule follows: catch the smallest error you can. except Exception is the easiest to write and the least useful.

else and finally

def read(text):
    try:
        value = int(text)
    except ValueError:
        print("bad:", text)
        return None
    else:
        print("good:", value)
        return value
    finally:
        print("done with", text)


print(read("10"))
print(read("ten"))

good: 10
done with 10
10
bad: ten
done with ten
None

else runs when nothing in the try block broke. It lets the inside of try stay small — only the line that can fail — with the work that follows success in else. Put too much inside try and you will catch things you never meant to.

finally always runs, whatever happened. Even after a return:

def risky():
    try:
        return "from try"
    finally:
        print("finally still runs")


print(risky())

finally still runs
from try

The return has settled the value, but finally runs before the function is left. It is the place for cleaning up — although for files, with has already done that job.

Raising your own

Throwing matters as much as catching.

def line_total(price, quantity):
    if quantity < 0:
        raise ValueError(f"quantity cannot be negative: {quantity}")
    return price * quantity


print(line_total(15.0, 3))
print(line_total(15.0, -1))

45.0
ValueError: quantity cannot be negative: -1

What was the alternative to raise? Returning None, or 0 — and then the mistake would travel. It would work its way into some total and surface much later as a strange number with no traceable origin.

An error stops the mistake where it was born, and says why in its message.

When writing that message, put the offending value inside it. "invalid quantity" and "quantity cannot be negative: -1" — only the second one lets you start work.

An error can also be caught and thrown again with a better message:

def parse(text):
    try:
        return int(text)
    except ValueError:
        raise ValueError(f"not a number: {text!r}")


print(parse("10"))
print(parse("ten"))

10
ValueError: not a number: 'ten'

{text!r} inserts the value through repr, so the quotes show — and the difference between not a number: 'ten' and not a number: ten matters when the value is really empty text or a stray space.

Asking first, or apologising after

prices = {"pen": 15}

if "bag" in prices:
    print(prices["bag"])
else:
    print("not stocked")

try:
    print(prices["bag"])
except KeyError:
    print("not stocked")

not stocked
not stocked

Both are right. The second is more common in Python, and for a reason: the first looks twice — once in in, once in [...] — and can still break if something changes in between.

For what happens routinely, though, an if reads better. Exceptions are for the exceptional; the name says so.

A complete example

orders.txt, with two deliberately bad lines:

pen,15.0,3
bag,eight,1

ink,120.0,2
clip,5.0

main.py:

"""Read an order file, and keep going when a line is wrong."""

from pathlib import Path

TAX_RATE = 0.15


def parse_line(raw, number):
    """Returns one order line, or raises ValueError saying which line was wrong."""
    parts = raw.split(",")
    if len(parts) != 3:
        raise ValueError(f"line {number}: expected 3 fields, got {len(parts)}")

    name, price, quantity = parts
    try:
        return name, float(price), int(quantity)
    except ValueError as err:
        raise ValueError(f"line {number}: {err}")


def read_order(path):
    """Returns the good lines and the complaints, never raising for bad data."""
    try:
        text = path.read_text(encoding="utf-8")
    except FileNotFoundError:
        return [], [f"no such file: {path}"]

    lines = []
    problems = []

    for number, raw in enumerate(text.splitlines(), start=1):
        if not raw.strip():
            continue
        try:
            lines.append(parse_line(raw, number))
        except ValueError as err:
            problems.append(str(err))

    return lines, problems


def main():
    lines, problems = read_order(Path("orders.txt"))

    total = 0.0
    for name, price, quantity in lines:
        amount = round(price * quantity * (1 + TAX_RATE), 2)
        total += amount
        print(f"{name:<6} {amount:>9.2f}")

    print(f"{'total':<6} {total:>9.2f}")

    print()
    print(f"{len(lines)} lines used, {len(problems)} skipped")
    for problem in problems:
        print(" -", problem)


if __name__ == "__main__":
    main()

pen        51.75
ink       276.00
total     327.75

2 lines used, 2 skipped
 - line 2: could not convert string to float: 'eight'
 - line 5: expected 3 fields, got 2

Four things worth looking at.

parse_line throws and read_order catches. The lower function has no idea what the program as a whole should do — it only knows this line is unusable. Deciding what to do about that belongs to the function above. Errors are raised low and caught high, and that division is the whole job of an exception system.

The line number is inside the message. could not convert string to float: 'eight' does not say where to look; line 2: ... does. Whatever you know at the moment of catching and will not know later belongs in the message.

read_order never breaks. It returns two things — what was usable and what was not. A missing file produces an answer of the same shape, so main does not need two separate paths.

The inner try block is exactly one line long. float(price) and int(quantity) — only the part that can fail is inside it. parts = raw.split(",") is outside, because it is not supposed to fail, and if it does I want to know.

When it breaks

The error is not being caught, although an except is there The kind does not match. The first word of the message is the name of the error — name that one, or catch its parent.

There is an except and the program still stops The failing line is not inside the try block. Check the indentation.

SyntaxError: expected 'except' or 'finally' block A try was written with no except. try cannot stand alone.

Everything "works" but the answers are wrong There is an except Exception: pass somewhere. Find it — that is where the problem is.

I raised something but no message appears raise ValueError was written rather than raise ValueError("..."). Add the brackets and the message.

A return inside finally loses the error It does, and that is why you should not write one there. finally is for cleaning up only.

Your turn

Go back to the previous chapter's summary.py, which read people.txt.

Now make it so it never crashes, and finishes by saying which lines were skipped and why. Write a parse_line(raw, number) that raises a ValueError with the line number in it for a bad line.

Then damage the file on purpose — remove a comma from one line, put letters where a number belongs in another, leave one line entirely blank — and check that the program runs to the end with four complaints.

Then four experiments:

• Replace except ValueError with except Exception: pass. What is the output, and why is it more dangerous?
• Remove the line number from parse_line's message. How long does finding a bad line take now?
• Make the try block bigger — put the whole body of the loop inside it. What happens now when one line is bad?
• Add a finally that prints something after every line, and check that it runs on the good ones and the bad ones alike.

The third experiment teaches this chapter's most useful habit: the smaller the try block, the more honest the catch.