The with statement

The purpose of the with statement is to make sure that something always gets done after executing a block of indented statements. For example, we can make sure that an input file always gets closed.

Documentation

  1. Predefined Clean-up Actions in the Python Tutorial. “Objects which, like files, provide predefined clean-up actions” are called context managers.
  2. Definitions in the Python Glossary:
    1. context manager
    2. asynchronous context manager
  3. with statement in the Python Language Reference
  4. with Statement Context Managers
    1. __enter__
    2. __exit__
  5. Context Manager Types
  6. Examples of context managers in the Python Standard Library:
    1. Any object of class io.Base is a context manager, so all objects of classes derived from io.Base (e.g., classes io.TextIOBase, io.TextIOWrapper) are also context managers.
    2. An object returned by the function open is a context manager because it is of a class derived from class io.Base.
    3. An object returned by the function urllib.request.urlopen is a context manager.
    4. An object returned by the assertRaises method of a unit tester is a context manager.
    5. An object returned by the decimal.localcontext function is a context manager.
    6. The threading module creates objects that can be used as context managers. See Using locks, conditions, and semaphores in the with statement.
  7. The contextlib module defines the @contextlib.contextmanager decorator.

The purpose of the with statement

This file is numbers.txt. 

10
20
30

The following two programs make no attempt to catch any exceptions raised by the open function. If the open failed, then the input file was never opened and there is nothing we need to close. But both programs will close the input file. lines is a context manager whose __exit__ method closes the file. 

"Read a text file of ints, one per line, and print their sum."

import sys

lines = open("numbers.txt")
sum = 0

for line in lines:
    try:
        sum += int(line)
    except ValueError as error:
        lines.close()
        print(error, file = sys.stderr)
        sys.exit(1)

lines.close()
print(f"sum = {sum}")
sys.exit(0)

sum = 60

"Read a text file of ints, one per line, and print their sum."

import sys

sum = 0

with open("numbers.txt") as lines:
    for line in lines:
        sum += int(line)

print(f"Has the input file been closed? {lines.closed}.")
print(f"sum = {sum}")
sys.exit(0)

Has the input file been closed? True.
sum = 60

Note that we could have computed the sum more simply with a list comprehension: 

"Read a text file of ints, one per line, and print their sum."

import sys

with open("numbers.txt") as lines:
    sum = sum([int(line) for line in lines])

print(f"Has the input file been closed? {lines.closed}.")
print(f"sum = {sum}")
sys.exit(0)

Has the input file been closed? True.
sum = 60

Class io.TextIOWrapper is a subclass of class contextlib.AbstractContextManager

import sys
import contextlib

lines = open("numbers.txt")
print(f"type(lines) = {type(lines)}")

if isinstance(lines, contextlib.AbstractContextManager):
    print("lines is an instance of class contextlib.AbstractContextManager.")

if issubclass(type(lines), contextlib.AbstractContextManager):
    print("lines belongs to a class that is a subclass of contextlib.AbstractContextManager.")

if "__enter__" in dir(lines) and "__exit__" in dir(lines):
    print("lines has methods named __enter__ and __exit__.")

lines.close()
sys.exit(0)

type(lines) = <class '_io.TextIOWrapper'>
lines is an instance of class contextlib.AbstractContextManager.
lines belongs to a class that is a subclass of contextlib.AbstractContextManager.
lines has methods named __enter__ and __exit__.

Create a context manager

A context manager is an object that has two methods named __enter__ and __exit__. The value of the expression after the keyword with must be a context manager. The context manager is usually a new context manager created by this expression. The context manager is then saved in the variable after the keyword as.

The __enter__ method of the context manager is automatically called before executing the indented block of statements under the with statement. The __exit__ method of the context manager is automatically called after executing the indented block of statements under the with statement. 

import sys

class ContextManager(object):
    def __enter__(self):
        print("About to enter the block.")

    def __exit__(self, exc_type, exc_value, traceback):
        print("Just exited from the block.")

with ContextManager():
    print("\tFirst statement of the block.")
    print("\tLast statement of the block.") 

print("All done.")
sys.exit(0)

About to enter the block.
	First statement of the block.
	Last statement of the block.
Just exited from the block.
All done.

A context manager that is used by the statements in the block

Suppose the cntext manager has additional methods and properties (e.g., the following method f) that the block wants to use. The return value of the __enter__ method can be stored in the variable after the keyword as. 

import sys

class ContextManager(object):
    def __enter__(self):
        print("About to enter the block.")
        return self

    def __exit__(self, exc_type, exc_value, traceback):
        print("Just exited from the block.")

    def f(self):
        print("\tUsing the context manager in the block.")

with ContextManager() as contextManager:
    print("\tFirst statement of the block.")
    contextManager.f()                      
    print("\tLast statement of the block.") 

print("All done.")
sys.exit(0)

About to enter the block.
	First statement of the block.
	Using the context manager in the block.
	Last statement of the block.
Just exited from the block.
All done.

A context manager that catches exceptions raised by the block

Even if the indented block of statements raises an exception and is not completely executed, the __exit__ method of the context manager is still called. 

import sys
import random

class ContextManager(object):
    def __enter__(self):
        print("About to enter the block.")
        return self

    def __exit__(self, exc_type, exc_value, traceback):
        print("Just exited from the block.")
        print(f"exc_type = {exc_type}")
        print(f"exc_value = {exc_value}")
        print(f"traceback = {traceback}")
        return True   #Finished handling the exception.

    def f(self):
        print("\tUsing the context manager in the block.")

with ContextManager() as contextManager:
    print("\tFirst statement of the block.")           
    contextManager.f()                                 
    if random.randrange(2) == 0:   #fifty-fifty chance.
        raise ValueError("unlucky random number")      
    print("\tLast statement of the block.")            

print("All done.")
sys.exit(0)

About to enter the block.
	First statement of the block.
	Using the context manager in the block.
	Last statement of the block.
Just exited from the block.
exc_type = None
exc_value = None
traceback = None
All done.

About to enter the block.
	First statement of the block.
	Using the context manager in the block.
Just exited from the block.
exc_type = <class 'ValueError'>
exc_value = unlucky random number
traceback = <traceback object at 0x1077b6888>
All done.

Propagate the exception

In the above program, the __exit__ method catches and prints any exception raised by the block. Nothing further is done with the exception because __exit__ returns True to indicate that the exception has been fully handled.

Now change the True to False. 

About to enter the block.
	First statement of the block.
	Using the context manager in the block.
	Last statement of the block.
Just exited from the block.
exc_type = None
exc_value = None
traceback = None
All done.

About to enter the block.
	First statement of the block.
	Using the context manager in the block.
Just exited from the block.
exc_type = <class 'ValueError'>
exc_value = unlucky random number
traceback = <traceback object at 0x109fa9948>
Traceback (most recent call last):
  File "/Users/myname/python/junk2.py", line 23, in <module>
    raise ValueError("unlucky random number")
ValueError: unlucky random number