closures

Introduction

You should first study the topic first-class functions

Functions that are defined inside other functions are called local functions (local functions)

They are defined after their def .

A new copy of the function object is created on every call. A separate name - function object binding every time.

Local functions are not members of their parent functions.

The call parent.local() will not work .

A closure in programming is a first-class function , in whose body there are references to variables declared outside the body of this function in the surrounding code and which are not its parameters.

In other words, a closure is a function that references free variables in its scope.

A closure, just like an object instance, is a way of representing functionality and data that are bound and packaged together.

A closure is a special kind of function. It is defined in the body of another function and is created every time that function runs.

Syntactically it looks like a function that sits entirely inside the body of another function. At the same time the nested inner function contains references to the local variables of the outer function.

Every time the outer function runs, a new instance of the inner function is created, with new references to the outer function's variables.

In the case of a closure, the references to the outer function's variables remain valid inside the nested function for as long as the nested function runs, even if the outer function has finished and the variables have gone out of scope.

A closure binds the function's code to its lexical environment (the place where it is defined in the code). A closure's lexical variables differ from global variables in that they do not occupy the global namespace. They differ from variables in objects in that they are bound to functions, not to objects.

Closures keep objects from the parent scope so that functions returned from other functions can work.

Among other things, you can create functions that remember state.

Example

Create a file closures.py

Inside it you need to create an outer function with a free variable message, and an inner function that will use the free variable.

def outer_func(): message = 'Hi' def inner_func(): print(message) return inner_func() outer_func()

python closures.py

Hi

Let's change the code

def outer_func(): message = 'Hi' def inner_func(): print(message) return inner_func outer_func()

python closures.py

'Hi' was not printed, because the inner function is not executed but only returned

ef outer_func(): message = 'Hi' def inner_func(): print(message) return inner_func my_func = outer_func() print(my_func) print(my_func.__name__) my_func()

python closures.py

<function outer_func.<locals>.inner_func at 0x7f6b693d9a60> inner_func Hi

def outer_func(msg): message = msg def inner_func(): print(message) return inner_func hi_func = outer_func('Hi') hello_func = outer_func('Hello') print(hi_func) print(hi_func.__name__) hi_func() print(hello_func) print(hello_func.__name__) hello_func()

python closures.py

<function outer_func.<locals>.inner_func at 0x7fce92122a60> inner_func Hi <function outer_func.<locals>.inner_func at 0x7fce92122af0> inner_func Hello

The code can be shortened by removing message

def outer_func(msg): def inner_func(): print(msg) return inner_func hi_func = outer_func('Hi') hello_func = outer_func('Hello') print(hi_func) print(hi_func.__name__) hi_func() hello_func()

Example

Let's look at an example of using a closure for logging. About the logging module, read here

import logging logging.basicConfig(filename='example.log', level=logging.INFO) def logger(func): def log_func(*args): logging.info('Running "{}" with arguments {}'.format(func.__name__, args)) print(func(*args)) return log_func def add(x, y): return x+y def sub(x, y): return x-y add_logger = logger(add) sub_logger = logger(sub) add_logger(3, 3) add_logger(4, 5) sub_logger(10, 5) sub_logger(20, 10)

python closures.py

6 9 5 10

cat example.log

INFO:root:Running "add" with arguments (3, 3) INFO:root:Running "add" with arguments (4, 5) INFO:root:Running "add" with arguments (3, 3) INFO:root:Running "add" with arguments (4, 5) INFO:root:Running "sub" with arguments (10, 5) INFO:root:Running "sub" with arguments (20, 10)

Example

Sorting strings by the last character

def sort_by_last_letter(strings): def last_letter(s): return s[-1] return sorted(strings, key=last_letter) cities = ["Helsinki", "Malaga", "Torremolinos"] sorted_cities = sort_by_last_letter(cities) print(sorted_cities)

['Malaga', 'Helsinki', 'Torremolinos']

Let's make a small change to the sort_by_last_letter() function to make sure that a new object is created each time.

def sort_by_last_letter(strings): def last_letter(s): return s[-1] print(last_letter) return sorted(strings, key=last_letter) cities = ["Helsinki", "Malaga", "Torremolinos"] sorted_cities = sort_by_last_letter(cities) print(sorted_cities)

g = "global" def outer(p="param"): l = "local" def inner(): print(g, p, l) inner() outer()

global param local

python
outer.inner()

Traceback (most recent call last): File "/home/andrei/python/outer.py", line 12, in <module> outer.inner() AttributeError: 'function' object has no attribute 'inner'

__closure__

How does a local function that has been returned retain access to the scope of its parent function?

Right after a function returns, the scope surrounding it should disappear.

This is achieved with the help of closures.

Objects from the surrounding scope are recorded and kept after it has disappeared. These objects are protected from the garbage collector.

Closures are implemented with the help of the attribute __closure__ in which references to the required objects are stored.

def enclosing(): x = 'closed over' def local_func(): print(x) return local_func lf = enclosing() lf() print(lf.__closure__)

python enclosing.py

closed over (<cell at 0x7f5ec5f47be0: str object at 0x7f5ec5eba970>,)

You can see that x, which was in the scope of the parent function, is stored in the __closure__ attribute of the local function that was returned.

Function factories

From the previous paragraphs it becomes obvious that you can write functions that return other functions, which will have both their own internal arguments and arguments from the parent function through closures.

A kind of combination of runtime function definition and a closure.

Let's look at a function factory that raises a number to a given power.

The desired power is passed into the factory and a function that raises to that power is returned.

def raise_to(exp): def raise_to_exp(x): return pow(x, exp) return raise_to_exp square = raise_to(2) print(square.__closure__) print(square(5)) # 25 print(square(11)) # 121 cube = raise_to(3) print(cube.__closure__) print(cube(3)) # 27 print(cube(10)) # 1000

python3 raise_to.py (<cell at 0x7f7cf6e01be0: int object at 0x958e40>,) 25 121 (<cell at 0x7f7cf6e01bb0: int object at 0x958e60>,) 27 1000

Scopes

Let's continue the discussion of variable scopes using closures as an example.

message = 'global' def enclosing(): message = 'enclosing' def local(): message = 'local' print('enclosing message:', message) local() print('enclosing message:', message) print('global message:', message) enclosing() print('global message:', message)

python3 scopes.py

global message: global enclosing message: enclosing enclosing message: enclosing global message: global

Calling enclosing() and local() does not change message, because inside local() a new binding is created between the name and the function body

In local functions you can use the global keyword - then the variable defined at module level will be changed.

message = 'global' def enclosing(): message = 'enclosing' def local(): global message message = 'local' print('enclosing message:', message) local() print('enclosing message:', message) print('global message:', message) enclosing() print('global message:', message)

python3 scopes.py

global message: global enclosing message: enclosing enclosing message: enclosing global message: local

To change, from the local() function, a variable from enclosing() that is neither local nor global, you need the nonlocal keyword.

nonlocal will look for a match "from the bottom up", that is, skipping local variables, it will go and look in Enclosing and, if it does not find it there, in the global ones. The first match found will be used.

message = 'global' def enclosing(): message = 'enclosing' def local(): nonlocal message message = 'local' print('enclosing message:', message) local() print('enclosing message:', message) print('global message:', message) enclosing() print('global message:', message)

python3 scopes.py

global message: global enclosing message: enclosing enclosing message: local global message: global

Let's look at another example of using nonlocal

import time def make_timer(): last_called = None # Never def elapsed(): nonlocal last_called now = time.time() if last_called is None: last_called = now return None result = now - last_called last_called = now return result return elapsed t = make_timer() print(t()) print(t()) print(t())

python make_timer.py

None 2.2411346435546875e-05 1.5735626220703125e-05

elapsed() uses the nonlocal variable last_called in order to store data there between calls to elapsed().

Each new call to make_timer() creates a new assignment of the value (new binding) last_called.

To demonstrate this feature, let's add a one-second delay and make two function objects.

import time def make_timer(): last_called = None # Never def elapsed(): time.sleep(1) nonlocal last_called now = time.time() if last_called is None: last_called = now return None result = now - last_called last_called = now return result return elapsed t1 = make_timer() t2 = make_timer() print("t1():", t1()) print("t1():", t1()) print("t2():", t2()) print("t1():", t1()) print("t1():", t1()) print("t1():", t1()) print("t1():", t1()) print("t1():", t1()) print("t2():", t2()) print("t2():", t2()) print("t1():", t1())

python make_timer.py

t1(): None t1(): 1.0011835098266602 t2(): None t1(): 2.002288579940796 t1(): 1.0011866092681885 t1(): 1.0011425018310547 t1(): 1.0012156963348389 t1(): 1.0011701583862305 t2(): 6.0070576667785645 t2(): 1.0012218952178955 t1(): 3.0036113262176514

You can see that the calls t1() and t2() do not affect each other.

Article author: Andrei Olegovich