PEP 572 and The Walrus Operator¶
Abstract¶
PEP 572 is one of the most contentious proposals in Python3 history because
assigning a value within an expression seems unnecessary. Also, it is ambiguous
for developers to distinguish the difference between the walrus operator
(:=) and the equal operator (=). Even though sophisticated developers
can use “:=” smoothly, they may concern the readability of their code. To
better understand the usage of “:=,” this article discusses its design
philosophy and what kind of problems it tries to solve.
Introduction¶
For C/C++ developer, assigning a function return to a variable is common due
to error code style handling. Managing function errors includes two steps;
one is to check the return value; another is to check errno. For example,
#include <stdio.h>
#include <unistd.h>
#include <string.h>
#include <errno.h>
int main(int argc, char *argv[]) {
int rc = -1;
// assign access return to rc and check its value
if ((rc = access("hello_walrus", R_OK)) == -1) {
fprintf(stderr, "%s", strerror(errno));
goto end;
}
rc = 0;
end:
return rc;
}
In this case, access will assign its return value to the variable rc
first. Then, the program will compare the rc value with -1 to check
whether the execution of access is successful or not. However, Python did
not allow assigning values to variables within an expression before 3.8. To fix
this problem, therefore, PEP 572 introduced the walrus operator for developers.
The following Python snippet is equal to the previous C example.
>>> import os
>>> from ctypes import *
>>> libc = CDLL("libc.dylib", use_errno=True)
>>> access = libc.access
>>> path = create_string_buffer(b"hello_walrus")
>>> if (rc := access(path, os.R_OK)) == -1:
... errno = get_errno()
... print(os.strerror(errno), file=sys.stderr)
...
No such file or directory
Why := ?¶
Developers may confuse the difference between “:=” and “=.” In fact, they
serve the same purpose, assigning somethings to variables. Why Python introduced
“:=” instead of using “=”? What is the benefit of using “:=”? One
reason is to reinforce the visual recognition due to a common mistake made by
C/C++ developers. For instance,
int rc = access("hello_walrus", R_OK);
// rc is unintentionally assigned to -1
if (rc = -1) {
fprintf(stderr, "%s", strerror(errno));
goto end;
}
Rather than comparison, the variable, rc, is mistakenly assigned to -1. To
prevent this error, some people advocate using Yoda conditions within an
expression.
int rc = access("hello_walrus", R_OK);
// -1 = rc will raise a compile error
if (-1 == rc) {
fprintf(stderr, "%s", strerror(errno));
goto end;
}
However, Yoda style is not readable enough like Yoda speaks non-standardized English. Also, unlike C/C++ can detect assigning error during the compile-time via compiler options (e.g., -Wparentheses), it is difficult for Python interpreter to distinguish such mistakes throughout the runtime. Thus, the final result of PEP 572 was to use a new syntax as a solution to implement assignment expressions.
The walrus operator was not the first solution for PEP 572. The original proposal
used EXPR as NAME to assign values to variables. Unfortunately, there are
some rejected reasons in this solution and other solutions as well. After
intense debates, the final decision was :=.
Scopes¶
Unlike other expressions, which a variable is bound to a scope, an assignment expression belongs to the current scope. The purpose of this design is to allow a compact way to write code.
>>> if not (env := os.environ.get("HOME")):
... raise KeyError("env HOME does not find!")
...
>>> print(env)
/root
In PEP 572, another benefit is to conveniently capture a “witness” for an
any() or an all() expression. Although capturing function inputs can
assist an interactive debugger, the advantage is not so obvious, and examples
lack readability. Therefore, this benefit does not discuss here. Note that
other languages (e.g., C/C++ or Go) may bind an assignment to a scope. Take
Golang as an example.
package main
import (
"fmt"
"os"
)
func main() {
if env := os.Getenv("HOME"); env == "" {
panic(fmt.Sprintf("Home does not find"))
}
fmt.Print(env) // <--- compile error: undefined: env
}
Pitfalls¶
Although an assigning expression allows writing compact code, there are many
pitfalls when a developer uses it in a list comprehension. A common SyntaxError
is to rebind iteration variables.
>>> [i := i+1 for i in range(5)] # invalid
However, updating an iteration variable will reduce readability and introduce bugs. Even if Python 3.8 did not implement the walrus operator, a programmer should avoid reusing iteration variables within a scope.
Another pitfall is Python prohibits using assignment expressions within a comprehension under a class scope.
>>> class Example:
... [(j := i) for i in range(5)] # invalid
...
This limitation was from bpo-3692. The interpreter’s behavior is unpredictable when a class declaration contains a list comprehension. To avoid this corner case, assigning expression is invalid under a class.
>>> class Foo:
... a = [1, 2, 3]
... b = [4, 5, 6]
... c = [i for i in zip(a, b)] # b is defined
...
>>> class Bar:
... a = [1,2,3]
... b = [4,5,6]
... c = [x * y for x in a for y in b] # b is undefined
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in Bar
File "<stdin>", line 4, in <listcomp>
NameError: name 'b' is not defined
Conclusion¶
The reason why the walrus operator (:=) is so controversial is that code
readability may decrease. In fact, in the discussion mail thread,
the author of PEP 572, Christoph Groth, had considered using “=” to implement
inline assignment like C/C++. Without judging “:=” is ugly, many developers
argue that distinguishing the functionality between “:=” and “=” is
difficult because they serve the same purpose, but behaviors are not consistent.
Also, writing compact code is not persuasive enough because smaller is not
always better. However, in some cases, the walrus operator can enhance
readability (if you understand how to use :=). For example,
buf = b""
while True:
data = read(1024)
if not data:
break
buf += data
By using :=, the previous example can be simplified.
buf = b""
while (data := read(1024)):
buf += data
Python document and GitHub issue-8122 provides many great examples about
improving code readability by “:=”. However, using the walrus operator
should be careful. Some cases, such as foo(x := 3, cat='vector'), may
introduce new bugs if developers are not aware of scopes. Although PEP 572
may be risky for developers to write buggy code, an in-depth understanding of
design philosophy and useful examples will help us use it to write readable
code at the right time.
