Python's exec(): Execute Dynamically Generated Code

Getting to Know Python’s exec()

Python’s built-in exec() function allows you to execute any piece of Python code. With this function, you can execute dynamically generated code. That’s the code that you read, auto-generate, or obtain during your program’s execution. Normally, it’s a string.

The exec() function takes a piece of code and executes it as your Python interpreter would. Python’s exec() is like eval() but even more powerful and prone to security issues. While eval() can only evaluate expressions, exec() can execute sequences of statements, as well as imports, function calls and definitions, class definitions and instantiations, and more. Essentially, exec() can execute an entire fully featured Python program.

The signature of exec() has the following form:

exec(code [, globals [, locals]])

The function executes code, which can be either a string containing valid Python code or a compiled code object.

If code is a string, then it’s parsed as a suite of Python statements, which is then internally compiled into bytecode, and finally executed, unless a syntax error occurs during the parsing or compilation step. If code holds a compiled code object, then it’s executed directly, making the process a bit more efficient.

The globals and locals arguments allow you to provide dictionaries representing the global and local namespaces in which exec() will run the target code.

The exec() function’s return value is None, probably because not every piece of code has a final, unique, and concrete result. It may just have some side effects. This behavior notably differs from eval(), which returns the result of the evaluated expression.

To get an initial feeling of how exec() works, you can create a rudimentary Python interpreter with two lines of code:

>>> while True:
...     exec(input("->> "))
...

->> print("Hello, World!")
Hello, World!

->> import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
    ...

->> x = 10
->> if 1 <= x <= 10: print(f"{x} is between 1 and 10")
10 is between 1 and 10

In this example, you use an infinite while loop to mimic the behavior of a Python interpreter or REPL. Inside the loop, you use input() to get the user’s input at the command line. Then you use exec() to process and run the input.

This example showcases what’s arguably the main use case of exec(): executing code that comes to you as a string.

You’ll commonly use exec() when you need to dynamically run code that comes as a string. For example, you can write a program that generates strings containing valid Python code. You can build these strings from parts that you obtain at different moments in your program’s execution. You can also use the user’s input or any other input source to construct these strings.

Once you’ve built the target code as strings, then you can use exec() to execute them as you would execute any Python code.

In this situation, you can rarely be certain of what your strings will contain. That’s one reason why exec() implies serious security risks. This is particularly true if you’re using untrusted input sources, like a user’s direct input, in building your code.

In programming, a function like exec() is an incredibly powerful tool because it allows you to write programs that generate and execute new code dynamically. To generate this new code, your programs will use information available at runtime only. To run the code, your programs will use exec().

However, with great power comes great responsibility. The exec() function implies serious security risks, as you’ll learn soon. So, you should avoid using exec() most of the time.

In the following sections, you’ll learn how exec() works and how to use this function for executing code that comes as strings or as compiled code objects.

Running Code From a String Input

The most common way to call exec() is with code that comes from a string-based input. To build this string-based input, you can use:

• Single lines of code or one-liner code snippets
• Multiple lines of code separated by semicolons
• Multiple lines of code separated by newline characters
• Multiple lines of code within triple-quoted strings and with proper indentation

A one-liner program consists of a single line of code that performs multiple actions at once. Say that you have a sequence of numbers, and you want to build a new sequence containing the sum of squares of all the even numbers in an input sequence.

To solve this problem, you can use the following one-liner code:

>>> numbers = [2, 3, 7, 4, 8]

>>> sum(number**2 for number in numbers if number % 2 == 0)
84

In the highlighted line, you use a generator expression to compute the square value of all the even numbers in the input sequence of values. Then you use sum() to compute the total sum of squares.

To run this code with exec(), you just need to transform your one-liner code into a single-line string:

>>> exec("result = sum(number**2 for number in numbers if number % 2 == 0)")
>>> result
84

In this example, you express the one-liner code as a string. Then you feed this string into exec() for execution. The only difference between your original code and the string is that the latter stores the computation result in a variable for later access. Remember that exec() returns None rather than a concrete execution result. Why? Because not every piece of code has a final unique result.

Python allows you to write multiple statements in a single line of code, using semicolons to separate them. Even though this practice is discouraged, nothing will stop you from doing something like this:

>>> name = input("Your name: "); print(f"Hello, {name}!")
Your name: Leodanis
Hello, Leodanis!

You can use semicolons to separate multiple statements and build a single-line string that serves as an argument to exec(). Here’s how:

>>> exec("name = input('Your name: '); print(f'Hello, {name}!')")
Your name: Leodanis
Hello, Leodanis!

The idea of this example is that you can combine multiple Python statements into a single-line string by using semicolons to separate them. In the example, the first statement takes the user’s input, while the second statement prints a greeting message to the screen.

You can also aggregate multiple statements in a single-line string using the newline character, \n:

>>> exec("name = input('Your name: ')\nprint(f'Hello, {name}!')")
Your name: Leodanis
Hello, Leodanis!

The newline character makes exec() understand your single-line string as a multiline set of Python statements. Then exec() runs the aggregated statements in a row, which works like a multiline code file.

The final approach to building a string-based input for feeding exec() is to use triple-quoted strings. This approach is arguably more flexible and allows you to generate string-based input that looks and works like normal Python code.

It’s important to note that this approach requires you to use proper indentation and code formatting. Consider the following example:

>>> code = """
... numbers = [2, 3, 7, 4, 8]
...
... def is_even(number):
...     return number % 2 == 0
...
... even_numbers = [number for number in numbers if is_even(number)]
...
... squares = [number**2 for number in even_numbers]
...
... result = sum(squares)
...
... print("Original data:", numbers)
... print("Even numbers:", even_numbers)
... print("Square values:", squares)
... print("Sum of squares:", result)
... """

>>> exec(code)
Original data: [2, 3, 7, 4, 8]
Even numbers: [2, 4, 8]
Square values: [4, 16, 64]
Sum of squares: 84

In this example, you use a triple-quoted string to provide the input to exec(). Note that this string looks like any regular piece of Python code. It uses appropriate indentation, naming style, and formatting. The exec() function will understand and execute this string as a regular Python code file.

You should note that when you pass a string with code to exec(), the function will parse and compile the target code into Python bytecode. In all cases, the input string should contain valid Python code.

If exec() finds any invalid syntax during the parsing and compilation steps, then the input code won’t run:

>>> exec("print('Hello, World!)")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
    File "<string>", line 1
      print('Hello, World!)
            ^
SyntaxError: unterminated string literal (detected at line 1)

In this example, the target code contains a call to print() that takes a string as an argument. This string isn’t properly ended with a closing single quotation mark, so exec() raises a SyntaxError pointing out the issue and doesn’t run the input code. Note that Python pinpoints the error at the start of the string rather than at the end, where the closing single quotation mark should go.

Running code that comes as a string, like you did in the example above, is arguably the natural way of using exec(). However, if you need to run the input code many times, then using a string as an argument will make the function run the parsing and compilation steps every time. This behavior can make your code inefficient in terms of execution speed.

In this situation, the most convenient approach is to compile the target code beforehand and then run the resulting bytecode with exec() as many times as needed. In the following section, you’ll learn how to use exec() with compiled code objects.

Executing Compiled Code

In practice, exec() can be quite slow when you use it to process strings containing code. If you ever need to dynamically run a given piece of code more than once, then compiling it beforehand will be the most performant and recommended approach. Why? Because you’ll be running the parsing and compilation steps only once and then reusing the compiled code.

To compile a piece of Python code, you can use compile(). This built-in function takes a string as an argument and runs a one-time bytecode compilation on it, generating a code object that you can then pass to exec() for execution.

The signature of compile() has the following form:

compile(source, filename, mode, flags=0, dont_inherit=False, optimize=-1)

In this tutorial, you’ll only use the three first arguments to compile(). The source argument holds the code that you need to compile into bytecode. The filename argument will hold the file from which the code was read. To read from a string object, you’ll have to set filename to the "<string>" value.

Finally, compile() can generate code objects that you can execute using either exec() or eval(), depending on the mode argument’s value. This argument should be set to "exec" or "eval", depending on the target execution function:

>>> string_input = """
... def sum_of_even_squares(numbers):
...     return sum(number**2 for number in numbers if number % 2 == 0)
...
... print(sum_of_even_squares(numbers))
... """

>>> compiled_code = compile(string_input, "<string>", "exec")
>>> exec(compiled_code)

>>> numbers = [2, 3, 7, 4, 8]
>>> exec(compiled_code)
84

>>> numbers = [5, 3, 9, 6, 1]
>>> exec(compiled_code)
36

Compiling often-repeated code up front with compile() can help you slightly improve your code’s performance by skipping the parsing and bytecode compilation steps on each call to exec().

Running Code From Python Source Files

You can also use exec() to run code that you’ve read from a reliable .py file in your file system or somewhere else. To do this, you can use the built-in open() function to read the file’s content as a string, which you can then pass as an argument to exec().

For example, say that you have a Python file named hello.py containing the following code:

# hello.py

print("Hello, Pythonista!")
print("Welcome to Real Python!")

def greet(name="World"):
    print(f"Hello, {name}!")

This sample script prints a greeting and a welcome message to the screen. It also defines a sample greet() function for testing purposes. The function takes a name as an argument and prints a customized greeting to the screen.

Now get back to your Python interactive session and run the following code:

>>> with open("hello.py", mode="r", encoding="utf-8") as hello:
...     code = hello.read()
...

>>> exec(code)
Hello, Pythonista!
Welcome to Real Python!

>>> greet()
Hello, World!

>>> greet("Pythonista")
Hello, Pythonista!

In this example, you first open the target .py file as a regular text file using the built-in open() function in a with statement. Then you call .read() on the file object to read the file’s content into the code variable. This call to .read() returns the file’s content as a string. The final step is to call exec() with this string as an argument.

This example runs the code and makes the greet() function and objects that live in hello.py available in your current namespace. That’s why you can use greet() directly. The secret behind this behavior has to do with the globals and locals arguments, which you’ll learn about in the next section.

Using the technique in the above example, you can open, read, and execute any file containing Python code. This technique may work when you don’t know up front which source files you’ll be running. So, you can’t write import module, because you don’t know the module’s name when you’re writing the code.

If you ever choose to use this technique, then make sure that you only execute code from trusted source files. Ideally, the most reliable source files are those that you have consciously created to run dynamically. You must never run code files that come from external sources, including your users, without inspecting the code first.