Why doesn't exec() create objects in local/global scope?

Question

I'm trying to find an IPython counterpart to Spyder's runfile. According to this page, "exec() will execute the input code in the current scope" by default. Therefore, I expect the following to create objects TestFunc and Doggy in the current scope:

# Script.py
#----------
def TestFunc():
    printf("I am TestFunc.")
Doggy = "Doggy"

To "source" the code from the IPython REPL, I found the following function from this tutorial, which I paste into the REPL:

def execute_python_file_with_exec(file_path):
    try:
        with open(file_path, 'r') as file:
            code = file.read()
            exec(code)
    except FileNotFoundError:
        print(f"Error: The file '{file_path}' does not exist.")
    except Exception as e:
        print(f"An error occurred: {e}")

I then use it to run Script.py and query the local and global namespace:

execute_python_file_with_exec('Script.py')

print("Locals:")
for item in dir():
    print(  item, end=", " )

print("Globals:")
for item in globals():
    print(  item, end=", " )

Neither of the namespaces contain TestFunc or Doggy.

Locals:
In, Out, _, _2, _5, _6, __, ___, __builtin__,
__builtins__, __doc__, __loader__, __name__,
__package__, __spec__, _dh, _i, _i1, _i2, _i3,
_i4, _i5, _i6, _i7, _i8, _ih, _ii, _iii, _oh,
execute_python_file_with_exec, exit,
get_ipython, item, open,

Globals:
__name__, __doc__, __package__, __loader__,
__spec__, __builtin__, __builtins__, _ih, _oh,
_dh, In, Out, get_ipython, exit, quit, open,
_, __, ___, _i, _ii, _iii, _i1,
execute_python_file_with_exec, _i2, _2, _i3,
_i4, _i5, _5, _i6, _6, _i7, item, _i8, _i9, In
[10]:

What am I misunderstanding about exec()?

I am using IPython version 8.15.0 from Anaconda. The %run command only works from the IPython prompt, but I'm also trying to replace the use of runfile within scripts. If I invoke a script using %run from the IPython prompt, and the script also contains %run, it is flagged as an error.

I also ruled out import, subprocess, and os.system(), but that is starting to drift from the topic of my question. For those interested, I describe the problems with those commands here.

Ideally, there would be an alternative to runfile that executes statements in a source file, but does so in the local scope of code (or REPL) from which runfile was issued. Furthermore, the alternative doesn't require a lot of busy code (like runfile). I realize that I'm wishing for the moon -- hoping that it exists, but prepared for the likelihood that it does not.

I considered @jasonharper's approach of explicity supplying "locals" and "globals" dictionaries as arguments to execute_python_file_with_exec, which then passes them to exec(). Unlike globals(), however, locals() only returns a copy of the local variables. Consequently, script Script.py will not be able to add objects to that scope. In fact, this SO answer confirms @jasonharper's explanation that local variables are determined at compile time, and therefore cannot be added to.

It does create objects in the local scope. The scope in which you ran exec was inside execute_python_file_with_exec; if you ran print(locals()) inside that function you'd see they were indeed added (alongside code, file and file_path). — jonrsharpe
– jonrsharpe, Commented May 1 at 22:16
How about ipython's own magic, %run? ipython.readthedocs.io/en/stable/interactive/… — hpaulj
– hpaulj, Commented May 1 at 22:53
%run seems to be literally it, and even from .. import * may meet your needs (although you will have issues doing that more than once) — Grismar
– Grismar, Commented May 2 at 0:32
@hpaulj AFAIKT that would be equivalent to ns = {"__name__":"__main__"}; exec(code, ns, ns); globals().update(ns) — juanpa.arrivillaga
– juanpa.arrivillaga, Commented May 2 at 1:48
@KellyBundy I don't know. In Python 3.12.0 on macOS, I see {'x': 1, 'y': 2} for that example. If that's running 3.13, could be related to docs.python.org/3/whatsnew/…. — jonrsharpe
– jonrsharpe, Commented May 2 at 6:48

blhsing · Accepted Answer · 2025-05-06 01:13:05Z

Function in Python 3 is considered to be an optimized scope, where local variables are compiled into what's known as "fast locals", stored in an array and accessed by offsets to the array through the LOAD_FAST* and STORE_FAST* bytecodes.

Code passed to the exec function, on the other hand, are executed in an unoptimized scope, where variables are accessed by names in the dicts of the local, globals and built-in namespaces. But when called from a function scope, local variables are not available as a dict, so a snapshot of local variables are created from the fast locals array as a new dict so it can be passed as locals for the exec call. Since the snapshot is a copy separate from the array storing the actual local variables, changes to the local namespace made within the exec call therefore cannot be reflected in the calling scope.

Since Python 3.13, however, with the implementation of PEP-667, the f_locals attribute of the frame object is now a write-through proxy mapping where changes made to f_locals can be reflected in actual array of local variables in the frame, so you can now reliably update variables in the local scope of a specific frame with exec by passing in the f_locals attribute of the frame as locals.

In you case, since you apparent want the caller of the function that calls exec to have the dynamically created local variable, you would want to use the caller's frame with sys._getframe(1):

import sys

def define_doggy():
    exec('doggy = "Doggy"', locals=sys._getframe(1).f_locals)

define_doggy()
print(locals()['doggy'])

This outputs:

Doggy

Demo here

Thank you for your very helpful explication. I ran your script at ato.pxeger.com, but the spinning wheel never stops turning. I also pasted your code into Spyder, but when I invoke define_doggy(), I get TypeError: 'locals' is an invalid keyword argument for exec(). However, help(exec) shows that locals is indeed a named argument.
ato.pxeger.com opens a WebSocket connection (wss://) for its program execution, and your browser may be connecting through a proxy server that doesn't support it. As mentioned in the answer, f_locals as a write-through proxy is a feature introduced in Python 3.13, so you need to make sure your environment has the right Python version installed.
Also note that locals as a keyword argument for exec is also only introduced in Python 3.13.
My organization does have a proxy server, but navigating how to work with it to access outside resources lies outside my area of expertise. About the f_locals, the error is actually complaining that locals is not a named argument in the signature for exec. My Python is 3.9. Puzzled...

user2153235 · Accepted Answer · 2025-05-02 20:29:59Z

0

According to this Q&A and this Q&A, the correct way to run source code in the current scope is

exec(open("filename.py").read())

According to this answer, throwing in the compile command makes for easier debugging (though noisier code):

with open("somefile.py") as f:
   code = compile(f.read(), "somefile.py", 'exec')
   exec(code, global_vars, local_vars)

answered May 2 at 20:29

user2153235

1,26511 silver badges23 bronze badges

Collectives™ on Stack Overflow

Why doesn't exec() create objects in local/global scope?

2 Answers 2

4 Comments

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Linked

Related