Skip to content

AnswerDotAI/safepyrun

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

185 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

safepyrun

safepyrun is an allowlist-based Python sandbox that lets LLMs execute code safely(ish) in your real environment. Instead of isolating code in a container (which cuts it off from the libraries, data, and tools it actually needs) safepyrun runs it in-process, while an audit layer blocks dangerous side effects: filesystem writes outside approved directories, subprocesses, network access, and anything else that could damage state, unless they happen inside a callable you have explicitly trusted.

It’s the Python counterpart to safecmd, which does much the same thing for bash.

Installation

Install from pypi

$ pip install safepyrun

Background

When an LLM needs to run code on your behalf, the standard advice is to sandbox it in a container. The problem is that the whole reason you want the LLM running code is so it can interact with your environment: your files, your libraries, your running processes, your data. A containerised sandbox either can’t access any of that, or it requires complex volume mounts and dependency mirroring that recreate your environment inside the container.

You could just exec the LLM’s code directly in your process, which would give full access to everything… but “everything” includes shutil.rmtree, os.remove, subprocess.run("rm -rf /"), etc!

safepyrun takes a middle path. It runs the LLM’s code in your real Python process, with access to your real objects, but it watches what the code does rather than what it names. Ordinary computation (string handling, math, JSON parsing, path inspection, data structures) just runs. Side effects are checked: writes are only allowed under approved directories, and subprocesses, sockets, and other risky operations are denied unless they happen inside a function you registered with allow().

The mechanism behind safepyrun is fastaudit, which builds on Python’s audit hook system. CPython raises an audit event for every sensitive operation: opening a file for writing, spawning a subprocess, connecting a socket, deleting a path. While sandboxed code runs, fastaudit denies those events unless a policy callback approves them. On Python 3.12 and newer it also uses sys.monitoring to raise events for calls into native extension modules, which would otherwise be invisible to audit hooks (popular audited libraries such as numpy are declared safe via the fastaudit_safe_native entry point and skipped).

safepyrun supplies the policy callback. When an event is about to be denied, it walks the call stack (plus fastaudit’s record of active async calls) to see whether the operation happened inside a callable registered with allow(). If so, the operation proceeds; if not, the code gets a PermissionError. A small AST check on the submitted code adds a few blanket rules: no importing modules like socket or importlib, no def or class statements, and no exec/eval/compile.

In-process sandboxing is no use against a determined adversary, and safepyrun doesn’t try to be. But an LLM is not a determined adversary. It’s a well-meaning but occasionally clumsy collaborator. The threat model is completely different: you don’t need to prevent deliberate escape attempts, you need to make it very unlikely that a hallucinated cleanup step or a misunderstood request causes damage. This is the same “safe-ish” philosophy used in safecmd for bash.

Once you internalise this, the design space opens up. It’s actually fine for the LLM to read files, parse data, and call into your libraries. The things you want to prevent are filesystem writes, spawned processes, network access you didn’t sanction, and overwritten state. Gating effects rather than names is what makes the sandbox pleasant: most code just works, and the checks only bite when something risky happens.

Usage

from safepyrun import *
from pyskills import *
import subprocess, httpx

The main entry point is python = RunPython(), which returns an async function that takes a string of Python code and executes it in the sandbox. The last expression in the code is returned as the result, and any print() output is captured separately. Errors are caught and reported rather than crashing the caller.

python = RunPython()
await python('1+1')
2

You can mix print() output with a return value. The printed output goes to the stdout key, and the last expression becomes result:

await python('print("hello"); 1+1')
hello

2

Modules can be imported. stderr is also captured:

await python('''
import warnings
warnings.warn('a warning')
"ok"
''')
<python_3>:2: UserWarning: a warning
  warnings.warn('a warning')

'ok'

Most of the standard library works out of the box, since ordinary computation raises no audit events:

await python('import re; re.findall(r"\\d+", "there are 3 cats and 10 dogs")')
['3', '10']

Text processing, math, data structures, iteration and functional tools, dates, encoding, serialization, and introspection all just run. Read-only filesystem access works too, since only writes and deletes are gated. The blanket exceptions come from the AST pre-check on submitted code: no importing socket or importlib (or safepyrun itself), no def/class, and no exec/eval/compile.

The allow() function

Functions from your namespace are callable from sandbox code, and pure ones just work with no registration. The difference comes when a function’s implementation needs an otherwise-denied operation, such as running a subprocess:

def echo(msg): return subprocess.run(['echo', msg], capture_output=True, text=True).stdout

try: await python('echo("hi")')
except PermissionError as e: print(f'Blocked: {str(e)[:80]}')
Blocked: Audit: subprocess.Popen blocked in sandbox with args: ('echo', ['echo', 'hi'], N
allow(echo)
await python('echo("hi")')
'hi\n'

Registering a callable marks it as trusted: while it runs, the operations its implementation performs are permitted. You can also register at definition time with the @allow decorator, for instance to hand the LLM a function that makes a network request:

@allow
def getexample(): return httpx.get('http://example.org')
await python('getexample()')
<Response [200 OK]>

Methods work the same way: pass the method object and it’s registered under its class, so it’s allowed on any instance of that class:

class Shell:
    def date(self): return subprocess.run(['date'], capture_output=True, text=True).stdout
    def whoami(self): return subprocess.run(['whoami'], capture_output=True, text=True).stdout

sh = Shell()
allow(Shell.date)
await python('sh.date()')
'Wed Jul  8 07:22:57 AEST 2026\n'

The dict form registers several methods on one class or module at once. The key is the actual module or class object, and the value is a list of method names:

allow({Shell: ['date', 'whoami']})
await python('sh.whoami()')
'jhoward\n'

Dict values can also be (name, policy) tuples for per-call validation (see write policies below), and several items can be registered in a single call:

allow(echo, {Shell: ['date', 'whoami']})

Callable instances are registered under the instance itself, so each one is trusted individually. This is how dynamically-generated client ops, like fastspec’s, are scoped one op at a time even though every op shares the same class and __call__.

Native extension modules are a special case: calls into them are monitored, so a native library either needs registering with allow() or, for popular audited libraries (numpy, pandas’ core, regex, orjson, and others), is declared safe via fastaudit’s fastaudit_safe_native entry point and works with no registration. The allow_matplotlib() and allow_pandas() helpers register sensible defaults for those two libraries, including save methods gated by write policies.

The _ suffix export convention

All symbols created in the sandbox are exported back to the caller’s namespace by default, unless the name already exists and the new value is callable or a module (to prevent accidental shadowing). Names ending with _ (but not starting with _) are always exported, even if they shadow:

await python('result_ = [x**2 for x in range(5)]')
result_
[0, 1, 4, 9, 16]

The exported symbols are real objects in your namespace:

await python('counts_ = {"a": 1, "b": 2}')
counts_
{'a': 1, 'b': 2}

This is particularly useful in LLM tool loops where the model might need to accumulate results across steps. The _ suffix is only needed when you want to force-export a name that would otherwise be blocked (because it shadows an existing callable or module).

Async support

The sandbox is async-native. If the code being executed contains await, async for, or async with expressions, they work as expected. Many modern Python libraries and LLM tool-calling frameworks are async, and you want the sandbox to be able to call into them without workarounds.

import asyncio
async def fetch(n): return n * 10
await python('''
await asyncio.gather(fetch(1), fetch(2), fetch(3))
''')
[10, 20, 30]

Writable path permissions

By default, RunPython() allows writes to the current working directory (.) and /tmp, and blocks writes elsewhere. You can pass ok_dests to restrict writes to a different set of directory prefixes:

python2 = RunPython(ok_dests=['/tmp'])
from pathlib import Path
await python2("Path('/tmp/test_write.txt').write_text('hello')")
5
try: await python2("Path('/etc/evil.txt').write_text('bad')")
except PermissionError as e: print(f'Blocked: {e}')
Blocked: open '/etc/evil.txt' not in ('/private/tmp',)

The same permission checking applies to open() in write mode, not just Path methods:

await python2("open('/tmp/test_open.txt', 'w').write('hi')")
2
try: await python2("open('/root/bad.txt', 'w')")
except PermissionError as e: print(f'Blocked: {e}')
Blocked: open '/root/bad.txt' not in ('/private/tmp',)

Read access is unaffected — only writes are gated:

await python2("open('/etc/passwd', 'r').read(10)")
'##\n# User '

Higher-level file operations like shutil.copy are also intercepted. The destination is checked against ok_dests:

await python2("import shutil; shutil.copy('/tmp/test_write.txt', '/tmp/test_copy.txt')")
'/tmp/test_copy.txt'
try: await python2("import shutil; shutil.copy('/tmp/test_write.txt', '/root/bad.txt')")
except PermissionError as e: print(f'Blocked: {e}')
Blocked: shutil.copyfile '/root/bad.txt' not in ('/private/tmp',)

By default, RunPython() uses default_ok_dests, which allows writes in . and /tmp but blocks writes elsewhere.

await python("Path('test_default_ok.txt').write_text('ok')")
await python("Path('/tmp/test_default_tmp.txt').write_text('tmp')")

try: await python("Path('/etc/nope.txt').write_text('bad')")
except PermissionError as e: print(f'Default blocked: {e}')

Path('test_default_ok.txt').unlink()
Default blocked: open '/etc/nope.txt' not in ('.', '/private/tmp', '/Users/jhoward/aai-ws', '/Users/jhoward/git')

If you want to disable write protection entirely, pass ok_dests=None:

python_un = RunPython(ok_dests=None)
un_path = Path.home()/'safepyrun-un.txt'
await python_un(f"Path({str(un_path)!r}).write_text('ok')")

You can use '.' to allow writes relative to the current working directory. Path traversal attempts (../, subdir/../../) are detected and blocked, so the sandbox can’t escape the permitted directory:

python_cwd = RunPython(ok_dests=['.'])

# Writing to cwd should work
await python_cwd("Path('test_cwd_ok.txt').write_text('hello')")
5
Path('test_cwd_ok.txt').unlink(missing_ok=True)

Writing to /tmp is blocked here since it’s not in ok_dests:

try: await python_cwd("Path('/tmp/nope.txt').write_text('bad')")
except PermissionError: print("Blocked /tmp as expected")
Blocked /tmp as expected

Parent traversal is blocked if it resolves to a location outside ok_dests:

try: await python_cwd("Path('../escape.txt').write_text('bad')")
except PermissionError: print("Blocked ../ as expected")
Blocked ../ as expected

Write policies

When ok_dests is set, safepyrun uses write policies to determine how to validate each callable’s destination arguments. Three built-in policy classes cover common patterns: checking a positional or keyword argument (PosAllowPolicy), checking the Path object itself (PathWritePolicy), and checking open() calls only when the mode is writable (OpenWritePolicy). You can also subclass AllowPolicy to create custom checks.

The simplest, PosAllowPolicy, checks a specific positional or keyword argument against the allowed destinations. Here, position 1 (or keyword dst) is validated — writing to /tmp is allowed, but /root is blocked:

pp = PosAllowPolicy(1, 'dst')
pp(None, ['src', '/tmp/ok'], {}, dict(ok_dests=['/tmp']))
try: pp(None, ['src', '/root/bad'], {}, dict(ok_dests=['/tmp']))
except PermissionError: print("PosAllowPolicy correctly blocked /root/bad")
PosAllowPolicy correctly blocked /root/bad

You can create custom write policies by subclassing AllowPolicy and implementing __call__. For example, here we show a policy that only allows writes to files with specific extensions — useful if you want the LLM to create .csv or .json files but not arbitrary scripts.

The __call__ signature receives (obj, args, kwargs, ok_dests) where obj is the object the method is called on (e.g. a Path instance), args/kwargs are the method’s arguments, and ok_dests is the list of permitted directory prefixes. Calling chk_dest first handles the directory check, then the custom logic adds the extension constraint on top.

class ExtWritePolicy(AllowPolicy):
    "Only allow writes to paths with specified extensions"
    def __init__(self, exts): self.exts = set(exts)
    def __call__(self, obj, args, kwargs, ok_dests):
        chk_dest(obj, ok_dests)
        if Path(str(obj)).suffix not in self.exts: raise PermissionError(f"{Path(str(obj)).suffix!r} not allowed")
ep = ExtWritePolicy(['.csv', '.json'])
ep(Path('/tmp/data.csv'), [], {}, ['/tmp'])
try: ep(Path('/tmp/script.sh'), [], {}, ['/tmp'])
except PermissionError: print("ExtWritePolicy correctly blocked .sh")
ExtWritePolicy correctly blocked .sh

You can register it with allow just like the built-in policies:

allow({Path: [('write_text', ExtWritePolicy(['.csv', '.json', '.txt']))]})

Configuration

safepyrun loads an optional user config from {xdg_config_home}/safepyrun/config.py at import time, after all defaults are registered. This lets you permanently extend the sandbox allowlists without modifying the package. The config file is executed with all safepyrun.core globals already available, so no imports are needed. This includes allow, allow_write_types, AllowPolicy, PathWritePolicy, PosAllowPolicy, OpenWritePolicy, and all standard library modules already imported by the module.

Example ~/.config/safepyrun/config.py (Linux) or ~/Library/Application Support/safepyrun/config.py (macOS):

import pandas

# Add pandas tools
allow({pandas.DataFrame: ['head', 'describe', 'info', 'shape']})

# Allow pandas to write CSV to ~/data
allow({pandas.DataFrame: [('to_csv', PosAllowPolicy(0, 'path_or_buf'))]})

If the config file has errors, a warning is emitted and the defaults remain intact.

CLI

safepyrun ships with a command-line tool that runs a Python script file in the sandbox. You can pass a file path, or pipe code in via stdin:

# Run a script file
$ safepyrun myscript.py

# Pipe code via stdin
$ echo "1+1" | safepyrun

The result of the last expression is printed to stdout, matching the behaviour of python in Python. Errors are reported to stderr.

About

Safe(ish) running of python code

Resources

License

Stars

10 stars

Watchers

1 watching

Forks

Contributors