Pysolate

A Semantics-Aware Sandbox
for Efficient Isolated Execution

Yuzhe Shi
MSc Individual Project · Imperial College London

Running students’ code

Imagine a data analysis class. Students need a platform to run their code.

We start with a Python server. It executes submissions in the same interpreter.

student_01.pystudent_02.pystudent_03.py

Python server

exec(submission)

Read the scores, then take their mean

Normal runMalloryBob
import numpy as npscores = np.loadtxt("scores.csv")[1., 2., 3.]result = scores.mean()2.0

Mallory prepares a replacement function

Normal runMalloryBob
import numpy as np def missing_file(path):    raise FileNotFoundError(path) np.loadtxt = missing_file

Mallory changes the shared interpreter

Normal runMalloryBob
import numpy as np def missing_file(path):    raise FileNotFoundError(path) np.loadtxt = missing_file

Bob runs the original program

Normal runMalloryBob
import numpy as npscores = np.loadtxt("scores.csv")FileNotFoundErrorresult = scores.mean()

Bob had never met Mallory. His interpreter had.

Normal runMalloryBob
import numpy as npscores = np.loadtxt("scores.csv")FileNotFoundErrorresult = scores.mean()

Bob had never met Mallory. His interpreter had.

Normal runMalloryBob
import numpy as npscores = np.loadtxt("scores.csv")FileNotFoundErrorresult = scores.mean()

How can every run start clean?

Every run needs to be fresh

Reuse one interpreterOne interpreter per requestRequestsPythonRequest 1PythonRequest 2PythonRequest 3PythonOld state survivesSetup is repeated

Can we reuse setup and keep runs separate?

Every run needs to be fresh

Reuse one interpreterOne interpreter per requestRequestsPythonRequest 1PythonRequest 2PythonRequest 3PythonOld state survivesSetup is repeated

Can we reuse setup and keep runs separate?

Introducing WebAssembly

To control interpreter state, we compile CPython to WebAssembly (Wasm), a portable instruction format.

Execution

A runtime loads the module and executes its instructions.

Memory

The runtime checks accesses to linear memory.

External access

The Host application connects the functions the module can call.

Why Wasm fits this design

Accessible linear memory

The Host can capture and remap memory for snapshots, COW and reset.

Explicit Host imports

Calls cross explicit imports, so the Host can inspect, log and schedule them.

One Host process manages separate Wasm instances.

Inside the Pysolate sandbox

Go HostWasm runtimeCPythonimport numpy as npscores = np.loadtxt("scores.csv")result = scores.mean()WASIFile accessWorkspacescores.csv

Inside the Pysolate sandbox

Go HostWasm runtimeCPythondoc = tools.fetch("report")tools.send_email(doc)WASIFile accessWorkspacescores.csv

Connect our own tools alongside WASI

Go HostWasm runtimeCPythondoc = tools.fetch("report")tools.send_email(doc)WASIFile accessWorkspacescores.csvOur Tool ABIHost tool handlerWeb & email

Load the CPython Wasm module

Host actions

1Load CPython module
2runtime_init
3runtime_prepare
4Snapshot

CPython instance

+ Wasm module

+ Python runtime

+ NumPy modules

Snapshot

Initialise Python inside the instance

Host actions

1Load CPython module
2runtime_init
3runtime_prepare
4Snapshot

CPython instance

+ Wasm module

+ Python runtime

+ NumPy modules

Snapshot

Warm the interpreter with NumPy

Host actions

1Load CPython module
2runtime_init
3runtime_prepare
4Snapshot

CPython instance

+ Wasm module

+ Python runtime

+ NumPy modules

Snapshot

Capture the warmed interpreter state

Host actions

1Load CPython module
2runtime_init
3runtime_prepare
4Snapshot

CPython instance

+ Wasm module

+ Python runtime

+ NumPy modules

Snapshot

The Host reads linear memory as []byte. After setup returns, we can save these bytes and restore them into a fresh instance.

Capture the warmed interpreter state

Host actions

1Load CPython module
2runtime_init
3runtime_prepare
4Snapshot

CPython instance

+ Wasm module

+ Python runtime

+ NumPy modules

Snapshot

Copy the whole image for every run?

Runs can share unchanged pages

Initial memory imageABCMalloryABCReferences the imageBobABCReferences the image

Mallory tries to write to shared page B

Initial memory imageABCMalloryABCFirst write to BBobABCStill uses the original B

Give Mallory a private copy of B

Initial memory imageABCMalloryABB′CPrivate copy of BBobABCStill uses the original B

Mallory changes her private page

Initial memory imageABCMalloryABB′Cnp.loadtxt is replacedBobABCBob’s result: 2.0

More instances, greater savings

image/svg+xml Matplotlib v3.10.6, https://matplotlib.org/ −40 −20 0 1 2 4 8 Instances +4.6 -2.9 -18.1 -48.4 Time (s) −600 −400 −200 0 +59.5 +42.7 -314.4 -534.0 Memory (MiB) COW − per-run initialisation Saved Added

Dirty pages reduce how many instances fit

With 28 GiB memory available

image/svg+xml Matplotlib v3.10.6, https://matplotlib.org/ 0 5 10 25 50 Pages made private per instance (%) 0 5k 10k 15k Estimated instances 14.9k 6.7k 2.1k 240

Reclaim the private page to reset the worker

Initial memory imageABCWorker 1ABB′CDrop private page B′Worker 2ABCReady for the next run

Faster reset for an answer-checking service

image/svg+xml Matplotlib v3.10.6, https://matplotlib.org/ 0 50 100 150 Median request latency (ms) Full-copy reset COW reset 154.5 ms 1.22 ms

One prepared state supports many fresh runs

Prepared snapshotRun 1Run 2Run 3

Isolated runs

Each run keeps its writes private.

Low memory use

Clean pages stay shared across runs.

Fast reset

Drop private changes and reuse the prepared state.

Short agent tasks fit a warm sandbox

Many short agent programs finish their calculations quickly, then wait for I/O.

Summarise data

values = np.array([12, 18, 24])result = values.mean()

Value a portfolio

acme = market.recent_price("ACME")beta = market.recent_price("BETA")portfolio_value = acme + beta

Extract fields

item = json.loads(payload)name = item["name"]

Build a message

text = "\n".join(notes)tools.send_email(text)

Example: value a two-stock portfolio

One share of ACME and one share of BETA.

acme = market.recent_price("ACME")beta = market.recent_price("BETA")portfolio_value = acme + beta

Both reads use the Tool ABI, so the Host can schedule them.

The complete program waits at each call

   acme = market.recent_price("ACME")beta = market.recent_price("BETA")portfolio_value = acme + beta

Each call waits for its result before Python continues.

PythonACME readBETA readWait for ACMEWait for BETA200 ms200 ms0200 ms400 ms

The complete program waits at each call

   acme = market.recent_price("ACME")beta = market.recent_price("BETA")portfolio_value = acme + beta

Both tickers are known. Why wait?

PythonACME readBETA readWait for ACMEWait for BETA200 ms200 ms0200 ms400 ms

Use the tool boundary to start requests early

q_acme = prepare(market.recent_price, "ACME")q_beta = prepare(market.recent_price, "BETA") acme = materialize(q_acme)beta = materialize(q_beta)portfolio_value = acme + beta

Our ABI identifies the tool and its arguments.

Prepare is non-blocking. It starts the request and returns a handle.

PythonACME readBETA read200 ms200 msPrepare returns handles immediately0200 ms400 ms

Materialize delivers the result at the original call

q_acme = prepare(market.recent_price, "ACME")q_beta = prepare(market.recent_price, "BETA") acme = materialize(q_acme)beta = materialize(q_beta)portfolio_value = acme + beta

The first materialize blocks until ACME is ready. Python then reaches the second materialize. BETA is already ready, so it returns immediately.

PythonACME readBETA read200 ms200 msBlocked in first materializeFirst materialize returns.Then the second runs and returns immediately.0200 ms400 ms

Materialize delivers the result at the original call

q_acme = prepare(market.recent_price, "ACME")q_beta = prepare(market.recent_price, "BETA") acme = materialize(q_acme)beta = materialize(q_beta)portfolio_value = acme + beta

Will the quote still be valid at the call?

PythonACME readBETA read200 ms200 msBlocked in first materializeFirst materialize returns.Then the second runs and returns immediately.0200 ms400 ms

The quote arrives while Python does heavy work

q_acme = prepare(market.recent_price, "ACME")q_beta = prepare(market.recent_price, "BETA")heavy_work()acme = materialize(q_acme)beta = materialize(q_beta)portfolio_value = acme + beta

Suppose independent work runs before the price call. The quote arrives while Python is still busy.

Quote10:00ObservedPythonIndependent heavy workCall is still ahead

Heavy work can make the prepared quote too old

q_acme = prepare(market.recent_price, "ACME")q_beta = prepare(market.recent_price, "BETA")heavy_work()acme = materialize(q_acme)beta = materialize(q_beta)portfolio_value = acme + beta

The tool allows quotes up to five minutes old. By the time this call runs, our saved quote is too old.

Quote10:00ObservedPythonHeavy work takes 6 minutes10:06Original call siteQuote exceeds the 5-minute limit

Linearize checks reuse at the original call

q_acme = prepare(market.recent_price, "ACME")q_beta = prepare(market.recent_price, "BETA")heavy_work()acme = materialize(linearize(q_acme))beta = materialize(linearize(q_beta))portfolio_value = acme + beta

Linearize checks reuse at the original call site. It accepts a valid request or starts a fresh one.

Prepare, Linearize, Materialize: PLM.

Quote10:00ObservedPythonHeavy work takes 6 minutes10:06Original call siteFresh request

Our function model

F(a, t, h)

a Arguments

t Call time

h Hidden state

Optimized call ≡ original call at the call site

Both calls must satisfy the same rule for values, errors and effects.

Here, h includes the market’s current data, which may change before the call.

Prepare

Start early when the Host rule allows it.

Linearize

Recheck arguments, time and relevant state.

Materialize

Deliver the value or error at the original call.

Our function model

F(a, t, h)

a Arguments

t Call time

h Hidden state

Optimized call ≡ original call at the call site

Both calls must satisfy the same rule for values, errors and effects.

Here, h includes the market’s current data, which may change before the call.

Prepare

Start early when the Host rule allows it.

Linearize

Recheck arguments, time and relevant state.

Materialize

Deliver the value or error at the original call.

Pysolate applies the optimization automatically

Original Python

listing = tools.listing("ACME")if listing["market_open"]:    ticker = listing["ticker"]    label = "Current price"    price = market.recent_price(ticker)

Generated schedule

listing = tools.listing("ACME")if listing["market_open"]:    ticker = listing["ticker"]    q = prepare(market.recent_price, ticker)    label = "Current price"    price = materialize(linearize(q))

Start tool requests as their calls arrive

Code arrives a piece at a time

Tool requests

acme = market.recent_price("ACME")Readybeta = market.recent_price("BETA")In progressportfolio_value = acme + beta

… more code is arriving

As each complete call arrives, the source pass starts its tool request.

The original program runs for the first time

User code

Tool requests

acme = market.recent_price("ACME")Cache hitbeta = market.recent_price("BETA")Waitingportfolio_value = acme + beta

Python starts for the first time. ACME hits its cache. Python waits at BETA.

BETA returns, then Python continues

User code

Tool requests

acme = market.recent_price("ACME")Cache hitbeta = market.recent_price("BETA")Returnedportfolio_value = acme + beta

BETA returns from the same prepared request. Python continues with the sum.

How much waiting can source generation hide?

One 1.5 s tool read overlapped a 1.4 s source stream.

End-to-end time fell by 22.3% compared with serial execution.

The 1.4 s saving is close to one full tool read.

Does overlap repay the source-pass cost?

Within a complete program, each read takes 200 ms.

image/svg+xml Matplotlib v3.10.6, https://matplotlib.org/ −25 −20 −15 −10 −5 0 5 Change in total time vs ordinary execution (%) 1 2 4 8 Independent calls Saved Added

More independent reads repay the rewrite cost

Within a complete program, each read takes 200 ms.

image/svg+xml Matplotlib v3.10.6, https://matplotlib.org/ −25 −20 −15 −10 −5 0 5 Change in total time vs ordinary execution (%) 1 2 4 8 Independent calls +3.3% -1.0% -9.3% -22.3% Saved Added

COW reuses memory. PLM overlaps tool work.

COW sandbox

A prepared CPython image with private pages on write.

Isolated runs, low memory use and fast reset.

99.2% lower latency

Shimmy, versus full-copy reset

PLM

A source pass and Host request table implement Prepare, Linearize and Materialize.

Tool work overlaps execution and code generation.

22.3% less total time

Source-stream benchmark, versus serial

Questions

What is shared, rebuilt and restored?

Saved linear memory

Python heap

Imported module objects

Prepared data

Built for each instance

An instance of the same module

Fresh globals and tables

Host resources and workspace

Map the sealed image

memfd / MAP_PRIVATE

Set up the module

_initialize

Restore the image bytes

MADV_DONTNEED

Guest pointers are offsets. The Host mapping can move.

How do Host and Guest exchange data?

alloc(n) → request_offsetwrite JSON bytes at request_offsetexecute(request_offset, n)response_offset → [u32 length][JSON]dealloc(request_offset)

Two call directions

  • Host → Guest: source and inputs enter as JSON bytes.
  • Guest → Host: tool requests use request offset/length and reply offset/capacity.
  • The Host writes reply bytes and returns their length.

Offsets are 32-bit positions in linear memory. The response frame begins with a little-endian 4-byte payload length.

How does execution reuse an early request?

One run in the Go HostAnalyser GuestFind known tool callsExecution GuestRun the full programPLM request tableCall, operation and argumentsPrepareFind and checkExternal provider

Pending
Wait at the call

Ready
Check and return

Invalid
Take the original call path

Where can the source pass place Prepare?

User code

ticker = "ACME"audit()label = "Latest quote"price = market.recent_price(ticker)result = [label, price]

Generated schedule

ticker = "ACME"audit()q = prepare(market.recent_price, ticker)label = "Latest quote"price = materialize(linearize(q))result = [label, price]

Prepare follows its inputs and movement barriers. It stays inside the controlling branch.

Which lifecycle does each experiment measure?

COW fan-outBuild stateCreate instancesRunCloseTimedShimmyInstall evaluatorHandle requestCopy responseResetTimedSource-streamPLMPrepare workersSource arrivesFinal executionCloseTimedWhole-programPLMPrepare engineGuest setupRewrite and runDecode and closeTimed

How does NumPy run inside the Guest?

CPython + NumPynative codestdlib + NumPyagent_runtime filesWasm reactorstatic native modulesembedded filesystem

How import finds the native code

PyImport_AppendInittab(    "numpy.core._multiarray_umath",    PyInit__multiarray_umath);
  • Register native initialisers before starting CPython.
  • Pack Python files with wasi-vfs.
  • Import uses registered built-ins and packaged source.

Who allows early work and checks reuse?

Example price-read ruleEarly reads are allowed.Maximum quote age at the call: 5 minutesBefore PrepareMay this request start early?At the original callDo the arguments and validity match?Start early or keep the ordinary callReuse or take the original call pathThe Host supplies the rule. The source pass places Prepare.

Why preserve syntax and exception order?

Check the full source

open("out.txt", "w").write("done")if (

A late syntax error stops Python before the file write.

Complete sourceSyntaxError

Deliver errors at the call

price = market.recent_price("ACME")write_receipt()

Moving delivery later can change what happens first.

At the callErrorStop

Delayed deliveryWriteError