Pysolate

A Semantics-Aware Sandbox
for Efficient Isolated Execution

Yuzhe Shi
MSc Individual Project · Imperial College London

Running students’ code

Imagine a data analysis class. Students need a platform to run their code.

We start with a Python server. It executes submissions in the same interpreter.

student_01.pystudent_02.pystudent_03.py

Python server

exec(submission)

Read the scores, then take their mean

Normal runMalloryBob
import numpy as npscores = np.loadtxt("scores.csv")[1., 2., 3.]result = scores.mean()2.0

Mallory prepares a replacement function

Normal runMalloryBob
import numpy as np def missing_file(path):    raise FileNotFoundError(path) np.loadtxt = missing_file

Mallory changes the shared interpreter

Normal runMalloryBob
import numpy as np def missing_file(path):    raise FileNotFoundError(path) np.loadtxt = missing_file

Bob runs the original program

Normal runMalloryBob
import numpy as npscores = np.loadtxt("scores.csv")FileNotFoundErrorresult = scores.mean()

Bob had never met Mallory. His interpreter had.

Normal runMalloryBob
import numpy as npscores = np.loadtxt("scores.csv")FileNotFoundErrorresult = scores.mean()

Bob had never met Mallory. His interpreter had.

Normal runMalloryBob
import numpy as npscores = np.loadtxt("scores.csv")FileNotFoundErrorresult = scores.mean()

How can every run start clean?

Every run needs to be fresh

Reuse one interpreterOne interpreter per requestRequestsPythonRequest 1PythonRequest 2PythonRequest 3PythonOld state survivesSetup is repeated

Can we reuse setup and keep runs separate?

Every run needs to be fresh

Reuse one interpreterOne interpreter per requestRequestsPythonRequest 1PythonRequest 2PythonRequest 3PythonOld state survivesSetup is repeated

Can we reuse setup and keep runs separate?

Introducing WebAssembly

We compile CPython, the Python interpreter, to WebAssembly (Wasm). The submitted code stays ordinary Python.

Execution

A runtime loads the module and executes its instructions.

Memory

The runtime checks accesses to linear memory.

External access

The Host application connects the functions the module can call.

Why Wasm fits this design

The Host manages memory

The Host can capture and remap memory for snapshots, COW and reset.

External calls go through the Host

External calls go through functions provided by the Host.

One Host process manages separate Wasm instances.

Inside the Pysolate sandbox

Go HostWasm runtimeCPythonimport numpy as npscores = np.loadtxt("scores.csv")result = scores.mean()WASIFile accessWorkspacescores.csv

Inside the Pysolate sandbox

Go HostWasm runtimeCPythondoc = tools.fetch("report")tools.send_email(doc)WASIFile accessWorkspacescores.csv

Connect our own tools alongside WASI

Go HostWasm runtimeCPythondoc = tools.fetch("report")tools.send_email(doc)WASIFile accessWorkspacescores.csvOur Tool ABIHost tool handlerWeb & email

Load the CPython Wasm module

Host actions

1Load CPython module
2runtime_init
3runtime_prepare
4Snapshot

CPython memory

+ Module data

+ Python objects

+ NumPy objects

Saved linear memory

Initialise Python inside the instance

Host actions

1Load CPython module
2runtime_init
3runtime_prepare
4Snapshot

CPython memory

+ Module data

+ Python objects

+ NumPy objects

Saved linear memory

Warm the interpreter with NumPy

Host actions

1Load CPython module
2runtime_init
3runtime_prepare
4Snapshot

CPython memory

+ Module data

+ Python objects

+ NumPy objects

Saved linear memory

Capture the warmed interpreter state

Host actions

1Load CPython module
2runtime_init
3runtime_prepare
4Snapshot

CPython memory

+ Module data

+ Python objects

+ NumPy objects

Saved linear memory

The Host reads linear memory as []byte. After setup returns, we can save these bytes and restore them into a fresh instance.

Capture the warmed interpreter state

Host actions

1Load CPython module
2runtime_init
3runtime_prepare
4Snapshot

CPython memory

+ Module data

+ Python objects

+ NumPy objects

Saved linear memory

Copy the whole image for every run?

Runs can share unchanged pages

Initial memory imageABCMalloryABCReferences the imageBobABCReferences the image

Mallory tries to write to shared page B

Initial memory imageABCMalloryABCFirst write to BBobABCStill uses the original B

Give Mallory a private copy of B

Initial memory imageABCMalloryABB′CPrivate copy of BBobABCStill uses the original B

Mallory changes her private page

Initial memory imageABCMalloryABB′Cnp.loadtxt is replacedBobABCBob’s result: 2.0

More instances, greater savings

image/svg+xml Matplotlib v3.10.6, https://matplotlib.org/ −40 −20 0 1 2 4 8 Instances +4.6 -2.9 -18.1 -48.4 End-to-end time (s) −600 −400 −200 0 +59.5 +42.7 -314.4 -534.0 Memory: PSS (MiB) COW − per-run initialisation Saved Added

Dirty pages reduce how many instances fit

With 28 GiB memory available

image/svg+xml Matplotlib v3.10.6, https://matplotlib.org/ 0 5 10 25 50 Pages made private per instance (%) 0 5k 10k 15k Estimated instances 14.9k 6.7k 2.1k 240

Reclaim the private page to reset the worker

Initial memory imageABCWorker 1ABB′CDrop private page B′Worker 2ABCReady for the next run

Faster reset for an answer-checking service

image/svg+xml Matplotlib v3.10.6, https://matplotlib.org/ 0 50 100 150 Median request latency (ms) Full-copy reset COW reset 154.5 ms 1.22 ms

One prepared state supports many fresh runs

Prepared snapshotRun 1Run 2Run 3

Isolated runs

Each run keeps its writes private.

Low memory use

Clean pages stay shared across runs.

Fast reset

Drop private changes and reuse the prepared state.

Short agent tasks fit a warm sandbox

Many short agent programs finish their calculations quickly, then wait for I/O.

Summarise data

values = np.array([12, 18, 24])result = values.mean()

Value a portfolio

acme = market.recent_price("ACME")beta = market.recent_price("BETA")portfolio_value = acme + beta

Extract fields

item = json.loads(payload)name = item["name"]

Build a message

text = "\n".join(notes)tools.send_email(text)

Example: value a two-stock portfolio

One share of ACME and one share of BETA.

acme = market.recent_price("ACME")beta = market.recent_price("BETA")portfolio_value = acme + beta

Both reads use the Tool ABI, so the Host can schedule them.

The complete program waits at each call

   acme = market.recent_price("ACME")beta = market.recent_price("BETA")portfolio_value = acme + beta

Each call waits for its result before Python continues.

PythonACME readBETA readWait for ACMEWait for BETA200 ms200 ms0200 ms400 ms

The complete program waits at each call

   acme = market.recent_price("ACME")beta = market.recent_price("BETA")portfolio_value = acme + beta

Both tickers are known. Why wait?

PythonACME readBETA readWait for ACMEWait for BETA200 ms200 ms0200 ms400 ms

Start both price reads early

q_acme = prepare(market.recent_price, "ACME")q_beta = prepare(market.recent_price, "BETA") acme = materialize(q_acme)beta = materialize(q_beta)portfolio_value = acme + beta

Our ABI identifies the tool and its arguments.

Prepare is non-blocking. It starts the request and returns a handle.

PythonACME readBETA read200 ms200 msPrepare returns handles immediately0200 ms400 ms

Materialize delivers the result at the original call

q_acme = prepare(market.recent_price, "ACME")q_beta = prepare(market.recent_price, "BETA") acme = materialize(q_acme)beta = materialize(q_beta)portfolio_value = acme + beta

The first materialize blocks until ACME is ready. Python then reaches the second materialize. BETA is already ready, so it returns immediately.

PythonACME readBETA read200 ms200 msBlocked in first materializeFirst materialize returns.Then the second runs and returns immediately.0200 ms400 ms

Materialize delivers the result at the original call

q_acme = prepare(market.recent_price, "ACME")q_beta = prepare(market.recent_price, "BETA") acme = materialize(q_acme)beta = materialize(q_beta)portfolio_value = acme + beta

Will the quote still be valid at the call?

PythonACME readBETA read200 ms200 msBlocked in first materializeFirst materialize returns.Then the second runs and returns immediately.0200 ms400 ms

The quote arrives while Python does heavy work

q_acme = prepare(market.recent_price, "ACME")q_beta = prepare(market.recent_price, "BETA")heavy_work()acme = materialize(q_acme)beta = materialize(q_beta)portfolio_value = acme + beta

Suppose independent work runs before the price call. The quote arrives while Python is still busy.

Quote10:00ObservedPythonIndependent heavy workCall is still ahead

Heavy work can make the prepared quote too old

q_acme = prepare(market.recent_price, "ACME")q_beta = prepare(market.recent_price, "BETA")heavy_work()acme = materialize(q_acme)beta = materialize(q_beta)portfolio_value = acme + beta

The tool allows quotes up to five minutes old. By the time this call runs, our saved quote is too old.

Quote10:00ObservedPythonHeavy work takes 6 minutes10:06Original call siteQuote exceeds the 5-minute limit

Linearize checks whether we can use the saved quote

q_acme = prepare(market.recent_price, "ACME")q_beta = prepare(market.recent_price, "BETA")heavy_work()acme = materialize(linearize(q_acme))beta = materialize(linearize(q_beta))portfolio_value = acme + beta

At the original call site, use the saved quote if it is for the same ticker and at most five minutes old. Otherwise, request a new quote.

Prepare, Linearize, Materialize: PLM.

Quote10:00ObservedPythonHeavy work takes 6 minutes10:06Original call siteFresh request

Our function model

F(a, t, h)

a Arguments

t Call time

h Hidden state

An early result may become invalid over time.

Check whether we can still use it.

Here, h includes the market’s current data, which may change before the call.

Prepare

Start the request early.

Linearize

Check the inputs and the saved result at the call.

Materialize

Return the value or error at that call.

Our function model

F(a, t, h)

a Arguments

t Call time

h Hidden state

An early result may become invalid over time.

Check whether we can still use it.

Here, h includes the market’s current data, which may change before the call.

Prepare

Start the request early.

Linearize

Check the inputs and the saved result at the call.

Materialize

Return the value or error at that call.

Pysolate applies the optimization automatically

Original Python

listing = tools.listing("ACME")if listing["market_open"]:    ticker = listing["ticker"]    label = "Current price"    price = market.recent_price(ticker)

Rewritten Python

listing = tools.listing("ACME")if listing["market_open"]:    ticker = listing["ticker"]    q = prepare(market.recent_price, ticker)    label = "Current price"    price = materialize(linearize(q))

Start tool requests as their calls arrive

Code arrives a piece at a time

Requests for this run

acme = market.recent_price("ACME")Readybeta = market.recent_price("BETA")In progressportfolio_value = acme + beta

… more code is arriving

Once the inputs are known, Pysolate can start these price reads.

The original program runs for the first time

User code

Requests for this run

acme = market.recent_price("ACME")Cache hitbeta = market.recent_price("BETA")Waitingportfolio_value = acme + beta

Python uses the saved ACME quote, then waits for BETA.

BETA returns, then Python continues

User code

Requests for this run

acme = market.recent_price("ACME")Cache hitbeta = market.recent_price("BETA")Returnedportfolio_value = acme + beta

BETA returns from the same prepared request. Python continues with the sum.

How much waiting can source generation hide?

One 1.5 s tool read overlapped a 1.4 s source stream.

End-to-end time fell by 22.3% compared with serial execution.

Total time: about 6.3 s → 4.9 s.

The 1.4 s saving is close to one full tool read.

Does the optimization save time overall?

The full program is ready. Each read takes 200 ms.
End-to-end time

image/svg+xml Matplotlib v3.10.6, https://matplotlib.org/ −25 −20 −15 −10 −5 0 5 Change in total time vs ordinary execution (%) 1 2 4 8 Independent calls Saved Added

More independent reads save more time

The full program is ready. Each read takes 200 ms.
End-to-end time

image/svg+xml Matplotlib v3.10.6, https://matplotlib.org/ −25 −20 −15 −10 −5 0 5 Change in total time vs ordinary execution (%) 1 2 4 8 Independent calls +3.3% -1.0% -9.3% -22.3% Saved Added

COW reuses memory. PLM overlaps tool work.

COW sandbox

A prepared CPython image with private pages on write.

Isolated runs, low memory use and fast reset.

99.2% lower latency

Shimmy, versus full-copy reset

PLM

A source pass and Host request table implement Prepare, Linearize and Materialize.

Tool work overlaps execution and code generation.

22.3% less total time

Source-stream benchmark, versus serial

Questions

What is shared, rebuilt and restored?

Saved linear memory

Python heap

Imported module objects

Prepared data

Built for each instance

An instance of the same module

Fresh globals and tables

Host resources and workspace

Map the sealed image

memfd / MAP_PRIVATE

Set up the module

_initialize

Restore the image bytes

MADV_DONTNEED

Guest pointers are offsets. The Host mapping can move.

How do Host and Guest exchange data?

alloc(n) → request_offsetwrite JSON bytes at request_offsetexecute(request_offset, n)response_offset → [u32 length][JSON]dealloc(request_offset)

Two call directions

  • Host → Guest: source and inputs enter as JSON bytes.
  • Guest → Host: tool requests use request offset/length and reply offset/capacity.
  • The Host writes reply bytes and returns their length.

Offsets are 32-bit positions in linear memory. The response frame begins with a little-endian 4-byte payload length.

How does execution reuse an early request?

One run in the Go HostAnalyser GuestFind known tool callsExecution GuestRun the full programPLM request tableCall, operation and argumentsPrepareFind and checkExternal provider

Pending
Wait at the call

Ready
Check and return

Cannot reuse
Call the tool now

Where can the source pass place Prepare?

User code

ticker = "ACME"audit()label = "Latest quote"price = market.recent_price(ticker)result = [label, price]

Rewritten Python

ticker = "ACME"audit()q = prepare(market.recent_price, ticker)label = "Latest quote"price = materialize(linearize(q))result = [label, price]

Prepare goes after the ticker is set and after audit().

What does each timer include?

COWBuild stateCreate instancesRunCloseIncludedShimmyInstall evaluatorHandle requestCopy responseResetIncludedSource-streamPLMPrepare workersSource arrivesFinal executionCloseIncludedWhole-programPLMPrepare engineGuest setupRewrite and runDecode and closeIncluded

How does NumPy run inside the Guest?

CPython + NumPynative codestdlib + NumPyagent_runtime filesWasm reactorstatic native modulesembedded filesystem

How import finds the native code

PyImport_AppendInittab(    "numpy.core._multiarray_umath",    PyInit__multiarray_umath);
  • Register native initialisers before starting CPython.
  • Pack Python files with wasi-vfs.
  • Import uses registered built-ins and packaged source.

Which tools can start early?

Price lookup in this exampleEarly reads are allowed.Maximum quote age at the call: 5 minutesBefore PrepareHas this tool been enabled for early calls?At the original callSame ticker? Quote age ≤ 5 minutes?Start early or keep the ordinary callUse the quote or request a new oneThe developer chooses which tools can start early.Pysolate checks the saved result when Python reaches the call.

Why preserve syntax and exception order?

Check the full source

open("out.txt", "w").write("done")if (

A late syntax error stops Python before the file write.

Complete sourceSyntaxError

Deliver errors at the call

price = market.recent_price("ACME")write_receipt()

A delayed error can allow the write to happen first.

At the callErrorStop

Error raised laterWriteError

Connecting Python code to external tools

Research direction · CRM and email example

customer = tools.crm.get_customer(customer_id)report = build_report(customer)Path("/workspace/report.md").write_text(report)tools.crm.update_customer(    customer_id, {"report_status": "ready"})tools.mail.send(    customer["email"], subject="Your report",    body=report)
HostWASIWorkspace filesTool ABICRM and email tools

Code analysis: order and dependencies

Tool implementations: inputs and conditions

Preview changes before applying them

Research direction · Same report, CRM update and email

Run and stageReviewCheck and applyReport in private workspaceReport contentCRM changes held for this taskCRM field changesEmail held for sendingRecipient and messageCheck versions and expiryApply the approved contentApproval records these exact changes.Needs a real result?Wait for external completion.Inputs changed?Run and review again.Submission interrupted?Report success, failure or unknown.