Introduction

Pysolate

A Semantics-Aware Sandbox
for Efficient Isolated Execution

Yuzhe Shi
MSc Individual Project · Imperial College London

Problem

Running students’ code

Imagine a data analysis class. Students need a platform to run their code.

We start with a Python server. It executes submissions in the same interpreter.

student_01.pystudent_02.pystudent_03.py

Python server

exec(submission)
Problem

Read the scores, then take their mean

Normal runMalloryBob
import numpy as npscores = np.loadtxt("scores.csv")[1., 2., 3.]result = scores.mean()2.0
Problem

Mallory prepares a replacement function

Normal runMalloryBob
import numpy as np def missing_file(path):    raise FileNotFoundError(path) np.loadtxt = missing_file
Problem

Mallory changes the shared interpreter

Normal runMalloryBob
import numpy as np def missing_file(path):    raise FileNotFoundError(path) np.loadtxt = missing_file
Problem

Bob runs the original program

Normal runMalloryBob
import numpy as npscores = np.loadtxt("scores.csv")FileNotFoundErrorresult = scores.mean()
Problem

Bob had never met Mallory. His interpreter had.

Normal runMalloryBob
import numpy as npscores = np.loadtxt("scores.csv")FileNotFoundErrorresult = scores.mean()
Problem

Bob had never met Mallory. His interpreter had.

Normal runMalloryBob
import numpy as npscores = np.loadtxt("scores.csv")FileNotFoundErrorresult = scores.mean()

How can every run start clean?

Problem

Give each run its own interpreter

MalloryInterpreternp.loadtxt is replacedBobInterpreterState is isolated
Problem

Each interpreter repeats the preparation

MalloryInterpreternp.loadtxt is replacedBobInterpreterState is isolatedInitial stateInitial state…
Problem

Each interpreter repeats the preparation

MalloryInterpreternp.loadtxt is replacedBobInterpreterState is isolatedInitial stateInitial state…

Can we keep runs independent and use fewer resources?

Execution model

WebAssembly (Wasm)

Wasm is a portable, low-level instruction format.CPython can be compiled to Wasm.Wasm runtimeInterpreterInstanceprogram.pyload
Execution model

How does the program read a file?

WASI: WebAssembly System InterfaceFiles and directories, standard input/output,clocks, random numbers, arguments and environment variables.HostWasm runtimeBob’s programWASISystem resourcesThe Host chooses which resources are available.
Execution model

Access other systems through named operations

HostWasm runtimePython programOperationHosthandlerValue / errorOthersystemsCredentials and network clients stay in the Host.
Execution model

Code mode: combine these operations in Python

acme = market.recent_price("ACME")beta = market.recent_price("BETA")portfolio_value = acme + beta

External call

External call

Local calculation

External operations are exposed as Python functions.

Execution model

A basic execution model

1Create Guestand workspace2Init Pythonand packages3Run program4Return result5Close GuestGoal: reduce setup cost and waiting.
Pysolate

Pysolate: efficient, isolated Python execution

CodePysolate / Go HostWasm runtimeGuestinstanceCPythonGuestinstanceCPythonGuestinstanceCPythonResult
Pysolate

A natural fit for agent-generated code

Generated programsSandbox requirementsShort tasksFast startupFrequent runsLow setup and memory costUntrusted codeIsolated state and controlled access
Pysolate

Our idea: reorganise memory and external work

One Process / One Logical RunDecompose / reorganiseStateReuse setupComputationFollow the programExternal workArrange requests
Reusing state

Avoid repeating interpreter setup

Run AInit PythonInit packagesRunRun BInit PythonInit packagesRunRun CInit PythonInit packagesRunPrepare the starting state once?
Reusing state

Load the CPython Wasm module

Host actions

1Load CPython module
2Init Python
3Init Packages
4Snapshot

Linear memory

+ Module data

+ Python objects

+ NumPy objects

Saved linear memory

Reusing state

Initialise Python inside the instance

Host actions

1Load CPython module
2Init Python
3Init Packages
4Snapshot

Linear memory

+ Module data

+ Python objects

+ NumPy objects

Saved linear memory

Reusing state

Load NumPy into the starting state

Host actions

1Load CPython module
2Init Python
3Init Packages
4Snapshot

Linear memory

+ Module data

+ Python objects

+ NumPy objects

Saved linear memory

Reusing state

Save after setup, before user code runs

Host actions

1Load CPython module
2Init Python
3Init Packages
4Snapshot

Linear memory

+ Module data

+ Python objects

+ NumPy objects

Saved linear memory

Reusing state

Full copy: one memory copy per run

MalloryInterpreterFull copy of all pagesBobInterpreterFull copy of all pagesABCABCInitial memory imageABCCopyCopy
Reusing state

A short run may change only a few pages

MalloryInterpreterFull copy of all pagesBobInterpreterFull copy of all pagesABCABCInitial memory imageABCCopyCopyCan runs share the unchanged physical pages?
Reusing state

Copy-on-write: map shared physical pages

MalloryInterpreterReady to runBobInterpreterReady to runShared physical pagesABCMapMapTwo instances. One shared set of physical pages.
Reusing state

Mallory is about to write page B

MalloryInterpreterAbout to write BBobInterpreterStill uses BShared physical pagesABCWrite B

What happens when Mallory writes?

Reusing state

A write creates a private page

MalloryInterpreterPrivate page readyBobInterpreterStill uses BShared physical pagesABCB′Copy B
Reusing state

The interpreters keep their own changes

MalloryInterpreternp.loadtxt is replacedBobInterpreterState is isolatedShared physical pagesABCB′Mallory’s changesBob still reads B
Reusing state

Reset an evaluator between requests

Worker AInterpreterReady for the next runWorker BInterpreterState is isolatedShared physical pagesABCB′Discard private changes. Reuse the starting state.
Reusing state

Lower request latency with COW reset

image/svg+xml Matplotlib v3.10.6, https://matplotlib.org/ 0 50 100 150 Median request latency (ms) Full-copy reset COW reset 154.5 ms 1.22 ms
Reusing state

More instances, greater savings

image/svg+xml Matplotlib v3.10.6, https://matplotlib.org/ −40 −20 0 1 2 4 8 Instances +4.6 -2.9 -18.1 -48.4 End-to-end time (s) −600 −400 −200 0 +59.5 +42.7 -314.4 -534.0 Process PSS (MiB) COW − per-run initialisation Saved Added
Reusing state

Dirty pages reduce how many instances fit

With 28 GiB memory available

image/svg+xml Matplotlib v3.10.6, https://matplotlib.org/ 0 5 10 25 50 Pages made private per instance (%) 0 5k 10k 15k Estimated instances 14.9k 6.7k 2.1k 240
Reusing state

COW summary: isolated runs with shared memory

One interpreterPythonShared changesNot isolatedSeparate interpretersMalloryPythonBobPythonMemoryMemoryIsolatedSeparate interpretersMalloryPythonBobPythonShared starting pagesIsolated and efficient
Overlapping calls

Now reduce waiting in the price program

acme = market.recent_price("ACME")beta = market.recent_price("BETA")portfolio_value = acme + betaACME readBETA readPython sum
Overlapping calls

Ordinary execution waits at each call

acme = market.recent_price("ACME")beta = market.recent_price("BETA")portfolio_value = acme + betaACME readBETA readPython sumExecutionWait for ACMEWait for BETASum
Overlapping calls

Ordinary execution waits at each call

acme = market.recent_price("ACME")beta = market.recent_price("BETA")portfolio_value = acme + betaACME readBETA readPython sumExecutionWait for ACMEWait for BETASum

Both tickers are known. Why wait?

Overlapping calls

Start both price reads early

q_acme = prepare(market.recent_price, "ACME")q_beta = prepare(market.recent_price, "BETA") acme = materialize(q_acme)beta = materialize(q_beta)portfolio_value = acme + beta

These price reads leave market data unchanged. Prepare starts a request and returns a handle. It does not wait for the result.

PythonACME readBETA readHandles returnedRequest startedRequest started0
Overlapping calls

Materialize delivers the result at the original call

q_acme = prepare(market.recent_price, "ACME")q_beta = prepare(market.recent_price, "BETA") acme = materialize(q_acme)beta = materialize(q_beta)portfolio_value = acme + beta

Materialize waits here if the result is not ready. Python then uses the result and continues.

PythonACME readBETA read200 ms200 msBlocked in first materializeFirst materialize returns.Then the second runs and returns immediately.0200 ms400 ms
Overlapping calls

Tool work can also overlap Python computation

Original order[ independent computation ]price = market.recent_price("ACME")Pysolate steps (pseudocode)q = prepare(market.recent_price, "ACME")[ independent computation ]price = materialize(q)
Overlapping calls

The quote arrives while Python is still computing

q_acme = prepare(market.recent_price, "ACME")q_beta = prepare(market.recent_price, "BETA")# ... independent computation ...acme = materialize(q_acme)beta = materialize(q_beta)portfolio_value = acme + beta

Suppose independent work runs before the price call. The quote arrives while Python is still busy.

Quote10:00ObservedPythonIndependent heavy workCall is still ahead
Overlapping calls

The quote arrives while Python is still computing

q_acme = prepare(market.recent_price, "ACME")q_beta = prepare(market.recent_price, "BETA")# ... independent computation ...acme = materialize(q_acme)beta = materialize(q_beta)portfolio_value = acme + beta

Suppose independent work runs before the price call. The quote arrives while Python is still busy.

Quote10:00ObservedPythonIndependent heavy workCall is still ahead

Can we still use the quote at the call?

Overlapping calls

Heavy work can make the prepared quote too old

q_acme = prepare(market.recent_price, "ACME")q_beta = prepare(market.recent_price, "BETA")# ... independent computation ...acme = materialize(q_acme)beta = materialize(q_beta)portfolio_value = acme + beta

The tool allows quotes up to five minutes old. By the time this call runs, our saved quote is too old.

Quote10:00ObservedPythonHeavy work takes 6 minutes10:06Original call siteQuote exceeds the 5-minute limit
Overlapping calls

Check the quote when Python reaches the call

q_acme = prepare(market.recent_price, "ACME")q_beta = prepare(market.recent_price, "BETA")# ... independent computation ...acme = materialize(linearize(q_acme))beta = materialize(linearize(q_beta))portfolio_value = acme + beta

Linearize checks the saved result at the original call site. The ticker must match and the quote must be at most five minutes old. Otherwise, request a new quote.

Quote10:00ObservedPythonHeavy work takes 6 minutes10:06Original call siteFresh request
Overlapping calls

Prepare, Linearize, Materialize

price = market.recent_price("ACME")EarlierWhen Python reaches the callPrepare• Start allowed work earlyReturn a handle immediatelyLinearize• Check inputs and freshnessat the callMaterialize• Wait only if neededReturn value or error
Overlapping calls

Pysolate adds the steps automatically

Original Python

acme = market.recent_price("ACME")beta = market.recent_price("BETA")portfolio_value = acme + beta

Pysolate adds the steps

q_acme = prepare(    market.recent_price, "ACME")q_beta = prepare(    market.recent_price, "BETA")acme = materialize(linearize(q_acme))beta = materialize(linearize(q_beta))portfolio_value = acme + beta

The Host tracks these requests and checks them at the original calls.

Overlapping calls

Rewriting takes time too

The full program is ready. Each synthetic read takes 200 ms.

Ordinary execution

Run the program

With PLM

Rewrite the calls
Then run the program

Does the optimization save time overall?

Overlapping calls

One read leaves little work to overlap

The full program is ready. Each synthetic read takes 200 ms.
End-to-end time

image/svg+xml Matplotlib v3.10.6, https://matplotlib.org/ −25 −20 −15 −10 −5 0 5 Change in total time vs ordinary execution (%) 1 2 4 8 Independent calls +3.3% Saved Added
Overlapping calls

More independent reads save more time

The full program is ready. Each synthetic read takes 200 ms.
End-to-end time

image/svg+xml Matplotlib v3.10.6, https://matplotlib.org/ −25 −20 −15 −10 −5 0 5 Change in total time vs ordinary execution (%) 1 2 4 8 Independent calls +3.3% -1.0% -9.3% -22.3% Saved Added
Overlapping calls

Prepare while code is still arriving

acme = market.recent_price("ACME")......

The first complete line gives us the call and its input. The Host can prepare the ACME read while more code arrives.

SourceACME readPythonMore code arrivingFirst linePrepareRequest startedNot running yet
Overlapping calls

Prepare while code is still arriving

acme = market.recent_price("ACME")beta = market.recent_price("BETA")portfolio_value = acme + beta

With the full source, we validate and run the program. At the ACME call, we check and reuse the early request.

SourceACME readPythonFull source receivedFirst linePreparePrice requestResult readyFull sourceRun full program
Overlapping calls

Requests can overlap code generation too

Preparing requests during code generation reduced end-to-end time by about 22%, compared with waiting for the full source and then running serially.

Conclusion

Pysolate: a Wasm sandbox with COW and PLM

A Wasm sandboxExperimental CPython + NumPy build for WASI Preview 1.High-density execution with COWShare starting memory and keep each run’s changes private.Tool semantics and PLMThe tool interface exposes operations and inputs to the Host.PLM overlaps allowed calls towards the program’s critical path.
Questions

Questions

Backup

What is shared, rebuilt and restored?

Saved linear memory

Python heap

Imported module objects

Prepared data

Built for each instance

An instance of the same module

Fresh globals and tables

Host resources and workspace

Map the sealed image

memfd / MAP_PRIVATE

Set up the module

_initialize

Restore the image bytes

MADV_DONTNEED

Guest pointers are offsets. The Host mapping can move.

Backup

How do Host and Guest exchange data?

alloc(n) → request_offsetwrite JSON bytes at request_offsetexecute(request_offset, n)response_offset → [u32 length][JSON]dealloc(request_offset)

Two call directions

  • Host → Guest: source and inputs enter as JSON bytes.
  • Guest → Host: tool requests use request offset/length and reply offset/capacity.
  • The Host writes reply bytes and returns their length.
Backup

How does execution reuse an early request?

One run in the Go HostAnalyser GuestFind known tool callsExecution GuestRun the full programPLM request tableCall, operation and argumentsPrepareFind and checkExternal provider

Pending
Wait at the call

Ready
Check and return

Cannot reuse
Call the tool now

Backup

Where can the source pass place Prepare?

User code

ticker = "ACME"unknown_call()label = "Latest quote"price = market.recent_price(ticker)result = [label, price]

Rewritten Python

ticker = "ACME"unknown_call()q = prepare(market.recent_price, ticker)label = "Latest quote"price = materialize(linearize(q))result = [label, price]

unknown_call() has unknown effects. Prepare stays after it.

Backup

Pysolate applies the optimization automatically

Original Python

listing = tools.listing("ACME")if listing["market_open"]:    ticker = listing["ticker"]    label = "Current price"    price = market.recent_price(ticker)

Rewritten Python

listing = tools.listing("ACME")if listing["market_open"]:    ticker = listing["ticker"]    q = prepare(market.recent_price, ticker)    label = "Current price"    price = materialize(linearize(q))
Backup

What does each timer include?

COWBuild stateCreate instancesRunCloseIncludedShimmyInstall evaluatorHandle requestCopy responseResetIncludedSource-streamPLMPrepare workersSource arrivesFinal executionCloseIncludedWhole-programPLMPrepare engineGuest setupRewrite and runDecode and closeIncluded
Backup

How does NumPy run inside the Guest?

CPython interpreter+ NumPy C extensionsPython source files:stdlib, NumPy,agent_runtimeWasm reactorstatic native modulesembedded filesystem

How import finds the native code

PyImport_AppendInittab(    "numpy.core._multiarray_umath",    PyInit__multiarray_umath);
  • Register native initialisers before starting CPython.
  • Pack Python files with wasi-vfs.
  • Import uses registered built-ins and packaged source.
Backup

Which tools can start early?

Price lookup in this exampleEarly reads are allowed.Maximum quote age at the call: 5 minutesBefore PrepareHas this tool been enabled for early calls?At the original callSame ticker? Quote age ≤ 5 minutes?Start early or keep the ordinary callUse the quote or request a new oneThe developer chooses which tools can start early.
Backup

Why preserve syntax and exception order?

Check the full source

open("out.txt", "w").write("done")if (

A late syntax error stops Python before the file write.

Complete sourceSyntaxError

Deliver errors at the call

price = market.recent_price("ACME")write_receipt()

An error at the original call stops the following write, preserving the program’s behaviour.

At the callErrorStop

Backup

Our function model

F(a, t, h)

a Arguments

t Call time

h Hidden state

Here, h includes the market’s current data, which may change before the call.

Prepare

Start the request early.

Linearize

Check the inputs and the saved result at the call.

Materialize

Return the value or error at that call.

Backup

Python can wait for approval and continue

Python local state stays with this run.Python computesHost waitsPython continuesKeep a local valueApprove → dispatchUse that same valueReject or expiry → no dispatch
Backup

What do the paging tests show?

Memory mapping testPython execution testBefore wait96 MiBDuring wait0 MiBAfter resume96 MiBRAM occupied by the mapped pagesObjects kept their values.Python continued after the wait.The next run started clean.
Backup

Moving pages while a run waits

Run BWaiting for a toolIts Python state is kept.Host request pendingRun B’s private pagesRAMDisk (swap)Page 1Available RAMfor other runsPage 2Page 3Swap out
Backup

Code and state in a Wasm instance

HostWasm runtimeCPython codeInstance stateGlobals and tablesLinear memoryCPython statePython objects and data
Backup

What does Guest memory COW capture?

Inside linear memoryCPython dataPython objects, modules and bytecodeGuest request and reply buffersOutside linear memoryCompiled CPython codeWasm call stack, globals and tablesHost resources and tool requestsUnix fork()Process address space is inherited.File descriptors are inherited too.Writes create private pages.Pysolate COWLinear-memory bytes form the image.Globals and tables are built per instance.The Host manages files and tools.Capture after setup returns → store in a sealed memfd
Backup

Evaluation data and reproducibility

  • Workloads and frozen data
  • Experiment scripts and parameters
  • Analysis, figures and plotting sources
github.com/bkmashiro/pysolate-thesis
evaluation/README.md