-
Notifications
You must be signed in to change notification settings - Fork 1
Description
At some point we'll want to parallelize Vivarium processes, mostly one thread per process but some Vivarium processes might be able to use multiple threads. Then we'd want the Arrow extension library to support free-threading.
Design alternatives:
- Leave the responsibility to the calling Python code to keep each
StochasticSysteminstance within a Python thread, like the rest of a Vivarium processes's state. This is the easiest approach. (Vivarium stores would need synchronization.) - Ensure thread safety by making each public method of the
StochasticSysteminstance lock the instance state so multiple threads can't simultaneously access it. - Get aggressive by allowing each
Arrowheadinstance to do its own internal parallelism, but I don't recall that it's a parallelizable algorithm.
Looking over the Arrow code, I don't see any thread-safety bugs like mutable global variables. All the state is kept in an instance variable of the StochasticSystem Python object, including an instance of the Arrowhead extension object. Good.
It doesn't look like it needs any atomic sections for the thread-local model (1). But we still have to:
- Read up on Python & Cython free-threading,
- Turn on Cython free-threading,
- Write a test.
When this becomes worthwhile depends on when we're ready to parallelize Vivarium processes.
Furthermore, I'm optimistic about the prospect of using Mojo to compile down and parallelize the Vivarium code and models, making use of multi-cores, SIMD, GPUs, and other parallel hardware. It could be easier and more effective to move Python code (gillespy2?) to Mojo than to support free-threading in this C code.