Introducing Obelisk 0.24.1

Obelisk is an open-source deterministic workflow engine that runs, stores, and replays WASM-based workflows using SQLite.

There have been many changes since the initial public release 0.19.1, mostly focused on structured concurrency ergonomics, adding more abilities to activities and exploring using other languages than Rust for authoring WASM Components.

Process spawning

... and disk IO in activities.

The Process API allows for spawning processes on the same VM that runs the activity. Moreover, with preopened directory support, WASM activities can use Rust and its standard library to access a temporary directory on the host. Both of these features are useful for when no WASM-native library is available. One can use the disk IO to set up a configuration file, download a remote file and then use the Process API to drive the execution, interacting with the process if needed.

Heterogenous join sets

... allowing mixing different functions as well as delays into one join set. Up until now, it was only possible to submit executions of the same function into one join set:

let join_set = new_join_set_generated(ClosingStrategy::Complete);
// use `-submit` extension
let execution_id_a = account_info_submit(&join_set_foo, a());
let execution_id_b = account_info_submit(&join_set_foo, b());
// use `-await-next` extension
let (_execution_id, first_response) = account_info_await_next(&join_set).unwrap();

It was also not possible to mix child executions and persistent sleep .

With the new functionality it is possible to do both:

let execution_id_account = account_info_submit(&join_set, &login);
let execution_id_settings = get_settings_json_submit(&join_set);
let delay_id = submit_delay(&join_set, ScheduleAt::In(Duration::Seconds(10)));
// Use `join-next` to get the first response.
let first_response_id = join_next(&join_set)?;

Stub Activities

... that represent work outside of the WASM sandbox. A stub activity may appear as a regular WASM activity to workflows - its exported functions can be executed directly, or submitted and then awaited in a Join Set . There is one big distinction though: no code is executed on behalf of stub activity. Instead, the response (be it success or an execution failure) must be supplied externally, either by an workflow , CLI , or a gRPC client.

Stub activities are useful when a workflow needs to evaluate an external execution, either a process or a human intervention. Since workflows can send and receive values using this mechanism, it can be useful when the regular parent - child RPC mechanism is insufficiet. Another use case is testing: workflow can be isolated and every activity response including errors can be simulated using stub activities.

Scoped Join Set Close

Closing of a Join Set referrs to the background processes needed to uphold the structured concurrency paradigm. Since cooperative cancellation is not implemented yet, this essentially means that the parent execution will be blocked until all of its child executions are completed. Previously this process happened only during the finalization of the parent execution: each join set that contained unprocessed child requests was issued one or many "Join Next" requests until all responses arrived.

This behavior has been changed slightly: join sets are now closed when they are dropped or get out of scope. This change follows the principle of least surprise, and also binds the structured concurrency to the lifetime of a join set. The old behavior of closing join sets during finalization can be enfoced by declaring join sets at the top level scope of the workflow function or by using std::mem::forget.

The following trace shows how scoped Join Set Close can group child executions into a batch and add backpressure. Check out the full example in the documentation.

Once the cooperative cancellation is implemented, this mechanism will do the right thing as well: cancel child executions that were not awaited as soon as the join set gets out of scope.

Ability to generate extension WIT files

The Extension WIT interfaces are automatically implemented by the runtime for every WASM export.

Previously, the generated WIT interfaces were available via the Web UI, but not through the CLI .

The new

obelisk generate extensions

fills this gap by (re-)generating the WIT files, making it easier to stay in sync with latest Obelisk version.

Experimental support for JavaScript and Go for activities, workflows and webhooks

Emphasis on experimental, elaborated in Comparing Rust, JavaScript and Go for authoring WASM Components blog post.

Bonus: fly.io activity

New repository activity-flyio provides functions for creating Fly.io apps and managing VMs. It can become a building block for larger workflows, such as building a SaaS :) .