Documentation v0.39.5 JavaScript Getting Started: Saga Pattern with Fly.io

Getting Started: Saga Pattern with Fly.io

This tutorial extends the Getting Started guide with two advanced topics:

You will build a two-workflow system that creates a fly.io app, runs a short-lived VM acting as an AI agent, and destroys the app afterwards — whether the agent succeeded or not.

Prerequisites

export FLY_API_TOKEN=your_token_here

The Saga Pattern

A saga is a sequence of steps where each step has a corresponding compensation action that undoes its effects. When any step fails you run the compensations in reverse — or in the simplest case, you run a single cleanup action at the end if the main action fails.

In Obelisk the saga guarantee is strong: even a server crash mid-workflow cannot skip the compensation, because the execution log is replayed on restart. The workflow continues from the last completed step.

Any abnormal activity failure — OOM panic, WASM trap, or an endless loop exhausting the retry budget — propagates as a regular err variant and is catchable.

A programming error in workflow code itself can also cause an execution failure. The outer saga workflow should therefore be kept minimal: only the action and its compensation. Move all complex logic — the resource creation, the agent exec, the polling loop — into the inner child workflow. That way a bug in the complex logic fails the child workflow (catchable), while the outer workflow stays simple and reliable enough that it will always reach the apps.delete call.

This tutorial implements the simplest form:

run (outer saga workflow)       ← kept intentionally minimal
├── agent (inner workflow)      ← all complex logic here; may fail for any reason
└── apps.delete                 ← always runs, regardless of agent's success or failure

The outer run workflow calls the inner agent workflow inside a try/catch, then unconditionally calls apps.delete. A try/catch is perfectly safe in a deterministic workflow: the branch taken is recorded in the execution log, so replay always follows the same path.

Clone and Navigate

cd demo-tutorial/fly-agent

The directory contains four files and a deployment.toml — nothing to compile.

The Components

Fly.io Activity (pre-built WASM)

deployment.toml references a compiled Rust activity from the Obelisk component registry:

[[activity_wasm]]
name = "activity_fly_http"
location = "oci://docker.io/getobelisk/components_fly_activity_fly_http:..."
exec.lock_expiry.seconds = 15
[[activity_wasm.allowed_host]]
pattern = "https://api.machines.dev"
methods = "*"
[activity_wasm.allowed_host.secrets]
env_vars = ["FLY_API_TOKEN"]
replace_in = ["headers"]

Obelisk downloads and caches the component on first run. The secrets block tells Obelisk to inject FLY_API_TOKEN from the environment into the Authorization header of every outbound request — the token is never written to disk or logged and only a placeholder is passed to the WASM activity.

The activity exposes the full Fly.io Machines API as callable functions. Their FFQNs follow the WIT package structure: obelisk-flyio:activity-fly-http/<interface>@1.0.0-beta.<function>.

Inner Workflow — workflow/agent.js

// demo:fly-agent/workflow.agent: func(app-name: string, org-slug: string, prompt: string) -> result<string, string>
import * as apps from "obelisk-flyio:activity-fly-http/apps@1.0.0-beta";
import * as machines from "obelisk-flyio:activity-fly-http/machines@1.0.0-beta";

export default function agent(app_name, org_slug, prompt) {
  // Step 1: Create the fly.io app.
  apps.put(org_slug, app_name);

  // Step 2: Launch a VM.
  // The init command runs the agent in the background and writes the result to
  // /result.txt. The foreground `sleep 3600` keeps the VM alive so we can read
  // the file via exec. Replace the backgrounded command with your agent.
  const machine_id = machines.create(
    app_name,
    "agent-vm",
    {
      image: "alpine:3.21",
      guest: { cpu_kind: "shared", cpus: 1, memory_mb: 256, kernel_args: null },
      auto_destroy: null,
      init: {
        entrypoint: null,
        cmd: [
          "/bin/sh",
          "-c",
          '(sleep 60 && printf "Prompt: %s\\nResult: %s\\n" "$PROMPT" "42 is the answer" > /result.txt) & sleep 3600',
        ],
        exec: null,
        kernel_args: null,
        swap_size_mb: null,
        tty: null,
      },
      env: [["PROMPT", prompt]],
      restart: { max_retries: null, policy: "no" },
      stop_config: null,
      mounts: null,
      services: null,
      files: null,
    },
    "ams",
  );

  // Step 3: Durable poll — obelisk.sleep is recorded in the execution log.
  let started = false;
  for (let i = 0; i < 20; i++) {
    const machine = machines.get(app_name, machine_id);
    if (machine !== null && machine.state === "started") {
      started = true;
      break;
    }
    obelisk.sleep({ seconds: 3 });
  }
  if (!started) throw "VM did not reach 'started' state within timeout";

  // Step 4: Poll until /result.txt appears.
  // Each exec is a fast `cat` — no fly.io exec timeout issues.
  // Throws if the VM becomes unreachable (e.g. stopped externally).
  let output = null;
  for (let i = 0; i < 30; i++) {
    const cat = machines.exec(app_name, machine_id, ["cat", "/result.txt"], {
      timeout_secs: 10,
      stdin: null,
    });
    if (cat.exit_code === 0) {
      output = (cat.stdout || "").trim();
      break;
    }
    obelisk.sleep({ seconds: 5 });
  }
  if (output === null) throw "Agent did not produce a result within timeout";
  return output;
}

Key points:

JSON encoding for WASM activities

Arguments are JSON-encoded using the WIT type schema. Key rules (see JS components for the full table):

WIT typeJS value
record { ... }Plain object; kebab-case → snake_case keys
option<T> someThe value
option<T> nonenull
enum { ... }Snake_case string, e.g. "shared"
tuple<A, B>Array [a, b]

In deployment.toml:

[[workflow_js]]
name = "agent"
location = "workflow/agent.js"
ffqn = "demo:fly-agent/workflow.agent"
params = [
  { name = "app-name", type = "string" },
  { name = "org-slug", type = "string" },
  { name = "prompt",   type = "string" },
]
return_type = "result<string, string>"

Outer Workflow — workflow/run.js

// demo:fly-agent/workflow.run: func(app-name: string, org-slug: string, prompt: string) -> result<string, string>
import { agent } from "demo:fly-agent/workflow";
import * as apps from "obelisk-flyio:activity-fly-http/apps@1.0.0-beta";

export default function run(app_name, org_slug, prompt) {
  let result = null;
  let error = null;

  // Call the inner workflow; capture any failure so cleanup still runs.
  try {
    result = agent(app_name, org_slug, prompt);
  } catch (e) {
    error = String(e);
    console.log(`Agent workflow failed: ${error}`);
  }

  // Saga compensation: always delete the app.
  // A crash between the try/catch and here is safe: Obelisk replays the log
  // on restart and continues from this point.
  console.log(`Deleting app: ${app_name}`);
  try {
    apps.delete(app_name, true);
    console.log("App deleted");
  } catch (e) {
    console.log(`App deletion failed (manual cleanup may be needed): ${e}`);
  }

  if (error !== null) throw error;
  return result;
}

The try/catch pattern is deterministic: the branch taken (error or success) is recorded in the execution log. On replay Obelisk replays the same branch — crash recovery is guaranteed regardless of where the server stopped.

In deployment.toml:

[[workflow_js]]
name = "run"
location = "workflow/run.js"
ffqn = "demo:fly-agent/workflow.run"
params = [
  { name = "app-name", type = "string" },
  { name = "org-slug", type = "string" },
  { name = "prompt",   type = "string" },
]
return_type = "result<string, string>"

Webhook — webhook/fly-agent.js

Route path segments starting with : are captured and exposed via process.env inside the handler. The webhook uses three segments for org, app name, and prompt:

import { run } from "demo:fly-agent/workflow";

// Route: /run/:org-slug/:app-name/:prompt
export default function handle(_request) {
  const org_slug = process.env["org-slug"];
  const app_name = process.env["app-name"];
  const prompt = process.env["prompt"];
  const result = run(app_name, org_slug, prompt);
  return new Response(`Agent completed:\n${result}\n`, { status: 200 });
}

In deployment.toml:

[[webhook_endpoint_js]]
name = "webhook"
location = "webhook/fly-agent.js"
routes = [{ methods = ["GET"], route = "/run/:org-slug/:app-name/:prompt" }]

See JS Webhooks for the full path parameter reference and Configuration for all options.

Run

cd demo-tutorial/fly-agent
obelisk server run --deployment deployment.toml

Trigger the saga — pick a globally unique app name on fly.io:

APP=my-fly-agent-$(date +%s)
curl "http://localhost:9090/run/personal/${APP}/what-is-42"

After ~75 seconds (VM creation + ~60 s background job + app deletion):

Agent completed:
Prompt: what-is-42
Result: 42 is the answer

List executions to see the full hierarchy:

curl -s "http://localhost:5005/v1/executions?show_derived=true&ffqn_prefix=demo:fly-agent" \
  -H 'Accept: text/plain'
E_01KN...  Finished(ok)  demo:fly-agent/workflow.run    ...
E_01KN...  Finished(ok)  demo:fly-agent/workflow.agent  ...

The Web UI at http://localhost:8080 shows the full trace. Enable Autoload children on the outer run execution to see the inner agent execution and all activity calls beneath it.

Saga Recovery in Action

The backgrounded sleep 60 gives you a 60-second window before /result.txt appears. Start a new saga, then stop the VM while the background job is still sleeping:

# terminal 1
APP=my-fly-agent-$(date +%s)
curl "http://localhost:9090/run/personal/${APP}/what-is-42"

# terminal 2 — stop the VM before /result.txt is written
fly machine stop agent-vm --app ${APP}

When the VM stops, the next cat /result.txt exec call returns an error (VM unreachable). The inner agent workflow throws, the outer run workflow catches it, and apps.delete fires automatically — no orphaned app left on fly.io.

Check the execution log to confirm:

curl -s "http://localhost:5005/v1/executions?show_derived=true&ffqn_prefix=demo:fly-agent" \
  -H 'Accept: text/plain'
E_01KN...  Finished(err)  demo:fly-agent/workflow.agent  ...
E_01KN...  Finished(ok)   demo:fly-agent/workflow.run    ...

The outer run finishes with ok because the compensation succeeded — the app is gone.

You can also test server-crash recovery: kill Obelisk while the result-polling loop is running and restart with obelisk server run. The workflow replays from the last completed step and continues polling; when the VM is eventually stopped, apps.delete still runs.

Next Steps