celld documentation

celld is a stateful distributed system. It runs server-side JavaScript on your machines, and it stores the long-term state in a bucket that you own: S3-compatible, Google Cloud Storage, or Azure Blob Storage. The JavaScript API follows the API of Cloudflare Workers and Durable Objects.

In Cloudflare terms, a cell is a Durable Object: a small server with a name and a private SQLite database. You make one cell for each user, each document, each chat room, or each AI agent. A cell serves HTTP, holds WebSocket connections, sets alarms, and makes outbound connections. Cells share no database, so the application divides into cells from the start. Each cell runs on one thread: a second request interleaves only while the first awaits, and storage operations are synchronous and never interleave, so the data in a cell stays consistent.

You run celld as one process on each machine. That process is a node, and the nodes that share one bucket are a fleet. Any node can serve any cell, so you add capacity by starting another node against the same bucket.

A cell has the same states as a Durable Object. A resident cell is in memory: it is active while it does work, and idle when it waits. celld removes an idle cell from memory. A cell that then keeps its hibernatable WebSocket clients, and stays on its node, is hibernated. A cell that no node holds is inactive. An inactive cell is only an object in the bucket, so it costs almost zero, and every cell starts in this state.

A cell keeps no memory across these transitions, so the constructor runs again on the next event. A hibernated cell wakes the same way a cold start does, except that its WebSocket clients stay connected and it stays on its node.

One 8 GB node holds 1,000 resident cells, so one resident cell costs approximately $0.05 each month.

Exactly one node serves a cell at a time. The nodes do not elect a leader or keep a membership list: a node claims a cell by writing a small record to the bucket, and it writes that record with a condition the storage enforces — the write succeeds only if nobody else changed the record first. Object storage therefore decides who wins, and two nodes cannot both claim the same cell. The claim expires unless the node keeps renewing it, so a machine that dies releases its cells without anyone having to declare it dead.

celld does not answer a write until the data survives a failure, so no write you were told succeeded is ever lost. That guarantee is called RPO=0, for a recovery point of zero.

How celld earns it depends on how many nodes you run. One node writes the data to the bucket first, which costs one storage round trip. Two or more nodes are faster: the node serving the cell sends each write to another node as well, and answers as soon as that node has the data on its own disk. celld uploads the data to the bucket afterwards, so the bucket still holds the long-term state. Run two or more nodes if write latency matters to you; a single node has nobody to send to, so every write waits for the bucket. CELLD_DURABILITY selects this behavior and defaults to fleet.

When a node stops, another node takes the cell over. It first collects whatever the stopped node had not uploaded yet, so the takeover starts from a complete history. The guarantees page gives the full mechanism and the exact properties that the bucket must provide.

When an event sets an alarm before its response boundary, celld does not send a successful response until a durable wake entry covers the alarm. celld does not let a later waitUntil alarm delay another event's response.

What do you build with cells

A cell fits a workload that divides into named, stateful units:

• Real-time applications. A multiplayer game, a chat room, or a collaborative document is one cell. The cell holds the WebSocket connections and the state of one room, so the room needs no lock and no external message bus.
• Agents. Each AI agent is one cell. The cell holds the memory, the schedule, and the inbox of one agent in its own SQLite database. An inactive agent has no resident process, so a large agent fleet costs almost nothing between events.
• Sharded web applications. One cell for each user, each tenant, or each device shards the application from the start. The contention of one shared database does not appear, because no shared database exists.

Contents

• What do you build with cells
• Install
• Configure object storage
• Deploy an application
• Operate D1 and KV
• Start a node
• Add nodes
• Shut down and roll out a node
• Diagnose a fleet
• List Durable Objects
• Environment variables
• Cloudflare compatibility
• What celld guarantees
• Limitations
• Security
• Telemetry
• Testing
• WebAssembly
• Rust library API

Install

The installer downloads the celld binary. Replication runs inside the celld process, so a node needs no external replicator.

curl -fsSL https://celld.dev/install.sh | sh

If the installer tells you, add ~/.local/bin to PATH. To install one exact release, set CELLD_VERSION to its tag, for example v0.0.1; to go back, run the installer again with the previous tag. The releases are on GitHub, and each release has a GitHub Actions build attestation: verify a downloaded file with gh attestation verify <asset> --repo denoland/celld.

Configure object storage

For an S3-compatible bucket, celld uses the standard AWS credential chain. On Amazon EKS, celld reads the Pod Identity credentials from the injected environment variables and the authorization-token file. For Cloudflare R2: create a bucket, create an S3 API token with access to that bucket, and set these variables:

export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
export AWS_REGION=auto
export S3_ENDPOINT=https://ACCOUNT_ID.r2.cloudflarestorage.com
export CELLD_BUCKET=s3://YOUR-BUCKET

For Google Cloud Storage, celld uses Application Default Credentials. Create a bucket. Then authenticate with gcloud auth application-default login, or point GOOGLE_APPLICATION_CREDENTIALS at a service-account key that has access to the bucket. Then set the bucket:

export CELLD_BUCKET=gs://YOUR-BUCKET

A gs:// bucket takes no S3_ENDPOINT and no AWS_* credentials, and celld ignores the storage region. On a Compute Engine instance, celld can use the attached service account, but the access scopes of the instance cap this credential, and the default scope permits only storage reads. Create the instance with the cloud-platform scope, so the IAM role of the service account controls the access.

For Azure Blob Storage, the bucket NAME is the container, and the storage account comes from the environment. Create a container. Then set the account and one credential:

export AZURE_STORAGE_ACCOUNT_NAME=YOUR-ACCOUNT
export AZURE_STORAGE_ACCOUNT_KEY=...
export CELLD_BUCKET=az://YOUR-CONTAINER

celld accepts three Azure credential families — a storage account key, a managed identity, and a workload identity — and you must configure exactly one. A system-assigned managed identity needs only the account name. A user-assigned managed identity needs exactly one selector among AZURE_CLIENT_ID, AZURE_OBJECT_ID, and AZURE_MSI_RESOURCE_ID, because two selectors can name two different identities. An identity must hold the Azure Blob data-plane permission to read, write, list, and delete blobs; the built-in Storage Blob Data Contributor role supplies it.

celld reads a managed identity from the Azure instance metadata service, which an Azure VM and an AKS node supply. Azure App Service and Azure Container Apps supply a different endpoint, so a managed identity does not work there; use a workload identity or an account key instead. The standard AKS workload identity environment (AZURE_AUTHORITY_HOST, AZURE_CLIENT_ID, AZURE_TENANT_ID, AZURE_FEDERATED_TOKEN_FILE) works with the public authority host, https://login.microsoftonline.com, with or without a trailing slash; celld rejects a sovereign or custom host. celld also rejects each recognized Azure configuration variable outside these families — another credential source, an endpoint override, the OneLake endpoint — and it ignores an AZURE_* name that object_store 0.12 does not recognize, because that name cannot change the client.

An az:// bucket takes no S3_ENDPOINT and no AWS_* credentials, and celld ignores the storage region. celld qualifies an az:// bucket for a production fleet: the conditional-write contract and the multipart upload path were tested against a live Azure account on 2026-08-18 (see guarantees). For local development against Azurite, set AZURE_STORAGE_USE_EMULATOR=true; Azurite is a development store, so celld does not qualify it for a fleet either.

The bucket credentials give full control of the fleet, so keep them safe. The bucket contains the deployments, the SQLite replicas, the ownership records, the node leases, and the peer-authentication secret.

A bucket value can add a key prefix: s3://YOUR-BUCKET/PREFIX. Every object of the fleet then goes below PREFIX/, so two fleets can share one bucket. A bucket value without a prefix keeps the objects at the root of the bucket, therefore an existing fleet does not move its data.

The store must provide conditional writes and read-after-write consistency, because the ownership records depend on them. Amazon S3, Cloudflare R2, Google Cloud Storage, Tigris, and Azure Blob Storage qualify; Backblaze B2, Hetzner, and DigitalOcean Spaces do not. MinIO (the community edition) passes the storage test, but celld has not qualified it for production; do not use RELEASE.2025-09-06T17-38-46Z, which rejects the conditional create that the first deploy sends (denoland/celld#162). See what celld guarantees for the exact requirements.

Deploy an application

If the project contains Worker code, install esbuild on PATH; an asset-only project does not need it. Then run celld deploy from an applicable Wrangler project:

git clone https://github.com/denoland/celld
cd celld/examples/counter
celld deploy . \
  --bucket "$CELLD_BUCKET" \
  --endpoint "$S3_ENDPOINT" \
  --region "$AWS_REGION"

celld deploy accepts module Workers, Durable Object bindings, service bindings, variables, cron triggers, D1 databases, KV namespaces, Workflows, WebAssembly modules, and static assets. An asset project can include a Worker or be asset-only, and the asset functions include the assets binding, HTML handling, not-found handling, worker-first routes, _headers, and _redirects. If the Wrangler configuration contains an unknown key, the deploy stops with an error. See the Cloudflare compatibility page for the complete deployment boundary.

A running node does not restart for a new deployment. Each node reads deploy/current.json every 30 seconds (CELLD_DEPLOY_POLL_S) and adopts a new deployment in place, and POST /reload on the internal listener makes a node adopt the pointer now. A node builds the new deployment beside the one it serves, and then it switches new requests to the new deployment in one step. A request that started on the previous deployment finishes on it. A deployment that does not build leaves the current deployment serving, and the node reports the failure in its log and in the /reload response. POST /reload also rebuilds an unchanged deployment, so an edit to CELLD_VARS_FILE takes effect without a restart.

A Durable Object that is not resident runs the new deployment at its next activation. A resident Durable Object moves to the new deployment at a safe point: no request runs in it, no alarm handler runs in it, no output waits for durability, and no regular WebSocket is open. A request that arrives while the object moves waits for the new code. The move keeps the object's storage, its epoch, and its hibernatable WebSockets, and it does not read or write the bucket. An object that reaches no safe point in CELLD_DEPLOY_MAX_AGE_S seconds (default 60) is forced: celld cancels its running work and closes its regular WebSockets with code 1012, which is what a Cloudflare deployment does to every object. A value of 0 forces every resident object at the adoption. In this window, a request on one deployment can call a Durable Object on the other, so two adjacent versions must accept each other's calls. The /state response reports the deployment a node serves, the deployments it still drains, the objects that are moving, and the deployment each resident object runs.

Develop an application locally

Run celld dev in a Wrangler project:

celld dev

The command opens a local object store, deploys the application, and starts one celld node. It does not require Docker or a cloud bucket. The Worker listener uses http://127.0.0.1:9876 by default. Use --port to select a different port:

celld dev --port 3000

Use --host to select the Worker listener interface:

celld dev --host 0.0.0.0

A non-loopback IP exposes the Worker listener to the network. The internal operator listener stays on loopback, so another machine cannot use its operator API.

The default display uses color to identify each status and the application URL. It hides the node warning and information logs, so the errors and the listener remain easy to find. Use --logs to show these logs:

celld dev --logs

Set NO_COLOR to disable color. Set FORCE_COLOR to enable color when the output is not a terminal. NO_COLOR always takes priority when both variables are set.

You can also give the command a project directory or a Wrangler configuration file:

celld dev ./examples/counter

The command stores the local objects and the celld work files in .celld/dev below the project directory. Add .celld/ to the application's .gitignore file. A normal shutdown keeps this directory, so the next invocation uses the same durable application state. Delete .celld/dev while celld dev is stopped to reset that state.

The command does not expose the local object store through a fleet flag. A regular node or an operator subcommand must use a supported cloud bucket.

The command watches the project directory. A source or configuration change builds a new deployment and restarts the local node. The current application continues to run during the build, and a failed build does not replace it. The restart retains the durable application state.

The watcher ignores .celld, .git, node_modules, and target directories at each depth. It does not watch a source file outside the project directory. Worker projects need esbuild on PATH, and asset-only projects do not need it.

Operate D1 and KV

celld d1 runs SQL and migrations against a deployed D1 database. The command uses the fleet bucket to find a node, and the node routes the operation to the database cell.

celld d1 migrations apply ledger --bucket "$CELLD_BUCKET"

celld kv reads and writes a deployed KV namespace. The bulk commands use the Wrangler file format, so wrangler kv bulk get can export data for celld kv bulk put.

celld kv bulk put sessions wrangler-export.json --bucket "$CELLD_BUCKET"

celld kv list prints at most 1000 keys, because a namespace can hold many more keys than an operator wants to read. The command reports on stderr that more keys exist, and it gives the --after KEY that continues the listing. Pass --all to read every key, or --json for one JSON object per key.

Every celld command writes its data to stdout and its messages to stderr, so a redirect or a pipe carries only data:

celld kv list sessions --all --json --bucket "$CELLD_BUCKET" > keys.ndjson

celld --help, celld --version, and the listener announcements use the same stdout sink. The sink treats a closed pipe as a successful stop, and it reports all other write and flush errors.

Start a node

For local development, the default listener is sufficient:

celld \
  --bucket "$CELLD_BUCKET" \
  --endpoint "$S3_ENDPOINT" \
  --region "$AWS_REGION"

For a fleet node, bind the public and internal listeners separately. The ingress can reach the public listener, and the other nodes can reach the internal listener:

celld \
  --bucket "$CELLD_BUCKET" \
  --endpoint "$S3_ENDPOINT" \
  --region "$AWS_REGION" \
  --listen 0.0.0.0:8080 \
  --internal-listen 10.0.0.12:8081 \
  --advertise node-a.internal:8081

An explicit advertised address requires an explicit internal-listener address: set both command-line options, or their equivalent environment variables. celld also rejects an explicit non-loopback public listener without an explicit internal listener, because that shape identifies an obsolete one-listener configuration.

celld cannot verify that an advertised hostname or a translated port reaches the internal listener. You must route the advertised address to the internal listener, and you must not route it to the public Worker listener.

Add nodes

Start each node with the same bucket settings. Give each internal listener a different address that the other nodes can reach, and set --advertise to that address. The nodes find each other through the leases in the bucket; there is no join command and no fixed membership list.

The bucket supplies discovery and authority, not network reachability. Cell fetch and RPC traffic has a protocol version but has no content signature, so the private network and its code are the security boundary. Peer-control and reserved-cell operator requests use the fleet HMAC, but celld does not terminate TLS. Put the advertised addresses on a private network that you trust, or use an encrypted overlay such as WireGuard or Tailscale. The internal listener also has an unauthenticated operator API, so it must not reach the public internet. See the security page for the complete boundary.

Shut down and roll out a node

celld shuts a node down gracefully on SIGTERM or SIGINT, the signals that systemctl stop, docker stop, and a Kubernetes pod delete send. The /.well-known/celld/health path reports the node as unhealthy, so a load balancer stops public routing to it. New public requests receive a 503 response, and celld closes each connection so the client can retry on a healthy node. The node finishes the public HTTP requests that it accepted before shutdown, and it continues to accept versioned peer traffic for cells that it has not handed off.

You must set a longer orchestrator stop grace, such as systemd TimeoutStopSec or Kubernetes terminationGracePeriodSeconds. The grace must cover the drain-token wait, the expected complete handoff, and one no-progress interval (the settings below). Otherwise, the orchestrator can send SIGKILL before celld completes the handoff.

The handoff works in batches. The node reserves a batch of cells and stops new local routes to them: an accepted request stays local until it finishes, and a new request waits for the successor route. The node proves the batch durable and tries to publish one full snapshot of each closed database. The snapshot is an L9 object, so the successor does not replay the complete transaction history. The remote L0 chain remains an additive fallback. If the snapshot retry window expires, the node releases the cell with this proven L0 chain. The node then releases each ownership record and asks a compatible peer to acquire it. The peer acknowledges after the ownership update and keeps the cell dormant, so the handoff does not restore an unused runtime; a later request starts the cell under the peer activation limit. The donor starts the next batch after each acknowledgement.

Four settings pace this work. CELLD_RELEASES sets the maximum number of complete handoffs in progress (default 8), covering activity cancellation, the durability proof, the final snapshot, the ownership release, and the successor update; CELLD_ACTIVATIONS limits the demand-driven restore and startup work. CELLD_SHUTDOWN_DRAIN_MS sets the maximum interval without a completed handoff (default 25000). Each successor acknowledgement starts the interval again, so a large handoff can continue while it makes progress. CELLD_SHUTDOWN_TOTAL_MS sets a 40000 ms default bound for the complete process stop. An orchestrator stop grace must be longer than this bound, so celld can finish its local durability shutdown. A same-node preserve operation uses the shorter drain value as its semantic limit because it has no successor acknowledgements.

Simultaneous stop signals do not flood the surviving nodes. A draining node claims a fleet drain token in the bucket before it releases cells, so concurrent donors hand off one node at a time. A donor that cannot claim the token within CELLD_DRAIN_TOKEN_WAIT_MS milliseconds (default 30000; 0 disables the token) proceeds without it, because the orchestrator grace is finite. The token is advisory: a dead holder's claim expires, and a handoff without the token is still safe.

A fresh process also holds its first healthy response until the fleet is settled. The process requires its live node lease, no active donor, and memory below every pressure low watermark on each live node. It also requires a total restore backlog no larger than one CELLD_ACTIVATIONS budget. The incumbent ownership counts must remain within one equal successor share above the fleet mean. A joining process can satisfy this condition when it advertises paced handoff support and owns fewer cells than the busiest incumbent. The process publishes this successor capacity before readiness, so an idle rollout can repair the ownership distribution during the next donor handoff.

An older peer does not publish the low-watermark result, so the gate uses that peer's pressure latch during a mixed-version update. An unreadable fleet or an unsettled condition holds readiness for up to CELLD_READY_FLEET_GATE_MS milliseconds (default 120000; 0 disables the gate). The process then reports healthy with a ready_gate_expired event. After the first healthy response, fleet state does not remove readiness again.

A deadline-cut handoff can leave a node-log recovery for the replacement. One process reads and uploads that dead session, and the other processes wait for its result. A waiting process can replace an unresponsive recovery after 30 seconds, so a failed recovery cannot block the fleet permanently. A recovery store error keeps the session unsealed, so a later recovery must read every retained bundle before the replacement can restore the acknowledged state.

The internal listener continues to accept /state requests during the drain, and the response reports the handoff and restore counters. The public health response identifies the active drain with a 503 status.

To roll out a new version, use the rolling update of your orchestrator: stop each node with SIGTERM, wait for its replacement to report healthy, then move to the next node. celld paces the cell handoffs inside each node shutdown, and the first-readiness gate paces the update against fleet recovery, so a deployer does not need a separate fleet-level handoff gate.

Some upgrades are exceptions:

• The upgrade from v0.1.0 to v0.2.0 must not be a rolling update. Stop every v0.1.0 node, then start the v0.2.0 nodes. Two changes require this: v0.2.0 nodes advertise the internal listener, so ownership records that v0.1.0 nodes wrote name an address that v0.1.0 peers can not follow to a v0.2.0 node; and v0.2.0 compacts replicated data into block objects that a v0.1.0 reader can not restore. A fleet must not mix the two versions.
• The upgrade from v0.2.1 to v0.3.0 can use a rolling update. v0.3.0 changes the default durability from bucket to fleet: a single node keeps the v0.2.1 behavior, and a fleet of two or more nodes activates fleet replication automatically. Stage the v0.3.0 binary on every node, and restart one node at a time. A mixed fleet stays safe, but a v0.3.0 node cannot replicate to a v0.2.x peer, so it acknowledges writes through the bucket and retries until the peer runs v0.3.0. Do not start a v0.2.x binary after that node runs v0.3.0 unless the shutdown log contains node-log close: sealed epoch. A graceful stop attempts this seal, but a stop under load can leave the record open, and a v0.2.x binary cannot read writes that wait in the replicated log or bundle objects — this downgrade can lose acknowledged writes.
• The upgrade from v0.3.0 to v0.4.0 must not use a rolling update. Stop every v0.3.0 node, and then start the v0.4.0 nodes. v0.4.0 moves every proxied cell call — the fetch, the RPC, and the WebSocket — onto one tunneled connection that carries plain HTTP, and the peer protocol refuses a different version, so the two versions cannot proxy calls to each other. v0.4.0 also stores each new large KV value under its ownership epoch and writes an epoch-qualified row reference. A v0.3.0 node cannot read that reference, so a mixed fleet can make a committed KV value unavailable. The tunnel establishment carries the version, so a later protocol change can negotiate instead of refuse.

The internal listener also provides an alpha operator API. /state reports the node state, POST /reload adopts the deployment pointer now, and POST /shutdown starts the same graceful handoff. The POST /shutdown?handoff=preserve request prepares a clean same-node reload and keeps the ownership records. A release can change this API, so keep the operator tooling and the celld release together.

Diagnose a fleet

celld diagnose reads the node leases in the bucket and sends a probe to each live peer. It does not take a lease, and it does not change ownership.

celld diagnose \
  --bucket "$CELLD_BUCKET" \
  --endpoint "$S3_ENDPOINT" \
  --region "$AWS_REGION"

To probe only some nodes, use --peer NODE_ID one or more times. The report identifies expired records, unsafe or incorrect advertised addresses, peers that it cannot reach, authentication failures, and protocol versions that do not agree.

Each node line also shows restoring: the count of cold routes that hold an activation permit or wait for one (a capacity waiter already holds a permit, so each cold route counts once). During a rolling update, wait for every node to report restoring=0 before you restart the next node, so one restart's cold work finishes before the next restart removes more warm capacity.

List Durable Objects

List the Durable Object instances in the bucket:

celld cell list \
  --bucket "$CELLD_BUCKET" \
  --endpoint "$S3_ENDPOINT" \
  --region "$AWS_REGION"

Each line is a Class:ID cell scope. The ID is the Durable Object ID: a 64-character hash unless the application supplies its own name. Give a class name to list only that class, and pass --json for one JSON object per line. An instance appears after the first event reaches it, because its owner then writes an ownership record to the bucket; an ID that an application only derives does not appear.

The listing also shows celld's own cells. A D1 database, a KV namespace, and a Workflow are each a cell in a reserved class, and their names start with __. They hold real data in the bucket, therefore the command shows them. The --json output marks each row with "reserved": true or false, so a script can select one kind:

celld cell list --all --json --bucket "$CELLD_BUCKET" |
  jq -r 'select(.reserved | not) | .scope'

A storage request returns at most 1000 instances, so the command stops at 1000 instances and writes this line to stderr:

1000 cells shown; more exist. Continue with --after Room:d99d9174b25e46310694dd931b47fbde70a7460bb7b210b546060651ea2ff6e0

Pass that --after SCOPE to read the next 1000 instances, or use --limit N for a different bound:

celld cell list --bucket "$CELLD_BUCKET" \
  --after Room:d99d9174b25e46310694dd931b47fbde70a7460bb7b210b546060651ea2ff6e0

Pass --all to read the whole listing in one command. This makes one request for each 1000 instances, so it can take minutes on a large fleet; the command reports its progress and its request count on stderr.

The order is the storage order, and --after continues from the last instance printed, so a sequence of --after commands lists each instance one time. An instance that an application creates during the sequence can appear or not appear, because the listing is not a snapshot.

The command writes the instances to stdout and every message to stderr, so a script reads only instance data:

celld cell list --all --json --bucket "$CELLD_BUCKET" > cells.ndjson

Hot-cell overload

celld admits a maximum of 64 concurrent fetch events for one Durable Object or Queue broker. Set CELLD_MAX_CELL_REQUESTS to use a different positive limit.

celld returns HTTP status 503 when the target reaches this limit, and it does not start the excess event. The response contains Retry-After: 1 and X-Celld-Overload: cell, so the application can retry or reject the work. A local or remote Queue owner uses the same status and headers when it refuses producer admission. The Queue response body is {"error":"cell admission refused"}. A fixed-rate client must count the response as rejected work, so an immediate retry does not increase the configured offered rate.

The runtime writes a cell_overload_refused log event when a target becomes saturated. The event contains the cell scope, the node, the region, the in-flight count, and the limit. Count these overload responses separately from the transport errors and the application failures.

Environment variables

For the full list, run celld -h. This table shows the primary settings:

The help output also shows the advanced tuning switches and their defaults.

An unset variable selects its documented default. A Boolean variable accepts only 0 or 1. celld exits during startup when a supplied value is invalid.