Manifest reference and multi-role jobs
Every spec key, how the four layers merge, and composing one job from several roles.
Verified with zb v0.1.
How a launch is assembled
`zb` assembles a launch from up to four layers. Higher layers override lower ones. It validates the merged result as a whole (unknown keys, kind rules, role rules) before it sends anything. `TYPE:COUNT` stands for a GPU type listed by `zb gpus` and a card count.
| Layer (lowest to highest) | Example | Notes |
|---|
| dotenv files | `env_file: .env`, `--secrets-file .env.secrets` | only fill env and secret keys nothing else already set |
| spec file | `zb launch train.yaml` | the paved road; `-` reads stdin |
| flags | `--gpu TYPE:COUNT --env SEED=1` | override the file field by field |
| `--set` overrides | `--set serve.replicas.max=3` | win over everything |
Merge rules
- Scalars: a set flag replaces the file's value (`--name`, `--gpu`, `--ttl`, and so on).
- `env:` and `secrets:` merge per key. The flag wins on a collision, and other keys from the file survive.
- `volumes:` (and every repeatable list flag): the flag list replaces the file's list entirely. Lists never splice.
- `--set path=value`: a dotted path into the manifest (`--set env.SEED=42`, `--set serve.replicas.max=3`). The value parses as a YAML scalar, so numbers and booleans keep their types.
- `zb` folds dotenv files in last, but they lose to every explicit key. A key in both `env_file` and `secrets_file` with no explicit winner is an error.
The full manifest
The table lists every key the spec accepts. `zb` rejects unknown keys.
| Key | Kinds | Meaning |
|---|
| `kind` | all | required: `job` | `dev` | `serve` | `box` (or `--kind`) |
| `name` | all | auto-generated when omitted; lowercase letters, digits, and dashes, max 40 chars |
| `description` | all | free text |
| `image` | job, dev, box | any pullable ref, or a preset name like `pytorch`; omit on `job` and `box` for the default preset; serve image mode takes the imported runtime image id |
| `run` | job, image-mode serve | the command; required on `job`, rejected on `dev` and `box` |
| `workdir` | job, dev, box | ship this local directory, read-only at `/workspace` |
| `outputs` | job, dev, box | results mount (absolute container path), downloadable with `zb pull` even after `zb down` |
| `outputs_s3` | job, dev, box | write results straight to your own S3 bucket: `{url, region, access_key, secret_key}`; needs `outputs:` set |
| `share` | job, dev, box | mount the tenant share volume at this absolute container path |
| `ttl` | all | autoclean after DURATION, anchored at launch |
| `resources.gpu` | all | `TYPE[:COUNT]` as listed by `zb gpus`; a bare type means 1; required on `serve` |
| `resources.cpu`, `resources.memory` | all | cores / GB; omit on GPU workloads for per-card sizing |
| `nodes` | job, serve | job: above 1 = distributed (gang-scheduled), exclusive with `roles:`; serve: shard one replica across N nodes (needs `replicas.min == max`) |
| `max_restarts` | job | automatic restart budget; a restart re-runs the whole gang from scratch |
| `roles` | job | multi-role gang, see below |
| `debug` | job, serve | hold a crashed container instead of restarting it, so logs keep the traceback (see Everyday commands) |
| `env` | all | plain env vars |
| `secrets` | job, serve | env vars marked sensitive; rejected on `dev` and `box` (see the secrets tiering callout on Launch and specs) |
| `env_file`, `secrets_file` | all | local dotenv folded in at lowest precedence; `secrets_file` is rejected on `dev` and `box` when the file holds secrets |
| `volumes` | job, dev, box | `NAME:/mount/path` entries; rejected on `serve` (model artifacts are mounted by the platform) |
| `queue` | all | default true: park `QUEUED` on quota exhaustion |
| `priority` | job, serve | 1 to 6, higher schedules first |
| `serve.model` | serve | managed mode (model name, or a model id starting with `M0`); add `image` plus `run` to serve it on a custom runtime (weights mount at `/home/model`) |
| `serve.served_name` | serve | image mode: the model name on the OpenAI API (required in image mode) |
| `serve.replicas` | serve | `{min, max}`; default `{1, 1}`; min 0 launches stopped; `zb stop` scales to zero afterwards |
| `dev.ide` | dev | `jupyter` (default) | `vscode` | `none`; `none` runs the image's own ENTRYPOINT with no IDE, and `kind: box` is the lane for a shell on it |
Exact syntax by kind
Manifests parse as YAML or JSON. `zb init -k KIND -` prints a ready-to-edit template for any kind. Below is every key at its exact spelling and nesting, per kind. `zb` rejects unknown keys and keys that break a kind rule at validation, before the request goes anywhere. A spec that passes that check can still be refused by the platform.
kind: job
kind: job
name: llama-ft # optional; lowercase letters, digits, dashes; max 40
description: llama finetune # optional
image: pytorch # preset | @TAG | @fav | any pullable ref; omit for the default preset
run: torchrun train.py # required
resources: # omitted entirely = CPU-only default sizing
gpu: TYPE:COUNT # a bare type means 1
cpu: 96 # cores (number); omit to size per card
memory: 1024 # GB (number); omit to size per card
nodes: 2 # >1 = distributed; mutually exclusive with roles:
max_restarts: 1
priority: 5 # 1-6, higher schedules first
queue: true # false = fail fast when GPU quota is exhausted
debug: true # hold a crashed container instead of restarting it
workdir: . # local directory to ship (read-only at /workspace)
outputs: /outputs # absolute container path
# outputs_s3: # optional: results write straight to your own bucket
# url: https://ENDPOINT/my-bucket/run-42
# region: us-west-2 # optional
# access_key: ACCESS_KEY
# secret_key: SECRET_KEY # use a bucket-scoped, fine-grained key
share: /share # absolute container path
ttl: 8h
volumes:
- training-data:/data # NAME:/mount/path
env:
WANDB_PROJECT: llama
secrets: # job and serve only
HF_TOKEN: TOKEN_VALUE
env_file: .env
secrets_file: .env.secrets
kind: serve
kind: serve
name: my-serve
resources:
gpu: TYPE:COUNT # required on serve
serve:
model: MODEL_NAME # managed mode: model name, or a model id starting with M0
replicas: # optional; default {min: 1, max: 1}; min 0 = launch stopped
min: 1
max: 3
# custom runtime for a managed model: keep serve.model, add image + run
# (weights mount at /home/model):
# image: IMG01... # imported runtime image id (zb image import prints it)
# run: python3 -m vllm.entrypoints.openai.api_server --model /home/model ...
# image-only mode (no serve.model; requires tenant enablement):
# image: IMG01...
# run: python3 -m vllm.entrypoints.openai.api_server ...
# serve:
# served_name: my-model # required in image mode: the name on the OpenAI API
# nodes: 2 # shard ONE replica across N nodes (model too big
# # for one node); needs replicas min == max
# also valid: description, env, secrets, env_file, secrets_file, ttl,
# queue, priority, debug. volumes: and share: are rejected on serve
kind: dev
kind: dev
name: notebook-1
image: pytorch # preset | @TAG | any pullable ref
dev:
ide: vscode # jupyter (default) | vscode | none
# also valid: description, resources, workdir, outputs, share, ttl,
# volumes, env, env_file, queue. run: and secrets: are rejected on dev
kind: box
kind: box
name: dev1
image: pytorch # a preset gets a keep-alive shell; a custom image's ENTRYPOINT must keep running
# also valid: description, resources, workdir, outputs, share, ttl,
# volumes, env, env_file, queue. run: and secrets: are rejected on box
Multi-role jobs
`roles:` composes one job out of several differently shaped pod groups. Examples: a trainer role beside a rollout role serving vLLM (the disaggregated-RL / verl topology), or a parameter server beside workers. Use it anywhere unlike pods must live and die as one unit.
kind: job
name: rl-disagg
image: ghcr.io/acme/rl:cu124 # top-level values are the defaults
resources:
gpu: TYPE:COUNT
roles:
- name: trainer # each role overrides what it needs
run: python train.py
- name: rollout
run: python -m vllm.entrypoints.openai.api_server --model /ckpt
resources:
gpu: TYPE:COUNT # gpu overrides as a TYPE:COUNT unit
replicas: 2 # pods for this role (default 1)
env:
VLLM_PORT: "8000" # merged over the job-level env
- `roles:` is job only, and `nodes:` is mutually exclusive with it. The gang size is the sum of role replicas.
- Defaults inherit downward. A role that omits `image`, `run`, or `resources` uses the job's top-level value. Every role must end with an effective image and run. `resources.gpu` inherits as one `TYPE:COUNT` unit. `cpu` and `memory` inherit individually. Role `env` merges over job `env`.
- Role names are lowercase letters, digits, and dashes (max 40), unique within the job. The name is what you see in `zb pods` (`trainer-0`, `rollout-1`). It is also the role's network hostname inside the job.
- One gang: the platform co-schedules every role as one unit. The job starts only when every pod of every role can start. All roles share the job's network, so intra-job DNS reaches every role across nodes.
- Tooling is role-aware. Multi-pod `zb logs` prefixes lines per pod. It narrows with `--role rollout`, or with `--role trainer --replica 1` for one raw stream. `--role` and `--pod` pick the target for `zb shell`, `zb exec`, and `zb cp`. `zb pods NAME` is the canonical list.
Topology rules for RL post-training
RL fine-tuning runs colocated. The rollout engine (vLLM) lives in the same job as the trainer and shares its GPUs (TRL's `vllm_mode` colocate, verl's hybrid engine). The topology rule on this platform is simple. Jobs have no inter-workload networking or ports, so the trainer cannot reach a separate rollout server. A `serve` endpoint cannot receive weight updates either. A frozen reward model or judge is the exception. Serve it with `kind: serve` and call its public URL (`zb url`, `zb key`) from the training loop. A single-pod colocated job does not need `roles:` at all. That is a plain job.
Distributed and multi-node jobs
Set `nodes: N` and the job gang-schedules across nodes. The platform injects `MASTER_ADDR`, `MASTER_PORT`, `WORLD_SIZE`, and `RANK` into job pods. A `torchrun` command that reads those variables runs a distributed trainer with no extra wiring (rank 0 hosts the rendezvous). Request the node's full GPU complement per node (`gpu: TYPE:COUNT`, with the count the node holds). Multi-node is meant to run whole nodes. If job pods have no Hugging Face or PyPI egress, bake dependencies and weights into the image.
Two gotchas. First, `nodes: N` sets the number of pods, not the number of machines. Requesting the node's full GPU count per pod is what spreads the pods across machines. Second, `RANK` and `WORLD_SIZE` are node-level, not global. `torchrun` and `accelerate` (`--machine_rank`/`--num_machines`) expand them correctly. A raw `torch.distributed` `env://` launch with one process per GPU must recompute the global rank itself (`RANK*nproc + local_rank`).
Multi-node serve
`nodes: N` on a `serve` shards one replica across N nodes (a leader plus N-1 workers). Use it for a model too big for a single node. It needs `replicas.min == max`, because a sharded replica cannot autoscale. Unlike jobs, the platform does not run a launcher for you. Every pod runs the same `run:`. You differentiate the worker yourself off `$LWS_WORKER_INDEX` (0 = leader), with `$LWS_LEADER_ADDRESS` as the head address. The platform injects the cross-node network interface as `NCCL_SOCKET_IFNAME` and `GLOO_SOCKET_IFNAME`.
run: |
EXTRA=""; [ "${LWS_WORKER_INDEX:-0}" != "0" ] && EXTRA="--your-frameworks-worker-flag"
your-serve-cmd --head-address "$LWS_LEADER_ADDRESS" $EXTRA ...
Two things will cost you time if you skip them. First, launch with `--debug`. Otherwise the platform recreates a crashed serve pod before you can read the traceback. Second, raise your serving framework's engine-ready timeout. Big models take many minutes to initialize across nodes, and the default is often too short. The framework's own multi-node environment (host-IP overrides, shared-memory transport, ready timeout) is your responsibility.
Next steps
[Everyday commands](https://docs.zettabyte.com/docs/zcloud/cli/everyday-commands): watch and manage a launched workload.