> See the full index of documentation pages at https://docs.zettabyte.com/llms.txt.

# Everyday commands

> Status, logs, shell, exec, chat, and the GPU box.

Source: https://docs.zettabyte.com/docs/cli/everyday-commands

Last updated: 2026-10-17

Every verb takes the workload name (or id). If a name exists in more than one kind, disambiguate with kind/name (`zb logs job/train-1`) or `--kind`.

## Watch and control

| Command | What it does |
| --- | --- |
| `zb status [NAME]` | one table across kinds, or a detail view; `--watch` refreshes the list view; alias `st` |
| `zb top [NAME]` | live GPU utilization and serving metrics; `-w` refreshes until interrupted |
| `zb wait NAME` | block until `RUNNING` or an end state; `--for STATUS`, `--timeout DURATION` |
| `zb logs NAME` | tail or follow with `-f`; `--tail N`, `--role`, `--pod`, `--container` |
| `zb events NAME` | recent platform events (scheduling, image pulls, restarts, out-of-memory kills); `--pod`, `--type warning` |
| `zb describe NAME` | the deep view: full spec, per-pod runtime, health details, nothing capped |
| `zb doctor NAME` | match logs and events against known failure signatures and suggest fixes |
| `zb pods NAME` | list a workload's pods (roles, replicas, status, nodes) |
| `zb stop NAME`, `zb start NAME` | serve only: scale to zero / resume (start takes `--replicas MIN[:MAX]`) |
| `zb scale NAME MIN[:MAX]` | serve only: set replica bounds ("2" means exactly 2) |
| `zb ttl NAME [DURATION\|off]` | show, extend, or clear the autoclean TTL |
| `zb down NAME...` | delete, any kind; volume data persists; alias `rm` |

## Get inside

| Command | What it does |
| --- | --- |
| `zb box [NAME]` | create a box if needed and shell in, one command (see Boxes below) |
| `zb shell NAME` | interactive shell in any workload; `--pod`, `--role`, `--replica` |
| `zb exec NAME -- CMD...` | one-off command; zb exits with the remote exit code |
| `zb url NAME` | serve: OpenAI-compatible endpoint; dev: IDE URL; box: none (use `zb shell`); `--open` |
| `zb chat [NAME] [PROMPT]` | talk to a serve: REPL, one-shot, stdin pipe, `-o json` |

## Code and results

| Command | What it does |
| --- | --- |
| `zb push NAME [DIR]` | copy a local directory into a running workload (default: cwd to `/workspace`) |
| `zb sync NAME [DIR]` | like push, but only files whose content changed; `--delete`, `--dry-run` |
| `zb pull NAME [DEST]` | download the outputs mount (works after `zb down`; `--watch` keeps syncing) |
| `zb cp SRC DST` | small files; one side is `NAME:/path` (uploads cap at 64 MiB) |

## Multi-pod workloads

Multi-pod workloads are jobs with `roles:` or `nodes` above 1, and serves with more than one replica. They show every pod's logs by default, in `-f` too. Each line is prefixed `[rollout-0]` style (`[0]` and `[1]` for serve replicas). `zb pods NAME` is the canonical list, with role, replica, status, and node per pod (`-o json` for scripts). `zb status NAME` shows a rollup plus the same per-pod rows.

Jobs narrow with `--role trainer` (that role's pods), `--role trainer --replica 1` (one pod, raw output), or `--pod trainer-1`. Serves narrow with `--replica 1` or `--pod`. The same flags pick the pod for `zb shell`, `zb exec`, and `zb cp` (default: replica 0, preferring running pods).

In `-o json` the multi-pod logs shape carries one entry per pod, with per-pod errors, so scripts see partial failures without the command failing:

```json
{
  "job_id": "J01ABC",
  "name": "rl-disagg",
  "pods": [
    { "pod_name": "trainer-0", "role": "trainer", "replica": 0, "container": "main", "log": "..." },
    { "pod_name": "rollout-1", "role": "rollout", "replica": 1, "container": "main", "error": "..." }
  ]
}
```

## Is it actually using the GPUs?

`zb status` says a workload is running. `zb top` says whether it is doing anything. With no argument it prints one line per workload: average GPU utilization, memory, and power across every card it holds. `idle` is the finding. A workload holding GPUs at near-zero utilization is usually stuck on data loading, running on CPU by mistake, or wedged. It bills the whole time. A serve that is still loading its weights reads `loading` instead, not idle.

With a NAME you get per-card detail. For a serve you also get the model server's own view of itself: requests running and waiting, KV-cache use and prefix hits, token throughput, and latency percentiles. `-w` refreshes until you interrupt it. It also switches the rates from totals-since-start to the last few seconds. `--pod` narrows to one pod. `--wide` adds every card plus per-pod CPU, memory, and node. `--raw` (serve only) prints the model server's untouched metrics. `-o json` is the stable shape for scripts. GPU samples come from a collector that runs about once a minute, so a workload that just started has nothing to report yet.

## Debugging a crash loop (--debug)

A container that crashes at startup normally restarts immediately. Typical causes are a wrong serve flag, a model that does not fit, or a bad path. Every restart wipes the one thing that names the mistake: the traceback. Relaunch with `--debug` (spec: `debug: true`, job and serve) and a crash holds the pod instead. The command runs. If it exits, the container prints the exit code and parks rather than dying.

```bash
zb launch -k serve --model llama-3-70b --gpu h100:4 --debug --ttl 4h
zb logs my-serve        # the full traceback, still there
zb shell my-serve       # get in, edit flags, rerun the command by hand
```

Iterate inside the held pod until the command starts cleanly, then relaunch without `--debug`. For a managed serve with no `run:` of your own, the platform fills its stock runtime command into the spec first. The logs then show exactly what ran. A held pod looks running but never becomes ready. It keeps its GPU until `zb down`. Pair `--debug` with `--ttl` so a forgotten pod cleans itself up.

## Digging deeper: events, describe, doctor

- `zb events NAME` shows the event trail, newest first: what the platform did with the workload. That covers scheduling waits, image pulls, restarts, and out-of-memory kills. `zb status` derives its one-line verdict from these events. The trail shows the sequence. Most events expire after about an hour, so look while the trouble is fresh. `--pod` narrows to one pod. `--type warning` shows just the trouble.
- `zb describe NAME` shows everything about the workload, with nothing capped. You get the full spec (every env var with secrets masked, mounts, role blocks, priority). You also get every pod with its raw state and node, and the health details behind the verdict. `-o json` emits the same document as `zb status -o json`: one machine shape, not two.
- `zb doctor NAME` does failure triage. Startup crashes read as unique, but they are highly patterned. Doctor matches the log tail and events against known signatures and suggests the fix for each match. The signatures include out-of-GPU-memory while sizing the KV cache, missing model files, tensor-parallel mismatches, and container-level OOM kills. To check how much GPU memory a model needs before you deploy it, see [GPU memory calculator for LLMs](https://docs.zettabyte.com/docs/deploy-and-serve/gpu-memory-calculator). It works on a live, crash-looping, or `--debug`-held workload alike.

Where the platform reports it, restart reasons annotate the counts everywhere they appear. `zb pods` shows `4 (oom 12m ago)` instead of a bare `4`. The raw form appears in a last-state column under `--wide` and as `last_state` in `-o json`.

## Chat with a serve

`zb chat [NAME] [PROMPT]` is the paved road to a serve endpoint. With no PROMPT it is a REPL (`/exit`, `/clear`, and Ctrl-C stops a reply without leaving). With a PROMPT it is a one-shot. A stdin pipe works for scripts, and `-o json` returns the raw completion. With exactly one running serve, bare `zb chat` targets it. The CLI mints and stores an inference API key automatically on first use (`--inference-key` or `ZB_INFERENCE_KEY` overrides it). Replies go to stdout. Everything else goes to stderr.

For SDKs and raw HTTP, create a key with `zb key create NAME` (the secret shows exactly once). Then point any OpenAI SDK at `zb url NAME`, with the key as the Bearer token and a `Model-Name` routing header. The body's `model` field is the runtime-registered name. `GET <url>/v1/models` returns it. See the Inference API pages for the HTTP details.

## Boxes

`zb box` is `docker run -it` for cloud GPUs. One command creates the box if it does not exist, waits for `RUNNING`, and attaches a shell. With no boxes and no `--image`, it creates one on the default preset: the zero-setup GPU sandbox.

```bash
zb box --gpu h100:1                       # no boxes yet? creates one on the default preset and shells in
zb box dev1 --image pytorch --gpu h100:1  # named create: pass an image (any pullable ref or a preset)
zb box dev1                               # re-attach any time
```

- Exiting the shell leaves the box running. The pod's lifetime is your image's ENTRYPOINT, not the shell session. `zb box dev1` re-attaches. With exactly one box live, bare `zb box` attaches to it. `zb down dev1` deletes it.
- Your own image must keep a foreground process running (for example `CMD ["sleep", "infinity"]`, an sshd, a server). If the ENTRYPOINT exits, the box crash-loops. `zb box` says so and how to fix it. Platform preset images (`--image pytorch`, or the bare-create default) are exempt. They get a keep-alive shell automatically.
- Boxes are persistent by default: no TTL unless you pass `--ttl 8h`.
- `--workdir .` ships local code (read-only at `/workspace`). `--volume`, `--env`, and `--cpu`/`--memory`/`--gpu` work like `zb launch`. `-d` creates without attaching. `zb push`, `zb cp`, and `zb exec` all work on a box.
- A box has no URL (`zb url` says so) and, like dev, no logs endpoint: the shell is the contract.
- `kind: box` is the only lane that runs your image's own ENTRYPOINT. When you want the ENTRYPOINT, use a box.

> [!NOTE]
> **Box or dev?**
> `kind: box` is the lane for running your image's own ENTRYPOINT and working in a shell. If you want a browser IDE (Jupyter or VSCode) instead, use `kind: dev`.