Skip to content

Run experiments

An experiment is a commit. You create one with submit, inspect it with diff and log, move between experiments with checkout, and read results on the dashboard.

Submit

submit snapshots your working tree, commits it as a new experiment, and queues it. There are three ways to use it, depending on how much you want the agent to own.

You wrote the code and it's final. Autolab commits + pushes it and moves you into the new experiment.

# ...edit code...
autolab submit -m "rotary embeddings"

Every field you set — including your code — is locked: the agent fills only the blanks (like a missing run command) and never overwrites your work.

A starting point you're happy for the agent to revise. You stay in your current experiment.

autolab submit -m "tune it for me" --soft

Just an instruction — the agent writes the code. Ignores your local changes. You stay in your current experiment. Requires -m.

autolab submit --nocode -m "try lr=0.3 with cosine decay"

Where do I end up after submit?

You advance into the new experiment only when the code is final — you pushed code and didn't pass --soft. With --soft (agent may revise) or --nocode (agent writes it), you stay put. This keeps your working tree predictable.

Nothing to submit?

A plain autolab submit with no changes and no message is refused — there'd be nothing for anyone to run or implement. Edit code first (autolab diff previews what would be sent), pass -m "<why>" to rerun the current code as a new experiment (e.g. to check run-to-run variance), or use --nocode -m "<idea>" to let the agent write the code.

Common options

autolab submit -m "bf16 + larger batch" \
  --name "bf16-bs512" \      # else the agent derives a name
  --run "python train.py" \  # else inherited / agent-filled
  --setup "uv sync" \        # environment prep before the run
  --node "8xH100" \          # node constraint
  --from <id> \              # branch from a specific experiment instead of main
  --priority 10              # queue priority

See all options in the CLI reference.

Inspect changes: diff

diff has two modes. With no argument it shows your uncommitted changes versus the experiment you're in — exactly what submit would send. With an experiment id it shows what that experiment changed versus the code it branched from. (Your .autolab/ control dir is never shown.)

By default it prints a summary — each changed file with its added/removed line counts, like git diff --stat. Pass -p/--patch to print the changed lines themselves:

autolab diff              # your changed files, ±lines each (new files included)
autolab diff -p           # …the full patch: the changed lines themselves
autolab diff <id>         # what experiment <id> changed vs its base, ±lines per file
autolab diff <id> -p      # …as a full patch

autolab diff <id> works once the experiment's code is final (it ran, or its code was locked). While the agent may still write or revise the code — queued or --nocode/--soft experiments — there is no meaningful diff yet and the CLI says so.

History: log

autolab log            # newest first; → marks the experiment you're in
autolab log --limit 50

log includes queued and pending experiments. The commit column stays empty until the code is final — for --soft/--nocode experiments the agent may still be coding it.

Lineage: graph

Where log is a flat list, graph shows the branch tree — what each experiment was forked from, so you can see how the research actually evolved:

autolab graph                 # the lineage tree; → marks the experiment you're in
autolab graph --root <id>     # only one experiment's subtree
autolab graph --format json   # the same graph as edges, for scripts and agents

alice/nanochat
└── 9f2a01c4  merged  loss 0.42  baseline
    └── 7b1e88aa  merged  loss 0.31  rotary embeddings
        ├── → 4c0d12ff  running  bf16 + larger batch
        └── a83b77e1  merged  loss 0.25  cosine decay

Each node shows the short id, status, primary metric, and title. A child hangs under the experiment it branched from — the one you chose with submit --from <id>, or the latest merged experiment by default. Status is colour-coded the same way as the dashboard's graph view.

Only experiments that have started appear — queued/pending ones have no lineage yet, so use log to see those. --format json emits one node per experiment with its resolved parent_experiment_id, parent_source (parent_id or base_commit), depth, and is_current — handy for an agent reasoning over the tree.

Move between experiments: checkout

Reset your working tree to any experiment's code, or back to main:

autolab checkout <id>       # this experiment's code
autolab checkout main       # the current main branch
autolab checkout <id> -f    # discard local changes while switching

checkout fetches first so the target commit is present. If you have uncommitted changes it stops and tells you how to keep them (submit) or discard them (-f).

Stay current: pull

autolab pull    # fetch the latest and fast-forward main

Read results

An experiment's run logs and the agent's activity stream on its dashboard page — open it straight from the terminal with autolab open <id>.

Artifacts aren't captured yet

The runner does not upload checkpoints or files, only logs.

Cancel

autolab cancel <id>    # cancel a queued or in-flight job

Cancelling a running job also stops it on its node — a not-yet-started assignment is dropped, and an in-flight run is killed (scancel on Slurm clusters) so the GPU frees up right away.

Open in the browser

autolab open <id>      # the experiment's dashboard page

Most experiment IDs accept a short prefix. If a prefix is ambiguous the CLI tells you — see Troubleshooting.