Droid in CI

In CI nobody is at the prompt. Everything a person would have approved has to be decided in advance by configuration. That makes CI the strictest test of your harness, and the place it earns its keep.

This guide collects the patterns for droid exec, the headless mode of the Droid CLI. Lab 9 builds one workflow end to end. This guide is the wider set.

The five facts that shape every pattern

droid exec is read-only by default. Without --auto, it can inspect files and run git reads (git status, git log, git diff), and nothing else.

--auto takes low, medium or high. There's no --auto off; leave the flag out for read-only.

Flag	Adds
(none)	Read-only inspection and git reads
`--auto low`	Project file creation and edits
`--auto medium`	Installs, builds, tests, `git commit`, `git checkout`, `git pull`
`--auto high`	Remote writes, deploys, migrations

It fails fast. If a requested action needs more autonomy than the run has, droid exec stops with a clear error, returns a non-zero exit code and makes no partial changes.
ask rules fail the run. A one-shot run can't collect an approval, so an effective ask isn't cleared even by --auto high. This is a feature: a git push ask rule means CI can't push, whatever the prompt says.
Sandbox violations are auto-denied. Interactively they prompt. In droid exec they're denied without asking.

And one fact that trips people up: in droid exec, -r means --reasoning-effort. In interactive droid, -r means --resume.

This is what happens to one shell command in a headless run. The orange nodes are decided by configuration, before the model's opinion matters:

Authentication and install

CI signs in with an API key. Create one at app.factory.ai/settings/api-keys and store it as a secret named FACTORY_API_KEY. The CLI reads it from the environment.

Install with npm and pin the version. npm builds ship with auto-update turned off, so every run uses the version you tested.

steps to reuse
- uses: actions/setup-node@v4
  with:
    node-version: 20
- run: npm install -g droid@0.174.0   # pin the version you tested
- run: droid --version

If you use the standalone installer (curl -fsSL https://app.factory.ai/cli | sh, then add $HOME/.local/bin to the path), it auto-updates by default. Select a version with droid update --version <version>, then set FACTORY_DROID_AUTO_UPDATE_ENABLED=false in the environment of every later step. Once that variable is set, droid update won't install another version, so do it in that order.

Pattern 1 — read-only analysis

The safest CI job. No --auto, so Droid can read and run git reads but can't change anything. Use it for pull-request summaries, risk notes, release notes and "does this diff touch an invariant?" checks.

> git diff origin/main...HEAD | droid exec "List every change that touches money arithmetic, rounding or account-number handling. Cite file and line."

Piping a diff in keeps the prompt about the change and nothing else. Put a longer prompt in a file and pass it with -f.

Pattern 2 — structured output and exit codes

-o json returns a single JSON object when the run ends. Branch on the exit code first, then read the fields.

scripts/droid-step.sh
#!/usr/bin/env bash
set -uo pipefail

droid exec --auto low -o json -f .ci/prompts/docs-update.md > droid-result.json
status=$?

jq -r '.session_id' droid-result.json   # keep this: it links the run to the session
jq -r '.result' droid-result.json > droid-summary.md

if [ "$status" -ne 0 ] || [ "$(jq -r '.is_error' droid-result.json)" = "true" ]; then
  echo "droid exec failed (exit $status). See droid-summary.md."
  exit 1
fi

The documented fields are type, subtype, is_error, duration_ms, num_turns, result and session_id. Upload the JSON as a build artifact. The session_id is how a reviewer gets from a pull request back to the session that produced it, which is the traceability a model-risk reviewer asks for.

Other output formats are text (the default), stream-json and stream-jsonrpc. The docs describe stream-json input as deprecated in favor of stream-jsonrpc. Check the current docs before you build on streaming.

Pattern 3 — bounded changes, human push

When CI makes a change, give it the lowest autonomy that works and let a person push.

--auto low for edits only: docs, generated files, simple fixes.
--auto medium when it needs to run tests or commit locally.
Never --auto high on a pull-request trigger. High permits remote writes and migrations. Keep it for jobs a person starts deliberately.

With a git push ask rule in .factory/settings.json, a run that tries to push fails instead. The job then pushes, or opens a pull request, with its own token in a separate step that you wrote and reviewed. Droid proposes, the pipeline you control publishes.

To add CI-only settings without changing the repository's file, --settings <path> merges a settings file on top "for this process only". Be careful what you put in it. That file is the Runtime level, and for permission rules Runtime claims an ID before Folder, Project and User do. A CI file with its own project/git-push rule set to allow would shadow the repository's ask. Only Org rules outrank it.

The session_id in the JSON names the run, but the session itself lives on the runner, which disappears with the job. With cloudSessionSync on (the default), CLI sessions are mirrored to Factory's web app, which is where you'd read a CI session afterwards. If your organization turns sync off, keep whatever evidence you need as artifacts.

Pattern 4 — restrict the tools

Autonomy controls approval, not which tools exist. To narrow the toolset for one run:

--only-tools <ids> allows only the listed tools
--remove-tools <ids> turns specific tools off
--add-tools <ids> adds tools to the default set
--list-tools prints the tools available for the selected model, so you can get the IDs right

Each flag also takes MCP:<server> (every tool from that server) or MCP:<server>/<tool> (one tool), so --only-tools 'Read,Grep,MCP:github' is a read-only reviewer with GitHub access. The docs still list these flags under their older names, --restrict-tools, --additional-tools and --disabled-tools. Current CLI builds accept the old names as hidden aliases, but droid exec --help shows only the new ones.

Tool IDs are case-sensitive. The shell tool is Execute. Which file-editing tools exist depends on the model: some models get Create and Edit, others get only ApplyPatch. Run --list-tools with the model your pipeline uses before you write a tool list. That's also why the harness hook matcher is Create|Edit|ApplyPatch: it covers every variant, so the hook fires whichever model is running.

Pattern 5 — the regression scoreboard

A set of real tasks with automatic checks, run headless, is the best regression test for a harness. Larkspur's runs droid exec --auto medium -o json -f scoreboard/tasks/Tn.md in a fresh worktree per run. Keep cost under control:

On pull requests that touch .factory/ or AGENTS.md: one run per task.
Nightly: the full run, three per task.

.github/workflows/droid-scoreboard.yml (illustrative excerpt)
on:
  pull_request:
    paths: ['.factory/**', 'AGENTS.md']
  schedule:
    - cron: '0 3 * * *'

jobs:
  scoreboard:
    runs-on: ubuntu-latest
    env:
      FACTORY_API_KEY: ${{ secrets.FACTORY_API_KEY }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm ci
      - run: npm install -g droid@0.174.0
      - name: Scoreboard
        run: |
          RUNS=$([ "${{ github.event_name }}" = "schedule" ] && echo 3 || echo 1)
          npm run scoreboard -- --runs "$RUNS" --json > scoreboard.json
      - uses: actions/upload-artifact@v4
        with:
          name: scoreboard
          path: scoreboard.json

The scoreboard script is Larkspur's, not Droid's. Yours will be a script that loops over task files and checks results. The pattern is what transfers.

Pattern 6 — parallel runs in worktrees

-w, --worktree [name] runs a task in its own git worktree, so parallel runs don't trip over each other. In exec, a clean worktree is removed on exit and a dirty one is kept. The branch is never deleted. To choose where worktrees go, set worktreeDirectory in ~/.factory/settings.json. (The older --worktree-dir <path> flag still parses, but the CLI help marks it deprecated.)

> droid exec --worktree codemod-a --auto medium "apply codemod A" &
> droid exec --worktree codemod-b --auto medium "apply codemod B" &
> wait

Pattern 7 — the protected-path check

This one doesn't use Droid at all, and it is the most important job in the file. Diff the pull request against its base and fail if a protected path changed without an approval your process recognizes. It holds whoever made the change: Droid, a person, or a script. It also holds whether or not a hook fired. Factory's docs don't state that project hooks in .factory/hooks.json run under droid exec, so confirm it on your version (Lab 5 walks through the check) and don't let a headless job's safety rest on it.

protected-paths job (excerpt)
- uses: actions/checkout@v4
  with:
    fetch-depth: 0
- name: Protected paths
  run: |
    changed=$(git diff --name-only origin/${{ github.base_ref }}...HEAD -- src/money migrations legacy)
    if [ -n "$changed" ]; then
      echo "Protected paths changed:"; echo "$changed"
      echo "Needs an approved change ticket and a human reviewer."
      exit 1
    fi

Larkspur's Stop hook also writes .factory/protected-changes.log when a session ends with a protected change. That log is gitignored, so it stays on the machine that ran the session and CI never sees it. It is local evidence; the pull request diff is the evidence CI trusts. See Lab 5 and Lab 9.

Pattern 8 — Droid code review on pull requests

Factory ships a GitHub Action for automated review, Factory-AI/droid-action (the same repository has a GitLab CI/CD component). The quickest setup is /install-code-review inside droid: it detects GitHub or GitLab, checks prerequisites, walks you through depth, security and triggers, and opens a pull request with the workflow files. Without a github_token input, the action posts as the Factory Droid GitHub App, which must be installed on the repository from your Factory organization settings; that is what the id-token: write permission is for.

review step
permissions:
  contents: write
  pull-requests: write
  issues: write
  id-token: write

steps:
  - uses: Factory-AI/droid-action@main
    with:
      factory_api_key: ${{ secrets.FACTORY_API_KEY }}
      automatic_review: true
      review_depth: deep   # or shallow

Put your team's review rules in .factory/skills/review-guidelines/SKILL.md; the docs say the action picks them up with no workflow change. The documented review workflow runs on opened, synchronize, reopened and ready_for_review, skips drafts, and posts inline comments. The docs say it focuses on clear bugs and skips style and architecture opinions, so put anything your team treats as a blocker in the guidelines file. (The P0 to P3 severity grades are documented for the local /review command, not for the action.)

Two cautions for a regulated repo. The action reference is @main. Pin it to a commit your security team has reviewed. And Droid approves a pull request when it finds no issues. Decide in advance whether that approval counts toward your required reviews. For most regulated teams it shouldn't. Configure branch protection so a human approval is still required.

Pattern 9 — Missions, headless

A Mission can run headless:

> droid exec --mission --auto high -f mission.md

The docs disagree here. The CLI reference and the Droid Exec page say --mission requires --auto high or --skip-permissions-unsafe, while the headless examples in the Missions reference omit --auto. Pass --auto high explicitly. Because that's High autonomy, run it only from a job a person triggers, on a branch, with the protected-path check and branch protection in place. A Mission is at least #features + 2 × #milestones runs, so budget before you start. See Lab 10.

Factory's own automations

You don't have to write every workflow by hand. Factory's Custom Automations start a Droid session from a schedule, a Slack message, a GitHub event or a webhook, and a GitHub automation runs in GitHub Actions from a workflow file that Factory adds through a setup pull request. Read the starter before you enable one in a regulated repository: some are built to act, such as the Review Comment Fixer, which pushes a commit. If your rule is that a human pushes, that starter conflicts with it. See Custom Automations.

What not to do in CI

Don't use --skip-permissions-unsafe. It skips every prompt. block rules still apply, but your ask rules stop meaning anything. It also can't be combined with --auto.
Don't put the API key in the prompt file, the repository or a log line.
Don't grant --auto high on a trigger anyone can fire, such as a comment on a public pull request.
Don't trust the exit code of droid rules check alone. It exits 0 even when the decision is block. Read --json.
Don't let one flaky run decide. Agents are stochastic. If a Droid job gates a merge, decide how many runs and what threshold before you turn it on.

A checklist before you merge the workflow

FACTORY_API_KEY is a repository or organization secret, not an environment variable in the YAML
The Droid version is pinned
Every droid exec has the lowest --auto that works, and pull-request triggers never use high
A git push ask rule is in .factory/settings.json
-o json output and the session_id are uploaded as artifacts
The protected-path check runs on every pull request
Factory-AI/droid-action is pinned, and branch protection still requires a human approval
Someone owns the monthly cost and knows where to see it

The five facts that shape every pattern​

Authentication and install​

Pattern 1 — read-only analysis​

Pattern 2 — structured output and exit codes​

Pattern 3 — bounded changes, human push​

Pattern 4 — restrict the tools​

Pattern 5 — the regression scoreboard​

Pattern 6 — parallel runs in worktrees​

Pattern 7 — the protected-path check​

Pattern 8 — Droid code review on pull requests​

Pattern 9 — Missions, headless​

Factory's own automations​

What not to do in CI​

A checklist before you merge the workflow​