folio materialize

folio materialize <SHEET> [TARGET] --actor <actor>
  [--ids <id1,id2,...>]   # repeatable
  [--force]

TARGET is a positional argument naming a single derivation. Omit it to materialize every derivation in the sheet.

Walks every derivation under derivations/ (in topological order) and, for every target record, executes the kind unless the cache says otherwise. See Materialize lifecycle for the full theory.

Output: the §10.6 envelope

{
  "materialized": 12,
  "skipped": 7,
  "failures": [
    {"record_id":"cust_006","field":"industry_tag","error":"...","error_type":"FolioError"}
  ],
  "total_cost": 0.0034
}

Field	Meaning
`materialized`	Cell writes performed in this run.
`skipped`	Cell writes avoided (cache hit, `respect_human_override`, etc.).
`failures`	Per-record × field errors. The other records keep going.
`total_cost`	Sum of `cost_usd` from `ai` calls. `0.0` if no `ai` derivations or all unknown models.

Targeting

# Only one derivation (positional target)
folio materialize ./customers country_code --actor agent:demo

# Multiple derivations? Re-run materialize for each. There is no
# multi-target flag — the topological walk is unambiguous one at a time.
folio materialize ./customers country_code --actor agent:demo
folio materialize ./customers industry_tag --actor agent:demo

# Only a few records (comma-separated or repeat --ids)
folio materialize ./customers --actor agent:demo \
  --ids cust_001,cust_002,cust_003

# Combination: one derivation, one record
folio materialize ./customers country_code --actor agent:demo --ids cust_001

Forcing a re-run

folio materialize ./customers --actor agent:demo --force

--force ignores both:

The cache (every record re-executes its kind).
respect_human_override (the derivation overwrites the human edit and appends a new derivation provenance line on top).

The previous human entries stay in provenance.jsonl (it’s append-only), so you can recover them via folio provenance --history.

Failures

Failures are reported, not raised. A 5,000-row materialize that has 3 bad rows still completes; you fix them and re-run.

{
  "record_id": "cust_006",
  "field": "industry_tag",
  "error": "scripts/lookup.py exited with 2: KeyError 'cust_006'",
  "error_type": "FolioError"
}

The fields:

record_id, field — which cell.
error — the human-readable message.
error_type — the exception type name (helps filter in tools).

A non-zero exit code is not what you get for partial failures. You get exit 0 and a non-empty failures[]. To turn that into a CI gate:

out=$(folio materialize ./customers --actor agent:demo)
echo "$out"
[[ "$(echo "$out" | jq '.failures | length')" == "0" ]] || exit 1

What is logged

For every materialized cell, one line in provenance.jsonl:

{"record_id":"cust_001","field":"country_code","source":"python",
 "actor":"agent:demo","at":"2026-05-10T10:16:35Z",
 "input_hash":"sha256:ce82..."}

ai lines also include model and cost_usd.

Atomicity

Every record’s writes are atomic:

Acquire .lock (single writer, 30 s timeout).
Read records.jsonl.
For each derivation × record (cache miss + executed):
- Compute input_hash.
- Execute the kind. (May fail; fall through to the failures list.)
- Update the in-memory record.
- Write the cache entry.
Atomically rename the new records.jsonl over the old one.
Append all the provenance lines for successful writes.
Release the lock.

If step 4 fails, neither records nor provenance update — but the cache entries already written stay. The next materialize cache-hits and re-attempts the records/provenance step.

Common shapes

# Routine "run everything"
folio materialize ./customers --actor agent:enrichment

# Quick development loop on one row
folio materialize ./customers --actor agent:dev \
  --ids cust_test --force

# Incremental after a derivation file edit
folio materialize ./customers --actor agent:dev    # cache misses re-run automatically