Caveman
Compression engineRecoverable compression

Recoverable compression

The engine's compressors drop bytes — that's the point. What keeps that honest is CCR: the original is stored first, under a content-addressed handle, and is always retrievable byte-for-byte.

Fig. — recoverable compression
Original bytesanswer-bearingCompressedS4 · inferredCCR handlecontent-addressedcompressstoreretrieve(handle) → the byte-exact original

CCR stands for the recoverable layer underneath every lossy result. It's why the engine can sit at safety class S4lossy · recoverablelossy — and still belong to an honesty-first product. The model reads a smaller payload; the full original never leaves disk.

The contract

Compression and recovery are two halves of one promise:

text
Compress(input)   → { output, ratio, recovery_handle }   // output is smaller; handle points at the original
Retrieve(handle) → input // byte-for-byte, exactly what went in

The store lives at ~/.caveman/ccr.db (SQLite). Handles are content-addressed — derived from the bytes themselves — so the same input always yields the same handle, and the handle is enough to get the original back.

CCR-or-pass-through

This is the rule that makes the whole thing trustworthy. A lossy (S4) result is emitted only if its original was successfully stored. If there's nowhere to store it, the engine doesn't compress — it passes the payload through unchanged.

Honesty rule

No store, no compression. The engine would rather hand back your original bytes than emit a smaller payload it can't reverse. "Lossy" never means "lost." A compression result that can't be recovered simply isn't produced.

So there are three honest outcomes, never a fourth:

  1. Compressed + recoverable — smaller output, plus a handle that restores the original exactly.
  2. Pass-through — the original bytes, unchanged, when the input can't be parsed, isn't smaller after compression, or couldn't be stored.
  3. RecoveredRetrieve(handle) returns the byte-exact original on demand.

Where it shows up

CCR isn't just an engine internal — recoverability propagates to everything built on the engine:

  • the MCP server returns a recovery_handle from caveman_compress, and caveman_retrieve brings the original back
  • Cavemem stores raw memories first and exposes a recovery handle for the detail dropped at recall time
  • Caveman Shrink keeps a CCR handle for every compressed tool catalog

For public local builds, CCR is a SQLite file under your Caveman home directory.

WASM builds use an in-memory store

The engine has two CCR backends — a SQLite store for native builds and an in-memory store for WASM and ephemeral sessions. The contract is identical; only the durability differs. The MCP server, for instance, opens a session-scoped in-memory store.