CacheIt

Tiered caching for programs and data: RAM first, then disk, crash-safe, with an API, a CLI and MCP.

CacheIt keeps bytes where they are fastest to get back. Hot values stay in RAM; everything else is on local disk (NVMe or SSD). Values survive restarts and never come back damaged. Programs use it over HTTP, the cacheit command line, or MCP (for AI agents). It is also a module of AgenticBotPlatform (ABP), which uses it to speed up its own data flow.

cargo build --release -p cacheit-daemon -p cacheit-cli     # needs Rust 1.85+
target/release/cacheit start                               # cacheitd in the background
target/release/cacheit put demo greeting "hello" --ttl 3600
target/release/cacheit get demo greeting                   # hello
target/release/cacheit put models tokenizer @./tokenizer.json --pin
target/release/cacheit stats
target/release/cacheit ops                                 # every operation the hub offers
target/release/cacheit stop                                # flushes, then stops

cacheitd options:

• --home DIR: where the store and the control file live. The default is %LOCALAPPDATA%\cacheit, ~/.local/share/cacheit, or ~/Library/Application Support/cacheit.
• --l1-mb / --l2-mb: the RAM and disk budgets (512 MB and 8 GB by default; --l2-mb 0 turns off the disk tier).
• --write-back: new values stay in RAM until they are evicted or flushed.
• --compress-min BYTES: values at least this big are zstd-compressed on disk (4096 by default; 0 turns it off).
• --max-value-mb: the biggest value accepted.
• --port (0 by default: any free port).

• L1 (RAM) is a segmented LRU with a byte budget.
  • A new value enters probation. A second hit moves it to protected (at most 80% of the budget).
  • Eviction takes probation's oldest entry first, so a one-time scan of many keys can't push out the ones used all the time.
  • Every operation is O(log n).
• L2 (disk) keeps one file per value, with an LRU byte budget.
  • Each file has a header: its namespace, key, expiry, flags, and a CRC32 of the stored bytes.
  • Writes go to a temporary file, are flushed to disk, then renamed over the old one. A crash leaves either the old value or the new one, never a torn file.
  • There is no separate index to corrupt: at start the store scans the headers. Leftover temporary files are removed; files with a bad header or checksum are deleted and counted.
• Moving between tiers. A disk hit is copied back into RAM (promotion). A value evicted from RAM that isn't on disk yet is written there (demotion).
• Write-through (the default) puts every value on both tiers. Write-back keeps new values in RAM and writes them on eviction, on flush, every 30 seconds, and at shutdown.
• Per value:
  • Namespaces: 1–64 characters of letters, digits, ., _ and -.
  • Keys: up to 1 KB.
  • TTL: expired values are dropped when read and swept every 30 seconds.
  • Pins: a pinned value is never evicted, and the pin is kept on disk.
  • Content addressing: put_cas stores a value under its SHA-256, so the same bytes are stored once.

When cacheitd starts, it writes <home>/control.json: {url, token, pid, version, api}. Every route except health needs Authorization: Bearer <token>. Errors come back as {"error": {"code", "message"}}.

The operations:

• cache.get, cache.put, cache.put_cas, cache.delete, cache.stat, cache.pin;
• cache.keys, cache.namespaces, cache.clear;
• cache.purge_expired, cache.flush;
• cache.stats, service.status.

cacheit mcp serves the hub's operations as MCP tools over stdio (cache_get, cache_put, ...). Tools that change something are marked, and destructive ones are flagged. An MCP client's config:

{"mcpServers": {"cacheit": {"command": "cacheit", "args": ["mcp"]}}}

ABP reads abp-module.toml. From there it can:

• clone CacheIt and keep it updated from this repo;
• build it, start and stop its hub, and run its pipeline;
• give ABP's agents the operations (module_call) and the MCP server.

ABP's own hot paths use it through a small client: web fetches, embeddings, model lists, the cluster's job data, and build caches. The plan is in ABP's docs/modules/ROADMAP.md (§5.6, phases CI-A to CI-G).

python ci/pipeline.py                  # preflight, static checks, tests, release build, a live smoke test
python ci/pipeline.py --install-hook   # run it on every git push (skip once: CACHEIT_SKIP_PIPELINE=1)

• Static checks. rustfmt runs on the crates kept format-clean (listed in ci/pipeline.py). Clippy's correctness and suspicious lints are errors; the older crates' style warnings are counted, not fatal.
• Smoke test. It starts a real cacheitd in a throwaway folder and checks that it:
  • answers health, and refuses a caller without the token;
  • lists its operations;
  • stores a value and reads it back;
  • round-trips 1 MB of bytes through the fast path;
  • serves MCP;
  • stops cleanly.

1. The TUI and desktop app move to the hub. The replacement algorithms become selectable for the RAM tier.
2. ABP's hot paths use CacheIt, each measured before and after.
3. Distributed: machines linked in an ABP cluster share one logical cache (consistent hashing, two replicas, reads from peers).
4. Volume caching.
   • Linux: managing dm-cache, bcache and dm-writecache.
   • Windows: a volume upper-filter driver. That is a signed kernel driver, so it will be its own project, with requirements and risks written down first.

Either of Apache-2.0 or MIT, at your option.