TRACE: Tracking Repository Alignment and Context Evaluation

TRACE is a lightweight metadata tracking utility and evaluation framework designed to map the co-evolution of a software codebase and a Large Language Model (LLM) agent's context.

During iterative software development guided by LLM agents, two streams of history evolve concurrently:

1. Code Evolution: Represented by n codebase states (Git commits, file changes, or deltas).
2. Agent Context Evolution: Represented by m context states (system prompts, user instructions, retrieved documents, and conversation histories).

These two streams do not map 1:1; multiple context modifications may lead to a single commit, or a single prompt context may result in several local commits. TRACE addresses this duality by registering both sequences and mapping their alignment over time in a space-efficient manner.

To avoid duplicating the entire context window at every turn (which leads to quadratic storage growth), TRACE uses a delta-based, event-sourced model:

• Git References: Codebase history remains in Git. TRACE only stores references to Git commit SHAs.
• Context Deltas: TRACE stores the base context once, and then records individual context transitions (user messages, system prompt updates, or reference updates) as parent-linked deltas.
• SQLite Backend: All metadata, deltas, and alignments are persisted locally in a SQLite database (trace_history.db).

TRACE requires Python 3 and has zero third-party library dependencies.

To verify the installation, run the test suite:

python3 -m unittest tests/test_tracker.py

The tracking utility is executed via core/trace_tracker.py.

Initialize the SQLite database schema:

python3 core/trace_tracker.py init-db

Register the initial system configuration (system prompt and model parameters):

python3 core/trace_tracker.py register-base --prompt "You are a software agent." --model "gemini-1.5-pro"

Register subsequent events (e.g., user message, assistant response, or prompt modification) by specifying the parent hash:

python3 core/trace_tracker.py register-delta \
  --parent <parent_hash> \
  --type user_message \
  --payload "Implement the db connection."

You can also load the payload from a file using --payload-file:

python3 core/trace_tracker.py register-delta \
  --parent <parent_hash> \
  --type user_message \
  --payload-file path/to/message.txt

Map a Git commit SHA to a context delta hash:

python3 core/trace_tracker.py align --commit <commit_sha> --delta <delta_hash>

Display tracking database stats and recent mappings:

python3 core/trace_tracker.py status

Reconstruct the full context history up to a specific delta hash:

python3 core/trace_tracker.py reconstruct --delta <delta_hash>

Display the code diff alongside the reconstructed context diff between two aligned checkpoints:

python3 core/trace_tracker.py diff --commit-a <commit_sha_1> --commit-b <commit_sha_2>