Add /context-size command to set context window for local providers

[Gustav-Simonsson]

Summary

Add /context-size command to make it easier to set max context size for local models.

[github-actions]

All contributors have signed the CLA.
_{Posted by the CLA Assistant Lite bot.}

[Gustav-Simonsson]

I have read the CLA Document and I hereby sign the CLA

[TheGreatAxios]

Hey @Gustav-Simonsson

Question here for you -- how often do you change the context window for local models and is this something you generally define globally for local models or per model? Wondering if it makes more sense to expand a bit on what we define within the provider/model section and allow this to be overridden on models with a "local" tag?

[TheGreatAxios]

Review · Request changes

Adds a /context-size <tokens> command that, for keyless/local providers, writes providers.<name>.contextWindow to global settings and installs an in-process per-provider override consulted by contextWindowFor/hasContextWindowFor.

Diff hygiene

The PR shows +34382/−15173 across 666 files, but the real change is one commit (bb4a9b60, 5 files, +93/−7). The branch carries rewritten SHAs of upstream history (HEAD^ tree == origin/main at 0.3.4), so the PR diff is 0.3.4 → 0.3.7 noise. git merge-tree against origin/main conflicts in CHANGELOG.md, package.json, src/agent/tools.ts — all artifacts of the duplicated history; with --merge-base=HEAD^ the feature commit applies cleanly. Please git rebase --onto origin/main HEAD^ (or cherry-pick bb4a9b60 onto a fresh branch) so the PR shows the 5-file diff.

Findings

• src/provider/context-window.ts:64-68, 84-97 — the override is keyed by providerFromModel, which requires a provider:model string. The status bar passes bare config.model (src/tui/runner.ts:2081, 2092) and compaction passes bare event.source?.model (src/agent/compaction.ts:100), so neither ever sees the override; only the /model picker (src/tui/model-catalog.ts:241, which passes option.id) does. After /context-size 128000 on Ollama, the meter and compaction threshold do not change. An Ollama tag like qwen3:30b also makes providerFromModel return qwen3.
• src/tui/runner.ts:2138-2143 — providers.X.contextWindow already has a meaning: it feeds maxTokensFor (src/config/inference-sources.ts:63-66) → defaults.maxTokens → max_tokens/max_output_tokens on the wire. After /context-size 131072 and a restart, every request carries max_tokens: 131072; llama.cpp/vLLM validating prompt + max_tokens <= n_ctx reject or clamp. This needs either a new settings field or reconciling what contextWindow means first.
• src/tui/runner.ts:2131-2146 — in-memory config.settings is not updated; a same-session /model switch (buildSessionSources() at :2377-2378) reads the old value, so behavior differs before and after restart.
• src/tui/runner.ts:2138-2144 — if (prov) silently skips the write when the provider is absent from the global file, but the command still reports "Context window set". loadGlobalSettingsWriteBase returns {providers:{}} for a missing file, so a provider defined elsewhere gets a false success.
• src/tui/commands/built-in.ts:235-238 — Number(args) accepts 12.5, 1e5, 0x20000 while the usage text says <positive-integer>; 12.5 round-trips into settings. Add Number.isInteger and an upper bound.
• No tests. src/provider/context-window.test.ts and src/tui/commands/built-in.test.ts cover the neighboring code.

Notes

• Registration follows /yolo (built-in.ts:193, registry.ts:28-29), the keyless gate is correct, and passing option.id to contextWindowFor in model-catalog.ts:238-242 is behavior-preserving.
• Checks: bun run typecheck, bun run lint (prettier clean; 4 pre-existing warnings in runner.ts outside the diff), bun test src/provider/context-window.test.ts src/tui/commands/built-in.test.ts src/tui/commands/registry.test.ts src/tui/model-catalog.test.ts (59 pass).