Skip to content

feat: /insights command — usage analytics, cost estimation & activity patterns#552

Merged
teknium1 merged 4 commits intomainfrom
feat/insights
Mar 7, 2026
Merged

feat: /insights command — usage analytics, cost estimation & activity patterns#552
teknium1 merged 4 commits intomainfrom
feat/insights

Conversation

@teknium1
Copy link
Copy Markdown
Contributor

@teknium1 teknium1 commented Mar 6, 2026

Summary

Adds an /insights command inspired by Claude Code's /insights, adapted for Hermes Agent's multi-platform architecture. Analyzes session history from state.db to produce comprehensive usage analytics.

What it does

Generates a report covering:

  • 📋 Overview — sessions, messages, tool calls, tokens, estimated cost, active time, averages
  • 🤖 Models Used — per-model breakdown with sessions, tokens, and cost estimation
  • 📱 Platforms — CLI vs Telegram vs Discord etc. (unique to Hermes's multi-platform nature)
  • 🔧 Top Tools — most-used tools ranked with call counts and percentages
  • 📅 Activity Patterns — day-of-week bar chart, peak hours, active days, best streak
  • 🏆 Notable Sessions — longest, most messages, most tokens, most tool calls

Cost Estimation

Includes real pricing data for 25+ models (OpenAI, Anthropic, DeepSeek, Google, Meta) with fuzzy model name matching — strips provider prefixes, handles keyword heuristics (opus, sonnet, haiku, etc.), falls back to defaults for unknown models.

Three Entry Points

  1. /insights slash command in CLI — full terminal-formatted report with box-drawing and bar charts
  2. /insights slash command in gateway (Telegram/Discord/etc.) — compact markdown format
  3. hermes insights CLI subcommand — standalone usage

Configurable

  • --days N — time window (default: 30 days)
  • --source <platform> — filter by platform (cli, telegram, discord, etc.)
  • Gateway shorthand: /insights 7 sets days=7

Example Output

  ╔══════════════════════════════════════════════════════════╗
  ║                    📊 Hermes Insights                    ║
  ║                       Last 30 days                       ║
  ╚══════════════════════════════════════════════════════════╝

  📋 Overview
  ────────────────────────────────────────────────────────
  Sessions:          507           Messages:        44,517
  Tool calls:        40,841        User messages:   1,848
  ...

  🤖 Models Used
  ────────────────────────────────────────────────────────
  Model                          Sessions       Tokens     Cost
  claude-opus-4.6                     487            0 $  0.00
  ...

  📅 Activity Patterns
  ────────────────────────────────────────────────────────
  Mon  █████           59
  Tue  ███████         80
  Wed  ██████████      107
  Thu  ███████         77
  Fri  ███████████████ 155
  Sat  █               14
  Sun  █               15

Files Changed

File Purpose
agent/insights.py Core insights engine — data gathering, computation, formatting
cli.py /insights slash command handler in CLI
gateway/run.py /insights slash command handler in gateway (+ help text)
hermes_cli/commands.py Command definition for autocomplete
hermes_cli/main.py hermes insights CLI subcommand
tests/test_insights.py 56 tests — pricing, formatting, engine, edge cases

Tests

56 new tests covering:

  • Pricing lookup (exact match, prefix, heuristics, unknown, null)
  • Cost estimation calculations
  • Duration/bar chart formatting helpers
  • Empty database handling
  • Full populated database with multi-platform sessions
  • Source and days filtering
  • Terminal and gateway formatting
  • Edge cases (no tokens, no end time, no model, no tools, single platform)

Full test suite: 1982 passed, 5 skipped, 0 failures.

Future Ideas

  • LLM-powered qualitative analysis (session goal categorization, friction detection) — like Claude Code's facet extraction
  • HTML report export
  • Cost tracking over time (trend charts)
  • Per-user breakdown for multi-user gateway instances
teknium1 added 4 commits March 6, 2026 14:04
Inspired by Claude Code's /insights, adapted for Hermes Agent's multi-platform
architecture. Analyzes session history from state.db to produce comprehensive
usage insights.

Features:
- Overview stats: sessions, messages, tokens, estimated cost, active time
- Model breakdown: per-model sessions, tokens, and cost estimation
- Platform breakdown: CLI vs Telegram vs Discord etc. (unique to Hermes)
- Tool usage ranking: most-used tools with percentages
- Activity patterns: day-of-week chart, peak hours, streaks
- Notable sessions: longest, most messages, most tokens, most tool calls
- Cost estimation: real pricing data for 25+ models (OpenAI, Anthropic,
  DeepSeek, Google, Meta) with fuzzy model name matching
- Configurable time window: --days flag (default 30)
- Source filtering: --source flag to filter by platform

Three entry points:
- /insights slash command in CLI (supports --days and --source flags)
- /insights slash command in gateway (compact markdown format)
- hermes insights CLI subcommand (standalone)

Includes 56 tests covering pricing helpers, format helpers, empty DB,
populated DB with multi-platform data, filtering, formatting, and edge cases.
Comprehensive guide for using Hermes Agent with alternative LLM backends:
- Ollama (local models, zero config)
- vLLM (high-performance GPU inference)
- SGLang (RadixAttention, prefix caching)
- llama.cpp / llama-server (CPU & Metal inference)
- LiteLLM Proxy (multi-provider gateway)
- ClawRouter (cost-optimized routing with complexity scoring)
- 10+ other compatible providers table (Together, Groq, DeepSeek, etc.)
- Choosing the Right Setup decision table
- General custom endpoint setup instructions

All of these work via the existing OPENAI_BASE_URL + OPENAI_API_KEY
custom endpoint support — no code changes needed.
Custom OAI endpoints, self-hosted models, and local inference should NOT
show fabricated cost estimates. Changed default pricing from $3/$12 per
million tokens to $0/$0 for unrecognized models.

- Added _has_known_pricing() to distinguish commercial vs custom models
- Models with known pricing show $ amounts; unknown models show 'N/A'
- Overview shows asterisk + note when some models lack pricing data
- Gateway format adds '(excludes custom/self-hosted models)' note
- Added 7 new tests for custom model cost handling
…, serialization

Issues found and fixed during deep code path review:

1. CRITICAL: Prefix matching returned wrong prices for dated model names
   - 'gpt-4o-mini-2024-07-18' matched gpt-4o ($2.50) instead of gpt-4o-mini ($0.15)
   - Same for o3-mini→o3 (9x), gpt-4.1-mini→gpt-4.1 (5x), gpt-4.1-nano→gpt-4.1 (20x)
   - Fix: use longest-match-wins strategy instead of first-match
   - Removed dangerous key.startswith(bare) reverse matching

2. CRITICAL: Top Tools section was empty for CLI sessions
   - run_agent.py doesn't set tool_name on tool response messages (pre-existing)
   - Insights now also extracts tool names from tool_calls JSON on assistant
     messages, which IS populated for all sessions
   - Uses max() merge strategy to avoid double-counting between sources

3. SELECT * replaced with explicit column list
   - Skips system_prompt and model_config blobs (can be thousands of chars)
   - Reduces memory and I/O for large session counts

4. Sets in overview dict converted to sorted lists
   - models_with_pricing / models_without_pricing were Python sets
   - Sets aren't JSON-serializable — would crash json.dumps()

5. Negative duration guard
   - end > start check prevents negative durations from clock drift

6. Model breakdown sort fallback
   - When all tokens are 0, now sorts by session count instead of arbitrary order

7. Removed unused timedelta import

Added 6 new tests: dated model pricing (4), tool_calls JSON extraction,
JSON serialization safety. Total: 69 tests.
@teknium1 teknium1 merged commit 6d38047 into main Mar 7, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant