Recommended models and effort settings for saving tokens

[kiril-buga]

Which is the cheapest model and effort level that still works well for this course? I'd like to learn without burning tokens on a model that's bigger than the task needs.

I see two use cases:

1. Using an AI agent as tutor (Claude Code, Codex, Cursor, etc. with the /learn, /start-learning and certification skills). Which model tier and effort or reasoning level do you use for courses?
2. Running lesson code that calls a provider API. Are there lessons where a small model is enough, and any where it isn't?

If you can, please share:

• Provider and model (Claude, OpenAI, Gemini, local/open-weight, etc.)
• Effort or reasoning setting

[TomasD1az]

I've been going through it with Claude Code, so here's what's worked for me:

1. Agent as tutor (/learn, /start-learning)

Claude Sonnet, medium effort. That's plenty for explaining lessons, quizzes and recall warm-ups. Most of the work is reading the lesson and asking you questions, not deep reasoning.
For the math-heavy phases (1, 3, 7, 10: backprop, attention, building the LLM from scratch) I sometimes raise effort to high when I get stuck on a derivation. I switch back after.
Haiku works for quick lookups with course-guide, but as a tutor it felt too shallow. It tends to give you the answer instead of guiding you.
For certification prep I'd stay on Sonnet. The practice questions are better and it catches wrong reasoning, not just wrong answers.

2. Lesson code that calls an API

Phases 0–10 barely need an API. It's mostly NumPy/PyTorch from scratch, so no tokens burned there.
From Phase 11 on (prompting, RAG, tool use), a small model is fine for most lessons: Haiku, GPT mini, Gemini Flash, or even a local model through Ollama (Qwen/Llama 8B). You're learning the pattern, not chasing the best output.
Where I've found small models struggle: multi-step agents, tool calling with several tools, structured output with nested schemas, and anything eval-related (LLM-as-judge). There I'd use a mid-tier model. With small ones you end up debugging the model instead of your code.

Rule of thumb: start cheap, and only move up when the failure is clearly the model and not your implementation.

[imMamdouhaboammar]

@kiril-buga @TomasD1az I'd be curious to see this measured as cost per successfully completed lesson, including all the rescue turns. A small model can look wonderfully cheap until one incorrect derivation sends you into five rounds of debugging the tutor. Congratulations, you've accidentally enrolled in an extra course.

The repo's learn skill already provides a decent starting point for testing this: comprehension questions, runnable code, post-lesson quizzes, and progress tracking in LEARNING.md

I'd compare a few model/effort combinations across equivalent lessons and record total token usage, retries, turns to completion, and code/test success. Then give the learner a fresh variation of the problem to solve unaided. Repeating the same quiz would contaminate the comparison as the learner becomes familiar with the answers.

I'd also keep subscription-based tutors and API-backed lessons separate. Subscription usage limits don't translate cleanly into API dollars, whereas API labs can be compared using fixed inputs and cost per passing run, including failed attempts.

@kiril-buga, would a small community results template be useful here? I'd be interested in seeing where the cheapest model actually stays cheap once retries enter the equation

[kiril-buga]

Yes, I think a small community results template would be useful.

It would make the answers much easier to compare than just listing the model and effort level. I like the idea of separating tutor usage from API-backed lessons as well.
I’d keep the first version simple so people can realistically fill it in. If there’s interest, I’d be happy to put together an initial Markdown template.

[rohitg00]

The first answer matches how the course is built, and measuring cost per completed lesson is the right next step. Some specifics from the repo:

Lesson code. Only three lessons make a live provider call:

• phases/00-setup-and-tooling/04-apis-and-keys (first_api_call.py and .ts, Anthropic; the TypeScript version also has a mock mode with no network)
• phases/11-llm-engineering/02-few-shot-cot (advanced_prompting.py, OpenAI client)
• phases/11-llm-engineering/16-langgraph-state-machines (main.py, Anthropic through LangChain)

Every other lesson runs offline: NumPy or PyTorch from scratch, or a simulated client, so you learn the pattern without a key. CI runs the lesson tests with no API keys configured.

All three read LLM_MODEL, so a small model is one environment variable away:

LLM_MODEL=claude-haiku-4-5 python3 phases/00-setup-and-tooling/04-apis-and-keys/code/first_api_call.py
LLM_MODEL=gpt-4o-mini python3 phases/11-llm-engineering/02-few-shot-cot/code/advanced_prompting.py

The OpenAI client also reads OPENAI_BASE_URL, so the few-shot lesson runs against a local model through Ollama's OpenAI-compatible endpoint:

OPENAI_BASE_URL=http://localhost:11434/v1 OPENAI_API_KEY=ollama LLM_MODEL=qwen3:8b python3 phases/11-llm-engineering/02-few-shot-cot/code/advanced_prompting.py

Tutor. The skills (/learn, /start-learning, /check-understanding, and the certification skills) do not pin a model or an effort level. They run on whatever your agent is set to. Most of their work is reading a lesson, asking questions, and checking answers, so a mid-tier model at its default effort covers it. Raise effort for derivation-heavy lessons (backprop, attention, the LLM-from-scratch phase) and for certification practice, where the tutor has to catch wrong reasoning and not only wrong answers.

Template. A results template would help. Keep the first version short: tool and model, effort, lesson path, total tokens or cost, turns to completion, retries, whether the code and quiz passed, and whether you then solved a fresh variation unaided. Post results as replies here so they stay comparable.