Skip to content

speed up MISRA mutation tests#2366

Open
adeebshihadeh wants to merge 2 commits intomasterfrom
cppcheck-experiments
Open

speed up MISRA mutation tests#2366
adeebshihadeh wants to merge 2 commits intomasterfrom
cppcheck-experiments

Conversation

@adeebshihadeh
Copy link
Contributor

@adeebshihadeh adeebshihadeh commented Mar 1, 2026

Summary

  • Skip scons build in mutation tests — cppcheck analyzes source files directly, doesn't need firmware binaries
  • Add MISRA_ONLY mode to test_misra.sh — uses --enable=style instead of --enable=all (style severity is needed for MISRA addon output, the extra warning/performance/portability checkers are not)
  • Filter copytree to skip .venv (211MB), caches, and other unnecessary dirs

Result: 73s → 60s for 3 mutation tests (~18% faster)

Profiling notes

Profiled against cppcheck 2.16.0 source to understand flag costs:

Component Time Notes
MISRA addon (parse 54MB XML dump) ~11s Dominant cost, unavoidable without patching misra.py
cppcheck tokenize + dump generation ~6s Required input for MISRA addon
--enable=all extra checkers ~1-2s Small savings from dropping
--check-level=exhaustive vs normal ~1-2s NOT the bottleneck (only affects ValueFlow depth)
scons build ~2-3s Unnecessary for static analysis
copytree(.venv) ~1.4s/test 211MB copied for no reason

Test plan

  • mutation tests still pass locally (clean=pass, mutations=detected)
  • CI green

🤖 Generated with Claude Code
adeebshihadeh and others added 2 commits February 28, 2026 16:32
@adeebshihadeh @claude
speed up MISRA mutation tests
ea98e8a 
- skip scons build (cppcheck analyzes source directly)
- add MISRA_ONLY mode: use --enable=style instead of --enable=all
- skip copying .venv and caches in copytree (211MB -> 5MB)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@adeebshihadeh @claude
add fail-fast MISRA addon for mutation tests
175f1a5 
The new misra_failfast.py wrapper patches misra.py's reportError to
sys.exit(1) on the first real violation, handling cppcheck's inline
and macro suppressions to avoid false positives on clean code.
Also removes sampling (all 12 tests run in ~38s with xdist) and
adds board/crypto and board/certs to ignored mutation paths since
they're only included from bootstub.c.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant

@adeebshihadeh