Anthropic recently shipped Legal Skills for Claude Desktop's Cowork, which triggered a market selloff. We ran its contract review feature against our own custom, legally engineered pipeline. The SaaS contract, playbook, and model (Opus 4.6) were all the same. The results were not:
- Findings: 222 vs. 18 (12.3x delta)
- Track changes applied to DOCX: 52 vs. 0
- Quantitative risk score: 93/100 vs. "CRITICAL" (a label, not a number)
- Market benchmarks with percentiles: 12 vs. 0
- Comparable contracts analyzed: 4 vs. 0
- Deliverables produced: 7 vs. 1
The pipeline also produced a send-ready negotiation email, a strategy report with 10 priorities each with target/fallback/walk-away positions, 8 anticipated pushback scenarios with prepared counters, 15 talking points citing market data, and analytics with risk scoring, playbook alignment, and market percentile benchmarks for every major term.
That said, Claude Desktop's output, while narrow, did correctly identify the 8 most dangerous provisions from the Provider's perspective: liability cap carve-outs, MFN pricing, escrow, uncapped indemnification. For a 5-minute investment, that's genuinely impressive. But even here, the depth gap was critical and obvious.
Only the pipeline connected Section 13.3(f), which carves out insurance-covered losses from the cap, to the insurance requirements in Section 16. The practical effect: Provider's own insurance policy limits become the real liability cap, not the negotiated number in Section 13. You negotiate a $1M cap. You carry $10M in cyber insurance. Your actual exposure is $10M. A single pass reads Section 13 and Section 16 independently. Sixteen specialists cross-referencing each other catch the interaction.
The same pattern repeats throughout. A 5-day outage simultaneously triggers SLA credits, DR termination, escrow source code release, and unlimited liability — a cascade across four sections. Three separate payment withholding mechanisms plus an express prohibition on service suspension effectively eliminates revenue protection. These mechanisms are not an accident. They're a coordinated conspiracy woven into the contract. And only AI specialists, working in parallel and synthesized together, are built to find it.
This is our fifth controlled experiment. The pattern holds across drafting, triage, legal research, redlining, and now contract review.
The model is not the product. The architecture around the model is the product. If you want to see what that looks like for your legal work: Taylor Legal Engineering, LLC.
Full interactive report with methodology, pipeline architecture, and all comparison data: https://lnkd.in/eyiaXcdd
#LegalTech #LegalEngineering #AI #ClaudeDesktop #ContractReview #LegalAI #LegalOps #MultiAgentAI #SaaS
![]()
![]()
25
4 Comments
