Harvey's margins crashed until it switched to an open model

CONFIRMED (reporting event): On Monday September 21, 2026, Bloomberg (Rebecca Torrence and Natasha Mascarenhas) published "Startups Like Harvey Embrace Open Models to Cut Reliance on Anthropic, OpenAI." The report describes AI-native startups migrating from frontier-lab APIs to open-weight models to survive the economics:

• COMPANY CLAIM / EARLY RESEARCH (per Bloomberg, citing a person familiar with the matter): Harvey's gross margins collapsed from roughly +50% at the start of 2026 to −50% by June 2026, after a March 2026 update to its AI agents spiked customer usage. Bloomberg reports token usage grew roughly 20x this year (attributed to the company, per TNW).
• COMPANY CLAIM / EARLY RESEARCH: Harvey post-trained its own model — Harvey Tenet, built on Moonshot AI's open-weight Kimi K3 base (announced as a research preview on August 20, 2026) — and gross margins turned positive again thereafter, according to people familiar with the effort.
• CONFIRMED (corroborated by multiple outlets + company materials): Harvey is valued around $15.5–15.6B (company-announced $15.5B on its $550M raise; Bloomberg cites $15.6B); Kimi K3 is a 2.8T-parameter open-weight MoE model released by Moonshot in mid-July 2026 with weights published July 27, 2026.
• CONFIRMED (Bloomberg + TNW): Abridge (clinical AI; building on Nvidia's Nemotron open weights), Decagon (customer-support AI; ~80% of queries already routed through its own models per TNW/Digital Today), and Ramp (fintech; raised $750M in June, weighing in-house training for the first time) are pursuing the same path; Sequoia Capital and General Catalyst are funding the shift; Rogo and Canva are also moving toward open/smaller models (per 36Kr English / Digital Today).

Bloomberg.com itself blocks direct fetching (403), so the article text was verified via search-engine excerpts, Google News RSS syndication of the same content, and an independent TNW article that cites the same figures with the same source attribution.

The premise that "renting frontier APIs is the standard cost of doing business for an AI-native startup" changed. Previously, being an app-layer company on top of OpenAI/Anthropic APIs was a badge; the Harvey numbers make it a gross-margin liability. Key change:

• Before: AI-native startups resold frontier models (Harvey originally built on OpenAI GPT-4-class models) under seat/subscription pricing, with model providers charging usage fees on top of base fees; gross margin ~+50% (Harvey, start of 2026) — standard SaaS-like economics.
• Change (March→June 2026): An agent update multiplied usage ~20x; because the underlying cost is variable per-token usage pricing from OpenAI/Anthropic, COGS swamped revenue — margin swung to −50%.
• Response (June→August 2026): Harvey built its own post-trained model (Tenet) on Moonshot's open-weight Kimi K3, with Fireworks research; launched research preview August 20; margins positive again (people familiar, per Bloomberg/TNW).
• After (Sep 2026): Open-weight post-training is repositioned from a cost experiment to the survival play for agentic startups; Abridge (Nemotron), Decagon (in-house), Ramp/Rogo (exploring) follow; investors (Sequoia, General Catalyst) explicitly funding the shift; Anthropic is publicly tracking the trend (investor-forum slide noting Harvey still needs Opus for hardest tasks, per TNW).

• BEFORE (start of 2026): Harvey, valued ~$15.5B, $350M+ annualized revenue run rate (per TNW Aug 2026), routes legal work through models from OpenAI, Anthropic (and Google). Gross margin ≈ +50%. Model calls on a customer's behalf = invoices from rival labs; OpenAI is an investor in Harvey (alongside Sequoia and a16z).
• CHANGE (March 2026): Agent update → usage spike → token usage grows ~20x on usage-based pricing. By June, gross margin ≈ −50% — every dollar of legal-AI revenue now loses money at the model layer.
• CHANGE (June–August 2026): Harvey pivots: research agenda ("building frontier legal intelligence using open-weight models"; "systems to allow law firms to build their own specialized models and own their intelligence"). Harvey Tenet = Kimi K3 base post-trained with Fireworks (GSPO RL, ~150 B300 GPUs, ~2 months); released as research preview Aug 20, 2026.
• AFTER (Sep 2026): Margins positive again (per people familiar — exact figure undisclosed). Harvey still uses frontier models for the hardest tasks (hybrid routing), but the flagship is now the in-house model. Peers copy the playbook; Bloomberg documents the wave.

The mechanics, per Harvey's own technical blog (Aug 20, 2026) and Moonshot's materials:

• Base: Kimi K3 — Moonshot's 2.8T-parameter open-weight Mixture-of-Experts model (16/896 experts active), Kimi Delta Attention + Attention Residuals, ~2.5x scaling efficiency vs Kimi K2, 1M-token context, native vision. Weights published July 27, 2026 under the "Kimi K3 License" (derivative models permitted; separate Moonshot agreement required for model-as-a-service operators above $20M revenue in any 12 months; prominent "Kimi K3" attribution required above 100M MAU).
• Post-training: Harvey + Fireworks ran asynchronous reinforcement learning (GSPO — group sequence policy optimization) with rank-64 LoRA over the full 2.8T network (~500k expert tensors). ~1,750 agentic legal task environments (closed-universe client matters with expert rubrics, ~50 criteria/task), >10,000 rollouts, ~150 NVIDIA B300 GPUs over ~2 months. LLM-as-a-judge grading (they converged on Kimi 2.6 as judge); no customer data used in training.
• Cost engineering: Harvey reports reward shaping that prefers token-efficient trajectories — because cost = price-per-token × tokens-used, they co-optimize both. Post-trained model roughly doubles held-out LAB task completions over base K3 (+9pp all-pass on LAB; +20% on LAB Contracts) and, on Review Table work, improves answer quality +3.6 and citation quality +12.1 at "roughly one-tenth the cost per cell." Their "intelligence-per-token" metric: 190.8 vs 129.3 for best frontier config (FACT per Harvey's blog — company-reported).
• INTERPRETATION — private-company parameters are unobservable, but the mechanism is not: the exact price/revenue mix that produced Harvey's −50% is undisclosed, yet any realistic combination (see the sensitivity analysis in labs/S08.md) reproduces the inversion above ~3x token growth; the numbers Bloomberg reported are plausible interior points of the model space, which is why the story reads as a mechanism lesson rather than an anomaly.
• Private-company economics: Harvey declined to comment on specific financials (Bloomberg/TNW); margin figures derive from a person familiar with the matter + company statements. Moonshot's own model card even lists a "Harvey Lab-AA 94.6" criterion pass rate for Kimi K3 — an independent marker of Harvey's benchmark appearing in the base model's eval table.

This is the first on-the-record account of the margin inversion hitting the AI-native layer:

1. It converts open-weight post-training — previously a cost experiment — into the standard survival play for agentic startups with high token consumption.
2. It reframes frontier-API dependence as a strategic liability, not a badge of quality, for high-volume workloads.
3. It names the mechanism: OpenAI and Anthropic charge usage on top of subscriptions; 20x usage growth turns a 50% gross margin into −50% when the COGS is a per-token metered bill.
4. It has a geopolitical flavor: the fix for a US startup was Chinese open weights (Moonshot) underneath privileged legal client work — immediately raising privilege, national-security, AI Act and licensing questions (TNW).
5. It arrives in a pricing-war window (Grok 4.7 at $2/$6 per M tokens the same day; Xiaomi's MiMo-V2.6 open weights 2 days earlier at MIT; OpenAI cutting GPT-6 Sol/Luna prices ~50% the next day) — the front-end evidence that frontier-lab pricing power is being contested from below.

• Post-training as a moat: A well-capitalized vertical AI startup can now take a 2.8T open-weight frontier-class base, add domain RL, and get legal-specific quality that beats the base while cutting cost per task by an order of magnitude (company-reported figures).
• "Owning your intelligence": Harvey's public positioning is that law firms (and other professional-services firms) can eventually own specialized models themselves — turning Harvey from a model intermediary into an intelligence-infrastructure provider.
• Inference-economics arbitrage: Hosted open-weight pricing (Kimi K3 at ~$2.85–3.00 / $14.25–15.00 per M tokens via DeepInfra/Moonshot) plus self-host options (MXFP4 weights ≈1.4TB for the full 2.8T model) gives startups a third option between "rent closed frontier" and "do nothing."
• Investor category formation: Sequoia and General Catalyst funding the open-weight pivot signals a new diligence criterion for AI-native startups: model-layer dependency and margin exposure.

1. What exactly changed in the March 2026 agent update that produced the usage spike?
2. What is Harvey's actual gross margin now — and does Tenet serve the majority of tokens, with frontier reserved for a minority of tasks?
3. Does Harvey's product constitute "model as a service" under the Kimi K3 License, and has it entered a separate agreement with Moonshot (required above $20M/12mo)?
4. Will frontier labs respond with pricing/contract changes for high-volume app-layer customers, or with cut-offs (Cursor precedent)?
5. How do Abridge/Decagon/Ramp quantitative stories land — and does the trend reach mid-market AI-native startups without $15B war chests?
6. Will regulators (EU AI Act; US policy) create guidance on Chinese open weights under professional-services privilege?
7. Does Harvey publish its promised Tenet technical report, and can the RL recipe be replicated at smaller scale?

• Short term (Q4 2026): more startups disclose margin-driven pivots (Ramp, Rogo, Canva per 36Kr); Harvey scales Tenet to production, may publish its technical report; frontier labs respond with price/contract moves; watch for the Moonshot–Harvey license agreement surfacing.
• Medium term (2027): post-training becomes a standard competency in vertical AI; enterprises adopt model-supply-chain audits; the EU AI Act "significant modification" guidance gets tested on post-trained open weights; expect at least one regulator/flagging on Chinese open weights under privileged data.
• Prediction (medium confidence): within two quarters, at least one more major vertical AI company will announce an open-weight flagship, and at least one frontier lab will introduce a "developer/app-layer" commercial program to slow the migration — mirroring Cursor-style lock-ins turned on their head.
• Prediction (medium confidence): the "own your model layer" pitch becomes the standard enterprise-AI procurement demand in legal and health by mid-2027.

The quiet assumption that AI-native application companies cannot exit their frontier-lab supply chains died on September 21, 2026. Harvey's swing from +50% to −50% gross margin on a 20x usage curve — and back to positive on a post-trained open-weight model — is the first fully documented instance of the metered-token business model inverting on its own tenants. The strategic lesson is blunt: when your COGS is a competitor's per-token invoice, your volume is their profit and your margin. Open-weight parity (Kimi K3, then MiMo-V2.6 days earlier) turned "we rent GPT-4" from a feature into a liability overnight, and the companies that own their model layer now hold both the cost curve and the geopolitical questions that come with it.