Harvey's Margin Went to -50%. Cheaper Tokens Didn't Fix It — Better Data Did

说明与用途

由于令牌成本激增导致利润率下降，通过切换到开源模型并进行领域特定后训练，将每个任务的成本降低了10倍。

问题

Agentic AI usage caused token consumption to spike 20x, turning a +50% gross margin into -50% despite revenue growth.

结果

Switching to a cheaper open-weight model alone was insufficient; domain-specific post-training and rigorous evaluation reduced cost per completed task by 10x and restored profitability.

工作流

• 1. Identify token cost spike due to agentic usage.
• 2. Switch to cheaper open-weight model (Moonshot AI Kimi K3 post-trained with Fireworks AI).
• 3. Collect domain-specific post-training data (SFT, RLHF, preference ranking, correction).
• 4. Build parity benchmark against production task distribution.
• 5. Evaluate new model for hallucinations, error modes, and performance parity.
• 6. Deploy new model with monitoring for drift and ongoing QA.
• 7. Iterate post-training and evaluation to maintain cost and quality.

证据

Harvey reported roughly a 10x reduction in cost per cell on its Review Tables product and a 90% reduction in cost per query on Firm Knowledge.

来源

The model swap is the easy 10% of the project. The post-training data and the evaluation harness that proves the new model is actually as good is the other 90%.

来源