Harvey's Margin Went to -50%. Cheaper Tokens Didn't Fix It — Better Data Did

Описание и применение

Снижение маржи из-за роста токенов, решение через переход на открытый весовой модель и обучение на доменных данных, что снизило стоимость выполнения задач в 10 раз.

Задача

Agentic AI usage caused token consumption to spike 20x, turning a +50% gross margin into -50% despite revenue growth.

Результат

Switching to a cheaper open-weight model alone was insufficient; domain-specific post-training and rigorous evaluation reduced cost per completed task by 10x and restored profitability.

Рабочий процесс

• 1. Identify token cost spike due to agentic usage.
• 2. Switch to cheaper open-weight model (Moonshot AI Kimi K3 post-trained with Fireworks AI).
• 3. Collect domain-specific post-training data (SFT, RLHF, preference ranking, correction).
• 4. Build parity benchmark against production task distribution.
• 5. Evaluate new model for hallucinations, error modes, and performance parity.
• 6. Deploy new model with monitoring for drift and ongoing QA.
• 7. Iterate post-training and evaluation to maintain cost and quality.

Доказательства

Harvey reported roughly a 10x reduction in cost per cell on its Review Tables product and a 90% reduction in cost per query on Firm Knowledge.

Источник

The model swap is the easy 10% of the project. The post-training data and the evaluation harness that proves the new model is actually as good is the other 90%.

Источник