Running Three Specialized Small Models for Self-Hosted AI

说明与用途

作者不再追求单一的完美大模型，而是转而运行三个小型的专用本地模型。这种方法提高了效率、隐私性和特定任务（聊天、编码、数学）的准确性，同时减少了在模型选择上花费的时间。

问题

The user was wasting time chasing a single 'perfect' large model, experiencing 'model-hop weekends' and frustration with generalist models making confident errors in specialized tasks like coding and math.

结果

The user achieved a more efficient and private AI workflow by splitting tasks among three small, specialized models. This eliminated the need to chase large models, reduced hardware requirements, and improved accuracy in specific domains like coding and math.

工作流

• Identify specific use cases: general chat, coding assistance, and mathematical reasoning.
• Select small, quantized models optimized for each specific task.
• Deploy the chat model in Open WebUI for daily general use and family access.
• Configure the coding model for tasks requiring careful import handling and minimal refactoring.
• Use the math model for numerical tasks where predictability is preferred over generalist confidence.
• Monitor resource usage (disk and power) which is minimal for small models.
• Evaluate success based on improvement over previous week's workflow rather than comparison to cloud frontier models.

证据

I run three. Not more. Each is quantized small enough to sit on hardware I already own, and each one is bad at the thing the other two are good at.

来源

The chat one is the workhorse... The coding one is picky... And the math one is the one I almost never admit to keeping.

来源

What I got back was my attention. No more model-hop weekends...

来源