Automated ML vs LLM Task Router using Oxlo.ai

Описание и применение

Автоматизированный маршрутизатор задач на Python, который использует традиционный ML (scikit-learn) и LLM (через Oxlo.ai) для определения оптимального подхода к решению задачи, экономя время на архитектурные дебаты.

Задача

Teams often debate whether a new project requires a classical ML pipeline with labeling or simply an LLM prompt, leading to potential over-engineering or under-utilization of resources.

Результат

The Task Router successfully distinguishes between tasks suitable for traditional ML (e.g., structured classification) and those requiring LLMs (e.g., open-ended generation), providing clear reasoning and confidence levels for each recommendation.

Рабочий процесс

• Set up the Oxlo.ai client using the OpenAI SDK with the Oxlo.ai base URL and API key.
• Train a traditional ML baseline (e.g., RandomForest with TF-IDF) on a small labeled dataset to serve as a comparison point.
• Define a strict system prompt for the LLM to output JSON recommendations based on task characteristics (structured vs. generative).
• Create a function to call the Oxlo.ai LLM (Llama 3.3 70B) with the task description and ML baseline context.
• Implement a comparison harness that runs both the traditional ML prediction and the LLM router side-by-side.
• Execute the router on test scenarios (e.g., classification vs. email generation) to validate the recommendations.

Доказательства

It compares the problem against a live traditional ML baseline, then uses an LLM hosted on Oxlo.ai to explain which paradigm fits and why, saving engineering hours and preventing over-engineering.

Источник

I use Oxlo.ai here because the flat per-request pricing... means I can iterate on the prompt and run many evaluations without watching token meters spin up.

Источник