RAG vs. Fine-Tuning: The Wrong Question to Ask When Building AI Systems

Description and use

<p>A platform team is evaluating an LLM-based assistant, still in testing, meant to answer developers' architecture questions using the company's ADRs (Architecture Decision Records) and internal documentation. During an evaluation round, the team itself notices that the assistant recommended a service-to-service authentication pattern the company abandoned last year and, on top of that, suggested calling an endpoint on an internal API already marked as deprecated. The answer was plausible, well written, and technically defensible in general terms. It just contradicted a standing decision.</p>

Problem

<p>A platform team is evaluating an LLM-based assistant, still in testing, meant to answer developers' architecture questions using the company's ADRs (Architecture Decision Records) and internal documentation. During an evaluation round, the team itself notices that the assistant recommended a service-to-service authentication pattern the company abandoned last year and, on top of that, suggested calling an endpoint on an internal API already marked as deprecated. The answer was plausible, well written, and technically defensible in general terms. It just contradicted a standing decision.</p>

Outcome

<p>A platform team is evaluating an LLM-based assistant, still in testing, meant to answer developers' architecture questions using the company's ADRs (Architecture Decision Records) and internal documentation. During an evaluation round, the team itself notices that the assistant recommended a service-to-service authentication pattern the company abandoned last year and, on top of that, suggested calling an endpoint on an internal API already marked as deprecated. The answer was plausible, well written, and technically defensible in general terms. It just contradicted a standing decision.</p>

Evidence

RAG vs. Fine-Tuning: The Wrong Question to Ask When Building AI Systems

Source