Generative AI Development Is 20% Prompting and 80% Everything Else

Description and use

<p>Last spring I built a working generative AI feature in a weekend: a support assistant that read a customer's question, pulled context from our help docs, and wrote a friendly, accurate answer. The demo went great. Everyone nodded.</p> <p>Shipping it took three more months.</p> <p>Not because the prompt was wrong. The prompt barely changed. The time went into everything around it: cleaning the knowledge base, building an evaluation set, handling the cases where the model confidently made things up, keeping costs from spiraling, and wiring it into systems that had never heard of an LLM.</p> <

Problem

<p>Last spring I built a working generative AI feature in a weekend: a support assistant that read a customer's question, pulled context from our help docs, and wrote a friendly, accurate answer. The demo went great. Everyone nodded.</p> <p>Shipping it took three more months.</p> <p>Not because the prompt was wrong. The prompt barely changed. The time went into everything around it: cleaning the knowledge base, building an evaluation set, handling the cases where the model confidently made things up, keeping costs from spiraling, and wiring it into systems that had never heard of an LLM.</p> <

Outcome

<p>Last spring I built a working generative AI feature in a weekend: a support assistant that read a customer's question, pulled context from our help docs, and wrote a friendly, accurate answer. The demo went great. Everyone nodded.</p> <p>Shipping it took three more months.</p> <p>Not because the prompt was wrong. The prompt barely changed. The time went into everything around it: cleaning the knowledge base, building an evaluation set, handling the cases where the model confidently made things up, keeping costs from spiraling, and wiring it into systems that had never heard of an LLM.</p> <

Evidence

Generative AI Development Is 20% Prompting and 80% Everything Else

Source