You Cannot Unit Test a Summary. You Can Still Test It.

Описание и применение

<p>Every developer I know has the same reflex when a feature is done: write a test that feeds it an input and asserts the output. That reflex is correct for almost everything we build. Click save, the record is saved. Parse the date, get the date.</p> <p>Then you ship your first AI feature and the reflex quietly stops working. Ask a model to summarise the same lecture twice and you get two different summaries, both fine. Ask it to classify a support message and it is right ninety-nine times and confidently wrong on the hundredth. <code>expect(output).toBe(expected)</code> has nothing to hold o

Задача

<p>Every developer I know has the same reflex when a feature is done: write a test that feeds it an input and asserts the output. That reflex is correct for almost everything we build. Click save, the record is saved. Parse the date, get the date.</p> <p>Then you ship your first AI feature and the reflex quietly stops working. Ask a model to summarise the same lecture twice and you get two different summaries, both fine. Ask it to classify a support message and it is right ninety-nine times and confidently wrong on the hundredth. <code>expect(output).toBe(expected)</code> has nothing to hold o

Результат

<p>Every developer I know has the same reflex when a feature is done: write a test that feeds it an input and asserts the output. That reflex is correct for almost everything we build. Click save, the record is saved. Parse the date, get the date.</p> <p>Then you ship your first AI feature and the reflex quietly stops working. Ask a model to summarise the same lecture twice and you get two different summaries, both fine. Ask it to classify a support message and it is right ninety-nine times and confidently wrong on the hundredth. <code>expect(output).toBe(expected)</code> has nothing to hold o

Доказательства

You Cannot Unit Test a Summary. You Can Still Test It.

Источник