ReplyDesk: a small tuned model for Google review replies, and the test it lost

Описание и применение

ReplyDesk использует небольшую модель Qwen3.5‑4B, дообученную на реальных ответах владельцев магазинов, чтобы автоматически генерировать персонализированные ответы на отзывы Google. Приложение показывает похожие прошлые ответы и проверяет обещания, позволяя владельцам быстро редактировать и публиковать ответы с минимальными затратами (~$0.0001 за ответ).

Задача

Small shop owners have many unanswered Google reviews; writing personalized replies is time‑consuming and generic chat‑assistant replies feel impersonal.

Результат

Fine‑tuning reduced stock phrases from 19.3% to 2.0% and specific promises from 18.0% to 11.3%, matching real owner reply length (34 words). Cost per reply dropped to $0.00010. However, a blind‑test judge favored the prompted model 40.3% of the time, indicating the app’s value is still uncertain.

Рабочий процесс

• 1. Owner logs into the app and sets up shop facts and past replies.
• 2. New reviews appear in the inbox.
• 3. Owner clicks a review to view a draft reply generated by the fine‑tuned model.
• 4. The app displays similar past replies and a promise‑check flag.
• 5. Owner edits the draft if needed, copies it, and posts it to Google Business Profile.
• 6. The app records the reply and updates the blind‑test database for future evaluation.

Доказательства

ReplyDesk writes replies to Google reviews for small shops: garages, salons, dentists, cafes, plumbers, small stores.

Источник

One reply costs about $0.0001 on the fine‑tuned model. A shop with 60 reviews a month costs less than a cent to serve.

Источник

Fine‑tuning did exactly what it was trained to do. The replies moved from where the prompted model sits to where real owners sit: the same length as real replies, far fewer stock phrases than even real owners, and fewer made‑up promises.

Источник

I wrote replies to 300 held‑out reviews with four writers: the same 4B with a careful prompt, my fine‑tuned 4B, Qwen3.8‑27B with the same prompt, and the real owner's reply.

Источник