ReplyDesk: a small tuned model for Google review replies, and the test it lost

说明与用途

ReplyDesk 使用小型 Qwen3.5‑4B 模型，基于真实店主回复进行微调，自动生成个性化的 Google 评论回复。应用展示类似的历史回复并检查未兑现的承诺，帮助店主快速编辑并发布回复，成本约为每条回复 0.0001 美元。

问题

Small shop owners have many unanswered Google reviews; writing personalized replies is time‑consuming and generic chat‑assistant replies feel impersonal.

结果

Fine‑tuning reduced stock phrases from 19.3% to 2.0% and specific promises from 18.0% to 11.3%, matching real owner reply length (34 words). Cost per reply dropped to $0.00010. However, a blind‑test judge favored the prompted model 40.3% of the time, indicating the app’s value is still uncertain.

工作流

• 1. Owner logs into the app and sets up shop facts and past replies.
• 2. New reviews appear in the inbox.
• 3. Owner clicks a review to view a draft reply generated by the fine‑tuned model.
• 4. The app displays similar past replies and a promise‑check flag.
• 5. Owner edits the draft if needed, copies it, and posts it to Google Business Profile.
• 6. The app records the reply and updates the blind‑test database for future evaluation.

证据

ReplyDesk writes replies to Google reviews for small shops: garages, salons, dentists, cafes, plumbers, small stores.

来源

One reply costs about $0.0001 on the fine‑tuned model. A shop with 60 reviews a month costs less than a cent to serve.

来源

Fine‑tuning did exactly what it was trained to do. The replies moved from where the prompted model sits to where real owners sit: the same length as real replies, far fewer stock phrases than even real owners, and fewer made‑up promises.

来源

I wrote replies to 300 held‑out reviews with four writers: the same 4B with a careful prompt, my fine‑tuned 4B, Qwen3.8‑27B with the same prompt, and the real owner's reply.

来源