Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

10 local LLMs, one brand prompt, 8 setups: what I learned generating product copy on an ASUS GX10

Описание и применение

Автоматизация генерации немецких описаний товаров для B2B-магазина с 10 000+ товаров на локальном ноутбуке ASUS GX10. Лучший результат показал Gemma 4 26B с простым запросом; брендовый запрос снизил оценку. Квантование QAT 4‑бит сохранило качество и ускорило генерацию, а пакетная обработка vLLM ускорила работу до 1,6 дня.

Задача

Generate correct German product descriptions for a B2B workwear shop with 10,000+ articles, adhering to brand rules and avoiding invented properties, on a local machine without discrete GPU.

Результат

Gemma 4 26B achieved the highest score (288/300) with plain prompt; brand prompt reduced scores by ~60 points. Quantization-aware training (4‑bit QAT) maintained quality while improving speed; post‑hoc 4‑bit quant lost 22 points. Batching with vLLM ×8 reduced generation time from 5.4 to 1.6 days for 10,000 articles.

Рабочий процесс

  • Collect supplier data and product metadata
  • Clean data in code before prompting
  • Apply fixed rules for pattern-based facts
  • Prompt LLM with brand voice prompt
  • LLM rewrites only flagged sentences
  • Validate output against rubric with Claude reviewer
  • Iterate on model build and quantization
  • Batch requests with vLLM for speed
  • Store generated HTML and SEO fields
  • Human spot‑check for quality assurance

Доказательства

Ten local models, 30 articles, same data, same prompt. The Gemma 4 family won (Gemma 4 26B: 288/300, with 12B and 31B right behind).

Источник

Scoring was stricter because tone counted, so look at the spread: 7 points with the plain prompt, 111 with the real one.

Источник

Full precision bought nothing. BF16 was slower and not better.

Источник

Batching is the speed lever. vLLM with 8 concurrent requests instead of Ollama with 2: 10,000 articles in about 1.6 days instead of 5.4.

Источник

  • Дата записи: 2026-10-22T00:26:20.759Z