Choosing an Open-Source LLM for Our Waste‑Sorting Assistant

Описание и применение

Выбор модели Qwen3‑4B‑Instruct‑2507 позволил создать помощника по сортировке отходов, работающего на бесплатном GPU Colab T4, обеспечивая точные и форматированные ответы без дополнительных ключей API.

Задача

Selecting a free, instruction‑following LLM that fits within a 16 GB GPU and can provide concise, format‑consistent answers for a waste‑sorting assistant.

Результат

The assistant now delivers consistent, instruction‑following responses within memory limits, enabling accurate waste sorting and rule explanation.

Рабочий процесс

• 1. Identify project constraints (budget, GPU memory, output format).
• 2. Search for open‑source LLMs ≤4 B parameters that fit 16 GB.
• 3. Evaluate candidates on size, license, context window, instruction following, and ease of use.
• 4. Test Qwen3‑4B‑Instruct‑2507 on Colab T4: download, load in float16, confirm memory usage.
• 5. Verify output format consistency and instruction compliance.
• 6. Deploy model in the assistant pipeline.
• 7. Monitor for memory limits and fallback to Qwen3‑1.7B if needed.

Доказательства

Our project is an assistant that tells people which bin an item belongs in, and why.

Источник

Our team has no budget and uses the free Google Colab T4 GPU, which has 16 GB of memory.

Источник

I chose Qwen3-4B-Instruct-2507 for four reasons.

Источник

It fits on the free Colab T4. I downloaded and loaded it on a free Colab T4. The model files are 8.04 GB, which matches my estimate of 4 billion × 2 bytes.

Источник

It follows instructions and gives clean answers. Our assistant must answer in the same format every time.

Источник

Limits. Qwen3-4B is the biggest of the four, so it is a little slower and uses the most memory.

Источник