Can I Ask a Robot to Pack My Picnic in My Own Words?

Описание и применение

Автор проверил, насколько модель SmolVLA может выполнять команды по упаковке пикника, если они сформулированы естественным языком. Тесты показали, что только шаблон с фразой "положить в корзину" работает примерно половину раз, а более естественные варианты почти не срабатывают.

Задача

Can a robot understand and execute picnic packing instructions given in natural language variations instead of a fixed template?

Результат

Out of 160 paraphrase trials, only 52 (33%) succeeded. The original template succeeded 100% of the time. Paraphrases A and D (containing "put ... in the basket") succeeded about half the time, while B and C failed almost entirely.

Рабочий процесс

• 1. Load SmolVLA model (lerobot/smolvla_libero) on CPU.
• 2. Run baseline evaluation with original instructions for all ten items (10 episodes each).
• 3. Record success rate (89/100 episodes).
• 4. Select four items that scored 10/10 and generate four paraphrases (A–D).
• 5. Run 10 episodes per paraphrase, swapping instruction while keeping initial states and seeds.
• 6. Aggregate success counts for each paraphrase.
• 7. Analyze results and hypothesize causes of failure.

Доказательства

The training sentence worked every time. The paraphrases split in two: A and D, which keep "put ... in the basket", worked about half the time (22/40 and 26/40), while B and C almost never worked (2/40 each).

Источник

89 of 100 episodes succeeded.

Источник

The robot is fast in simulation and slow in real life. Each SmolVLA call plans 0.5 s of motion but takes 1.15 s on this CPU.

Источник