Can I Ask a Robot to Pack My Picnic in My Own Words?

说明与用途

作者评估了 SmolVLA 在接收自然语言改写指令时包装野餐物品的能力。仅包含 "put ... in the basket" 的模板取得了中等成功率，其他自然表达几乎完全失败。

问题

Can a robot understand and execute picnic packing instructions given in natural language variations instead of a fixed template?

结果

Out of 160 paraphrase trials, only 52 (33%) succeeded. The original template succeeded 100% of the time. Paraphrases A and D (containing "put ... in the basket") succeeded about half the time, while B and C failed almost entirely.

工作流

• 1. Load SmolVLA model (lerobot/smolvla_libero) on CPU.
• 2. Run baseline evaluation with original instructions for all ten items (10 episodes each).
• 3. Record success rate (89/100 episodes).
• 4. Select four items that scored 10/10 and generate four paraphrases (A–D).
• 5. Run 10 episodes per paraphrase, swapping instruction while keeping initial states and seeds.
• 6. Aggregate success counts for each paraphrase.
• 7. Analyze results and hypothesize causes of failure.

证据

The training sentence worked every time. The paraphrases split in two: A and D, which keep "put ... in the basket", worked about half the time (22/40 and 26/40), while B and C almost never worked (2/40 each).

来源

89 of 100 episodes succeeded.

来源

The robot is fast in simulation and slow in real life. Each SmolVLA call plans 0.5 s of motion but takes 1.15 s on this CPU.

来源