On-Device Continual Learning for Android Notification Routing with Online-SDFT

说明与用途

Online-SDFT 是一个原型，允许 2.3 亿参数的语言模型在 Android 设备上通过延迟的用户反馈进行持续学习。它使用自蒸馏和 LoRA 适配器在本地更新模型，无需服务器通信，从而提高了通知路由的准确性。

问题

Small language models on mobile devices typically stop learning after deployment. Standard fine-tuning requires clean labels which are unavailable for delayed, ambiguous user interactions (e.g., notification dismissals), and reinforcement learning requires reliable scalar rewards which are difficult to define for personal AI tasks.

结果

In synthetic tests, Online-SDFT achieved 70.3% preference accuracy and 44.8 cumulative regret, outperforming frozen models (28.2%), RAG (50.0%), and rejection fine-tuning (52.8%). On a physical Android device, the model successfully learned to suppress notifications upon repeated dismissal and resume showing them when requested.

工作流

• Provision the frozen 230M base model and LoRA adapter on the Android device.
• At decision time, the student model samples an action (e.g., show, save, archive notification) based on current context.
• Execute the action and wait for the user's outcome/hindsight (e.g., dismissal, opening, rewriting).
• The teacher model (same base, LoRA disabled) reviews the context plus the observed outcome to generate a soft target distribution.
• If the outcome is causally supported and reliable, add the interaction to a bounded on-device replay buffer.
• Sample balanced interactions from the replay buffer and update the LoRA adapter using ONNX Runtime Training.
• Persist adapter checkpoints and replay state in app-private storage.

证据

Online-SDFT matched the hidden sampled preference on 506 of 720 decisions.

来源

In one physical-phone test, repeatedly dismissing a notification teaches the model to keep it quiet.

来源