On-Device Continual Learning for Android Notification Routing with Online-SDFT

Описание и применение

Прототип Online-SDFT позволяет языковой модели объемом 230M параметров непрерывно обучаться на Android-устройстве, используя задержанную обратную связь от пользователя. Система использует самодистилляцию и LoRA-адаптеры для обновления модели локально, без отправки данных на сервер, что повышает точность маршрутизации уведомлений.

Задача

Small language models on mobile devices typically stop learning after deployment. Standard fine-tuning requires clean labels which are unavailable for delayed, ambiguous user interactions (e.g., notification dismissals), and reinforcement learning requires reliable scalar rewards which are difficult to define for personal AI tasks.

Результат

In synthetic tests, Online-SDFT achieved 70.3% preference accuracy and 44.8 cumulative regret, outperforming frozen models (28.2%), RAG (50.0%), and rejection fine-tuning (52.8%). On a physical Android device, the model successfully learned to suppress notifications upon repeated dismissal and resume showing them when requested.

Рабочий процесс

• Provision the frozen 230M base model and LoRA adapter on the Android device.
• At decision time, the student model samples an action (e.g., show, save, archive notification) based on current context.
• Execute the action and wait for the user's outcome/hindsight (e.g., dismissal, opening, rewriting).
• The teacher model (same base, LoRA disabled) reviews the context plus the observed outcome to generate a soft target distribution.
• If the outcome is causally supported and reliable, add the interaction to a bounded on-device replay buffer.
• Sample balanced interactions from the replay buffer and update the LoRA adapter using ONNX Runtime Training.
• Persist adapter checkpoints and replay state in app-private storage.

Доказательства

Online-SDFT matched the hidden sampled preference on 506 of 720 decisions.

Источник

In one physical-phone test, repeatedly dismissing a notification teaches the model to keep it quiet.

Источник