On-Device Continual Learning for Android Notification Routing with Online-SDFT

Description and use

Online-SDFT is a prototype enabling a 230M-parameter language model to continually learn on an Android device using delayed user feedback. It uses self-distillation and LoRA adapters to update the model locally without server communication, improving notification routing accuracy.

Problem

Small language models on mobile devices typically stop learning after deployment. Standard fine-tuning requires clean labels which are unavailable for delayed, ambiguous user interactions (e.g., notification dismissals), and reinforcement learning requires reliable scalar rewards which are difficult to define for personal AI tasks.

Outcome

In synthetic tests, Online-SDFT achieved 70.3% preference accuracy and 44.8 cumulative regret, outperforming frozen models (28.2%), RAG (50.0%), and rejection fine-tuning (52.8%). On a physical Android device, the model successfully learned to suppress notifications upon repeated dismissal and resume showing them when requested.

Workflow

• Provision the frozen 230M base model and LoRA adapter on the Android device.
• At decision time, the student model samples an action (e.g., show, save, archive notification) based on current context.
• Execute the action and wait for the user's outcome/hindsight (e.g., dismissal, opening, rewriting).
• The teacher model (same base, LoRA disabled) reviews the context plus the observed outcome to generate a soft target distribution.
• If the outcome is causally supported and reliable, add the interaction to a bounded on-device replay buffer.
• Sample balanced interactions from the replay buffer and update the LoRA adapter using ONNX Runtime Training.
• Persist adapter checkpoints and replay state in app-private storage.

Evidence

Online-SDFT matched the hidden sampled preference on 506 of 720 decisions.

Source

In one physical-phone test, repeatedly dismissing a notification teaches the model to keep it quiet.

Source