Running Apple's On-Device AI From Flutter, With No API Key and No Server

Description and use

Flutter apps can now leverage Apple’s on-device AI for text processing, image creation, and room scanning without external APIs, reducing latency, cost, and privacy risks.

Problem

Flutter developers typically rely on hosted AI services that require API keys, billing, and network calls, leading to latency, cost, and privacy concerns.

Outcome

Flutter apps can now leverage Apple’s on-device AI for text processing, image creation, and spatial scanning without external APIs, reducing latency, cost, and privacy risks.

Workflow

• 1. Add the appropriate package (apple_foundation_models, apple_intelligence, or roomplan) to pubspec.yaml.
• 2. Check feature availability using the package’s capability check before showing UI.
• 3. For language models, create a LanguageModelSession with instructions and call respond() to get structured output.
• 4. For image generation, use ImageCreator.generate(prompt) and handle streamed partial images for progressive UI.
• 5. For room scanning, instantiate RoomScanController and retrieve structured geometry or USDZ output.
• 6. Handle fallbacks for unsupported OS, hardware, or disabled Apple Intelligence.
• 7. Integrate tool calling by exposing local functions to the model for private data access.

Evidence

Apple ships a language model, an image generator and a LiDAR room scanner on the device. No key, no account, no per-token cost, no network — and nothing the user types leaves their phone.

Source

The model is constrained during sampling to tokens that keep the output valid against your schema.

Source

Image Playground generates on the device too, and the API streams partial images as they are made.

Source

RoomPlan uses the LiDAR sensor to scan a room and hand back structured geometry.

Source