Running Apple's On-Device AI From Flutter, With No API Key and No Server

Описание и применение

Flutter‑приложения теперь могут использовать встроенные модели Apple для обработки текста, генерации изображений и сканирования помещений без внешних API, снижая задержки, расходы и риски конфиденциальности.

Задача

Flutter developers typically rely on hosted AI services that require API keys, billing, and network calls, leading to latency, cost, and privacy concerns.

Результат

Flutter apps can now leverage Apple’s on-device AI for text processing, image creation, and spatial scanning without external APIs, reducing latency, cost, and privacy risks.

Рабочий процесс

• 1. Add the appropriate package (apple_foundation_models, apple_intelligence, or roomplan) to pubspec.yaml.
• 2. Check feature availability using the package’s capability check before showing UI.
• 3. For language models, create a LanguageModelSession with instructions and call respond() to get structured output.
• 4. For image generation, use ImageCreator.generate(prompt) and handle streamed partial images for progressive UI.
• 5. For room scanning, instantiate RoomScanController and retrieve structured geometry or USDZ output.
• 6. Handle fallbacks for unsupported OS, hardware, or disabled Apple Intelligence.
• 7. Integrate tool calling by exposing local functions to the model for private data access.

Доказательства

Apple ships a language model, an image generator and a LiDAR room scanner on the device. No key, no account, no per-token cost, no network — and nothing the user types leaves their phone.

Источник

The model is constrained during sampling to tokens that keep the output valid against your schema.

Источник

Image Playground generates on the device too, and the API streams partial images as they are made.

Источник

RoomPlan uses the LiDAR sensor to scan a room and hand back structured geometry.

Источник