Running Apple's On-Device AI From Flutter, With No API Key and No Server

说明与用途

Flutter 应用现在可以利用 Apple 的设备内 AI 进行文本处理、图像生成和房间扫描，无需外部 API，降低延迟、成本和隐私风险。

问题

Flutter developers typically rely on hosted AI services that require API keys, billing, and network calls, leading to latency, cost, and privacy concerns.

结果

Flutter apps can now leverage Apple’s on-device AI for text processing, image creation, and spatial scanning without external APIs, reducing latency, cost, and privacy risks.

工作流

• 1. Add the appropriate package (apple_foundation_models, apple_intelligence, or roomplan) to pubspec.yaml.
• 2. Check feature availability using the package’s capability check before showing UI.
• 3. For language models, create a LanguageModelSession with instructions and call respond() to get structured output.
• 4. For image generation, use ImageCreator.generate(prompt) and handle streamed partial images for progressive UI.
• 5. For room scanning, instantiate RoomScanController and retrieve structured geometry or USDZ output.
• 6. Handle fallbacks for unsupported OS, hardware, or disabled Apple Intelligence.
• 7. Integrate tool calling by exposing local functions to the model for private data access.

证据

Apple ships a language model, an image generator and a LiDAR room scanner on the device. No key, no account, no per-token cost, no network — and nothing the user types leaves their phone.

来源

The model is constrained during sampling to tokens that keep the output valid against your schema.

来源

Image Playground generates on the device too, and the API streams partial images as they are made.

来源

RoomPlan uses the LiDAR sensor to scan a room and hand back structured geometry.

来源