Building an iOS Moderation Layer with Apple Foundation Models framework and Firebase

Описание и применение

Гибридная архитектура модерации для iOS использует локальные модели Apple Foundation для быстрой проверки текста и Firebase для окончательного решения, обеспечивая быстрый, приватный и последовательный контроль контента на всех устройствах.

Задача

Developers need a moderation system that works fast, privately, and consistently across devices, including those without Apple Intelligence support.

Результат

The hybrid approach delivers fast, private preflight moderation on supported devices, while maintaining consistent server‑side enforcement and fallback paths for unsupported devices, resulting in a robust moderation layer suitable for iOS apps.

Рабочий процесс

• 1. User composes text in the app.
• 2. App calls classifyDraft using Apple Foundation Models; if available, receives a structured ModerationSignal with label, confidence, PII flag, and user message.
• 3. If the model is unavailable, the app skips preflight and proceeds to step 4.
• 4. App sends content, local signal (if any), and auth context to a Firebase Callable Function submitContent.
• 5. Server checks for PII, spam, and local signal labels to decide action: publish, review, or block.
• 6. Server logs decision reason and stores content in Firestore.
• 7. For media uploads, a Cloud Storage trigger runs a separate moderation function and writes moderation state back to Firestore.
• 8. Client UI reflects moderation state and provides feedback to the user.

Доказательства

Apple’s Foundation Models framework gives fast, private, on-device text classification and rewrite suggestions.

Источник

Firebase callable functions automatically include Auth and App Check tokens when available, and App Check enforcement can be turned on serverside.

Источник

Cloud Storage triggers for handling media separately, because images and video are their own special form of pain.

Источник