Floating Dictation Orb on Android

Описание и применение

RMBLR – Android‑приложение с плавающей орбитой диктовки, которое появляется только при фокусе на текстовом поле, использует AccessibilityService для обнаружения фокуса и потоковую передачу аудио в LLM для низкой задержки.

Задача

Existing dictation tools fail when speaking in mixed languages; they lock to one language and require retyping.

Результат

RMBLR provides a lightweight, privacy‑preserving floating dictation orb that appears only when a text field is focused, supports mixed‑language input, and streams audio for low‑latency transcription.

Рабочий процесс

• 1. Implement AccessibilityService to detect focused editable fields using findFocus(FOCUS_INPUT) and isEditable or EditText class check.
• 2. Configure service XML to listen to focus, click, text change, selection, window state/content events with notificationTimeout 80ms.
• 3. Create TYPE_APPLICATION_OVERLAY window with FLAG_NOT_FOCUSABLE to keep caret active; use FLAG_LAYOUT_NO_LIMITS for status bar cutout.
• 4. Handle touch on overlay: long press 300ms, movement >12dp cancels long press, angle-based chip selection, flick detection for instant action.
• 5. On press, open socket and stream audio in 200ms chunks; use server-side voice detection to determine session end after quiet period.
• 6. Buffer full audio to fallback to batch endpoint if stream fails.
• 7. Paste transcribed text into target field using ACTION_PASTE or ACTION_SET_TEXT as fallback.
• 8. Deploy app with user-provided API key (Gemini, Groq, etc.).

Доказательства

Android gives you exactly one API that can answer "is there a caret in an editable field right now" for an app you did not write: an AccessibilityService.

Источник

The orb is a TYPE_APPLICATION_OVERLAY window, and if it takes focus, the text field loses the caret, and now findFocus(FOCUS_INPUT) returns null and the orb hides itself.

Источник