Floating Dictation Orb on Android

说明与用途

RMBLR 是一款 Android 应用，只有在文本框获得焦点时才会显示浮动语音输入球，利用 AccessibilityService 检测焦点并将音频流式传输到 LLM，实现低延迟转录。

问题

Existing dictation tools fail when speaking in mixed languages; they lock to one language and require retyping.

结果

RMBLR provides a lightweight, privacy‑preserving floating dictation orb that appears only when a text field is focused, supports mixed‑language input, and streams audio for low‑latency transcription.

工作流

• 1. Implement AccessibilityService to detect focused editable fields using findFocus(FOCUS_INPUT) and isEditable or EditText class check.
• 2. Configure service XML to listen to focus, click, text change, selection, window state/content events with notificationTimeout 80ms.
• 3. Create TYPE_APPLICATION_OVERLAY window with FLAG_NOT_FOCUSABLE to keep caret active; use FLAG_LAYOUT_NO_LIMITS for status bar cutout.
• 4. Handle touch on overlay: long press 300ms, movement >12dp cancels long press, angle-based chip selection, flick detection for instant action.
• 5. On press, open socket and stream audio in 200ms chunks; use server-side voice detection to determine session end after quiet period.
• 6. Buffer full audio to fallback to batch endpoint if stream fails.
• 7. Paste transcribed text into target field using ACTION_PASTE or ACTION_SET_TEXT as fallback.
• 8. Deploy app with user-provided API key (Gemini, Groq, etc.).

证据

Android gives you exactly one API that can answer "is there a caret in an editable field right now" for an app you did not write: an AccessibilityService.

来源

The orb is a TYPE_APPLICATION_OVERLAY window, and if it takes focus, the text field loses the caret, and now findFocus(FOCUS_INPUT) returns null and the orb hides itself.

来源