Offline On-Device AI Agents with Gemma & Local Tool Calls (feat. Shivay Lamba)

Описание и применение

Врач использует локальные модели STT и Gemma для транскрипции и генерации рецепта, избегая передачи данных в облако, что повышает конфиденциальность и снижает задержки.

Задача

Privacy and latency concerns when doctors send patient audio and medical records to cloud LLMs for prescription generation.

Результат

Prescription order is generated and queued locally, eliminating cloud transmission and reducing wait times.

Рабочий процесс

• 1. Doctor records consultation and prescription notes.
• 2. On-device Speech-to-Text transcribes audio to text offline.
• 3. Gemma (12B) parses transcript, extracts medication order, and calls internal pharmacy API locally.
• 4. Pharmacy API queues prescription, ready for patient pickup.

Доказательства

When a doctor consults with a patient and writes a prescription, traditional cloud LLM pipelines require sending sensitive patient audio and medical records over the network, or handing the patient a physical slip to wait in line at a pharmacy.

Источник

Instead of routing sensitive patient conversations to external cloud APIs, Shivay demonstrates a two-stage local workflow running directly on a doctor's laptop or smartphone:

Источник

1. Consultation Transcription – Local Speech-to-Text (STT) – On-Device (Laptop / Phone) – Transcribes the doctor-patient consultation and prescription notes completely offline.

Источник

1. Structured Tool Calling – Gemma (12B Parameter Model) – On-Device Local Runtime – Parses the completed transcript, extracts the medication order, and triggers an automated pharmacy tool call.

Источник

1. Pharmacy Fulfillment – Internal Order API – Local / Hospital Network – Queues the prescription so the order is ready for pickup by order number as soon as the patient arrives.

Источник