Offline On-Device AI Agents with Gemma & Local Tool Calls (feat. Shivay Lamba)

说明与用途

医生使用本地 STT 和 Gemma 模型离线转录并生成处方，避免云端数据传输并降低延迟。

问题

Privacy and latency concerns when doctors send patient audio and medical records to cloud LLMs for prescription generation.

结果

Prescription order is generated and queued locally, eliminating cloud transmission and reducing wait times.

工作流

• 1. Doctor records consultation and prescription notes.
• 2. On-device Speech-to-Text transcribes audio to text offline.
• 3. Gemma (12B) parses transcript, extracts medication order, and calls internal pharmacy API locally.
• 4. Pharmacy API queues prescription, ready for patient pickup.

证据

When a doctor consults with a patient and writes a prescription, traditional cloud LLM pipelines require sending sensitive patient audio and medical records over the network, or handing the patient a physical slip to wait in line at a pharmacy.

来源

Instead of routing sensitive patient conversations to external cloud APIs, Shivay demonstrates a two-stage local workflow running directly on a doctor's laptop or smartphone:

来源

1. Consultation Transcription – Local Speech-to-Text (STT) – On-Device (Laptop / Phone) – Transcribes the doctor-patient consultation and prescription notes completely offline.

来源

1. Structured Tool Calling – Gemma (12B Parameter Model) – On-Device Local Runtime – Parses the completed transcript, extracts the medication order, and triggers an automated pharmacy tool call.

来源

1. Pharmacy Fulfillment – Internal Order API – Local / Hospital Network – Queues the prescription so the order is ready for pickup by order number as soon as the patient arrives.

来源