We ran HealthBench on our health AI's safety layer. It scored lower than the bare model

Description and use

Testing Tabibu’s safety layer on HealthBench revealed that the layer lowers benchmark scores, but policy tweaks improve them and uncover dose‑quoting bugs.

Problem

Assessing the impact of a safety layer on a health AI’s performance on a public benchmark.

Outcome

The safety layer reduced benchmark scores due to its conservative policies, but policy tweaks improved performance while uncovering a safety bug involving prescription dose quoting.

Workflow

• Run HealthBench on a 150‑conversation subset with a gpt‑4.1‑mini grader
• Compare scores of (1) full Tabibu pipeline, (2) bare gpt‑6‑luna, (3) bare gpt‑4o‑mini
• Identify per‑criterion failures
• Modify safety policies to allow general answers and tiered doses
• Re‑run benchmark
• Implement safety fix for prescription dose quoting
• Re‑run benchmark

Evidence

The full pipeline scored 0.362 on 150 HealthBench conversations

Source

The pipeline was 0.158 below the bare model on the same conversations (paired difference, ±0.039).

Source

I changed one policy: let the assistant answer general, low‑risk questions from the model's own knowledge, with a clear label saying the answer is not from its curated sources, instead of refusing.

Source

With the original rules, the pipeline could quote prescription doses straight out of a retrieved national formulary

Source