Rebuilding a brain-to-text decoder: from 50% to 23.5% word errors

说明与用途

通过架构调整、改进解码和更强大的语言模型，将基于运动皮层的语音 BCI 解码器的词错误率从 50% 降低到 23.5%。

问题

Improve word error rate of a speech BCI decoder trained on motor cortex activity from 50% to 23.5% using offline reanalysis of published neural data.

结果

The decoder’s word error rate was reduced from 50.0% to 23.5% (23.0% on a held‑out half) by architectural changes, improved decoding, and a better language model.

工作流

• 1. Use only area 6v features (threshold crossings and spike‑band power) and z‑score per recording block.
• 2. Shrink the GRU to 3 layers of 512 units and group sentences by length to reduce padding.
• 3. Decode words directly from CTC output with a lexicon and 3‑gram LM, then let a large language model (Claude Sonnet 4.6) pick the best of the top 10 candidates.
• 4. Replace the training‑sentence LM with a Tatoeba English 3‑gram (≈2 M sentences) and restrict the lexicon to the LM’s 6,569 words.
• 5. Evaluate on a split test set (even sentences for dev, odd for eval) and report the held‑out error rate.

证据

The model trained on CPUs for about five hours, and the word error rate on 880 test sentences went from 50.0% to 23.5% over four changes

来源

I shrank the GRU from 5 layers and 1,024 units to 3 layers and 512 units (37.8 million parameters) and grouped sentences by length to cut padding.

来源

I decoded words directly from the CTC output with a lexicon and a 3-gram, using torchaudio's flashlight decoder, and gave the LLM the 10 best sequences to choose from by number

来源

With the full CMU dictionary (about 130,000 words) the error rate jumped to around 90%. My LM gave <unk> about 7% of the probability mass, so every rare dictionary word scored as well as an unknown word and won often.

来源

With it, 297 of 880 sentences came out exactly right, and the oracle (best of the 10 candidates, chosen with the answer) is 19.4%

来源