TokenGate: Adapting LLMs for pre-fill with learned skimming for ~11x speedup

TokenGate adapts large language models for pre-fill using a learned skimming technique, achieving roughly an 11× speedup, making it valuable for prototypes and