RAT: Bridging RNN Efficiency and Attention Accuracy in Language Modeling
Paper • 2507.04416 • Published • 1
XiuyingWei
barpitf
AI & ML interests
None yet
Recent Activity
authored a paper 4 days ago
RAT: Bridging RNN Efficiency and Attention Accuracy in Language Modeling authored a paper 4 days ago
RAT+: Train Dense, Infer Sparse -- Recurrence Augmented Attention for Dilated Inference updated a model 3 months ago
barpitf/RATOrganizations
authored 2
papers 4 days ago
authored 4
papers 8 months ago
QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models
Paper • 2310.08041 • Published • 1
Lossy and Lossless (L$^2$) Post-training Model Size Compression
Paper • 2308.04269 • Published
From Markov to Laplace: How Mamba In-Context Learns Markov Chains
Paper • 2502.10178 • Published
Building on Efficient Foundations: Effectively Training LLMs with Structured Feedforward Layers
Paper • 2406.16450 • Published