Rachit Bansal
RL Excursions during Pre-Training: Re-examining Policy Optimization for LLM training
Rachit Bansal*, Clara Mohri*, Tian Qin*, David Alvarez-Melis, Sham Kakade
NeurIPS (under review) 2026
Paper Blog
Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retention
Jing Huang, Daniel Wurgaft, Rachit Bansal, Laura Ruis, Naomi Saphra, David Alvarez-Melis, Andrew Kyle Lampinen, Christopher Potts, Ekdeep Singh Lubana
NeurIPS (under review) 2026
Paper
Interleaved Head Attention
Sai Surya Duvvuri*, Chanakya Ekbote*, Rachit Bansal, Rishabh Tiwari, Devvrit Khatri, David Brandfonbrener, Paul Liang, Inderjit Dhillon, Manzil Zaheer
NeurIPS (under review) 2026
Paper
Let's (not) just put things in Context: Test-Time Training for Long-Context LLMs
Rachit Bansal, Aston Zhang, Rishabh Tiwari, Lovish Madaan, Sai Surya Duvvuri, Devvrit Khatri, David Brandfonbrener, David Alvarez-Melis, Prajjwal Bhargava, Mihir Sanjay Kale, Samy Jelassi
ICLR 2026
Paper
The Art of Scaling Reinforcement Learning Compute for LLMs
Devvrit Khatri*, Lovish Madaan, Rishabh Tiwari, Rachit Bansal, Sai Surya Duvvuri, Manzil Zaheer, Inderjit S. Dhillon, David Brandfonbrener, Rishabh Agarwal
ICLR (Oral Presentation) 2026
Paper
Adam or Gauss-Newton? A Comparative Study In Terms of Basis Alignment and SGD Noise
Bingbin Liu*, Rachit Bansal*, Depen Morwani*, Nikhil Vyas, David Alvarez-Melis, Sham M. Kakade
NeurIPS (under review) 2026
Paper
LLM Augmented LLMs: Expanding Capabilities through Composition
Rachit Bansal, Bidisha Samanta, Siddharth Dalmia, Nitish Gupta, Shikhar Vashishth, Sriram Ganapathy, Abhishek Bapna, Prateek Jain, Partha Talukdar
ICLR 2024
Paper Code
Measures of Information Reflect Memorization Patterns
Rachit Bansal, Danish Pruthi, Yonatan Belinkov
NeurIPS 2022
Paper Code Website
Linear Connectivity Reveals Generalization Strategies
Jeevesh Juneja, Rachit Bansal, Kyunghyun Cho, João Sedoc, Naomi Saphra
ICLR 2023
Paper Code
Evaluating Explanations: How much do explanations from the teacher aid students?
Danish Pruthi, Rachit Bansal, Bhuwan Dhingra, Livio Baldini Soares, Michael Collins, Zachary C. Lipton, Graham Neubig, William W. Cohen
TACL 2021
Paper Code
CoSe-Co: Text Conditioned Generative CommonSense Contextualizer for Language Models
Rachit Bansal, Milan Aggarwal, Sumit Bhatia, Jivat Neet Kaur, Balaji Krishnamurthy
CSKB AKBC 2021; NAACL 2022
Paper
No Need to Know Everything! Efficiently Augmenting Language Models With External Knowledge
Jivat Neet Kaur, Sumit Bhatia, Milan Aggarwal, Rachit Bansal, Balaji Krishnamurthy
CSKB AKBC 2021; NAACL (Findings) 2022
Paper
How Low is Too Low? A Computational Perspective on Extremely Low-Resource Languages
Rachit Bansal, Himanshu Choudhary, Ravneet Punia, Niko Schenk, Jacob L Dahl, Émilie Pagé-Perron
ACL SRW 2021
Paper Code