Simran Arora
Welcome! My name is Simran and I'm a PhD student in Computer Science at Stanford, where I am advised by Christopher Ré. I am part of the Hazy Research Lab and supported by a Stanford Graduate Fellowship as the Sequoia Capital Fellow. I am also an academic partner of Cartesia AI and Together AI.
My research is in AI systems. I focus on expanding the Pareto frontier between quality and efficiency, to unlock new AI capabilities, by considering AI algorithms, hardware, and applications in lockstep. Recently, I've looked at:
I am on the academic job market 2024-25!
My research is in AI systems. I focus on expanding the Pareto frontier between quality and efficiency, to unlock new AI capabilities, by considering AI algorithms, hardware, and applications in lockstep. Recently, I've looked at:
- Algorithms: How do we build AI architectures that scale efficiently?
(Zoology, Based, JRT, LoLCATS, Monarch Mixer) - Hardware: As the complexity of AI hardware increases, how can we make it easier to extract high utilization for new AI algorithms?
(ThunderKittens, KernelBench) - Applications: How do we bridge the efficiency gaps that emerge as models are deployed in new settings, such as data management?
(AMA for personal data, Evaporate, Data Wrangling)
I am on the academic job market 2024-25!
Select Research
Complete list in CV
ThunderKittens: Simple, Fast, and Adorable AI KernelsB. Spector, S. Arora, A. Singhal, D. Fu, C. RéICLR 2025 (Spotlight Award)
LoLCATs: On Low-Rank Linearizing of Large Language ModelsM. Zhang, S. Arora, R. Chalamala, A. Wu, B. Spector, A. Singhal, K. Ramesh, C. RéICLR 2025
Just read twice: closing the recall gap for recurrent language modelsS. Arora, A. Timalsina, A. Singhal, B. Spector, S. Eyuboglu, X. Zhao, A. Rao, A. Rudra, C. RéICML ES-FoMo 2024
Simple linear attention language models balance the recall-throughput tradeoffS. Arora*, S. Eyuboglu*, M. Zhang*, A. Timalsina, S. Alberti, J. Zou, A. Rudra, C. RéICML 2024 (Spotlight Award) and ICML ES-FoMo 2024 (Oral Award, Best Paper Award)
Zoology 🐘🦁🦒: Measuring and Improving Recall in Efficient Language ModelsS. Arora*, S. Eyuboglu*, A. Timalsina, I. Johnson, M. Poli, J. Zou, A. Rudra, C. RéICLR 2024 and NeurIPS ENSLP 2023
Monarch Mixer 🦋: A Simple Sub-Quadratic GEMM-Based ArchitectureD. Y. Fu, S. Arora*, J. Grogan*, I. Johnson*, S. Eyuboglu*, A. W. Thomas*, B. F. Spector, M. Poli, A. Rudra, C. RéNeurIPS 2023 (Oral Award)
Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data LakesS. Arora, B. Yang, S. Eyuboglu, A. Narayan, A. Hojel, I. Trummer, C. RéVLDB Journal 2024
Ask Me Anything: A simple strategy for prompting language modelsS. Arora*, A. Narayan*, M. Chen, L. Orr, N. Guha, K. Bhatia, I. Chami, F. Sala, C. RéICLR 2023 (Spotlight Award)
Notes
- Efficient language models as arithmetic circuits (2024)
- ThunderKittens: Bringing fp8 to theaters near you (2024)
- Easier, Better, Faster, Cuter (2024)
- GPUs Go Brrr (2024)
- Linearizing LLMs with LoLCATs (2024)
- Just read twice: closing the recall gap for recurrent language models (2024)
- Long-Context Retrieval Models with Monarch Mixer (2024)
- Based: Simple linear attention language models balance the recall-throughput tradeoff (2024)
- Monarch Mixer: Revisiting BERT, Without Attention or MLPs (2023)
- Zoology: Overview (2023)
- The Safari of Deep Signal Processing: Hyena and Beyond (2023)
- Year in Review 2022: Technical Advances, Applications, and Social Responsibility (2023)
- Building a National AI Research Resource: A Blueprint for the National Research Cloud (2022)
- On the Opportunities and Risks of Foundation Models (2021)
- Ask Me Anything: Leveraging Foundation Models for Private & Personalized Systems (2023)
- Foundation Models are Entering their Data-Centric Era (2022)
- What Data Centric AI is Not (2021)
Recent Talks
Understanding and Improving Efficient Language ModelsSimons Institute of Computing (Fall 2024), UC Berkeley NLP Group (Summer 2024), Stanford NLP Lunch (Summer 2024), Cornell Tech (Winter 2024), Princeton PLI (Winter 2024), Microsoft AI Research (Winter 2024), 56th ACM Symposium on Theory of Computing (STOC) Workshop Keynote Speaker (Summer 2024)
Building High Throughput Data Management SystemsNeurIPS TRL Keynote Speaker (Fall 2023)
Ask Me Anything: How are Foundation Models Changing the Way We Build Data Management Systems?Snorkel Foundation Model Summit (Winter 2023), Apple Machine Learning Research Team Reading Group (Winter 2023), ICLR Spotlight Presentation (Spring 2023), CRFM Research Spotlight Talks (Fall 2023)
Teaching
CS 229s Systems for Machine Learning (Fall '23) - Co-Creator and Co-InstructorStanford University, Department of Computer Science
CS 528 Machine Learning Systems Seminar (Fall '23) - Course InstructorStanford University, Department of Computer Science
MCIT 595 Computer Systems Coursera course (Spring '19) - Co-Creator & Course AssistantUniversity of Pennsylvania, Department of Computer Science
CIS 380 Operating Systems (Fall '18) - Course AssistantUniversity of Pennsylvania, Department of Computer Science
CIS 160 Discrete Math (Fall '17, Spring '18) - Course AssistantUniversity of Pennsylvania, Department of Computer Science