Simran Arora

Welcome! My name is Simran and I'm a PhD student in Computer Science at Stanford, where I am advised by Christopher Ré. I am part of the Hazy Research Lab and supported by a Stanford Graduate Fellowship as the Sequoia Capital Fellow. I am also an academic partner of Cartesia AI and Together AI.

My research is in AI Systems. I work on developing AI that scales and uses the hardware efficiently.
I recently co-created and taught a new systems for machine learning course (CS 229s) at Stanford!

I am on the academic job market 2024-25!

Select Research

Complete list in CV
ThunderKittens: Simple, Fast, and Adorable AI KernelsB. Spector, S. Arora, A. Singhal, D. Fu, C. Ré
LoLCATs: On Low-Rank Linearizing of Large Language ModelsM. Zhang, S. Arora, R. Chalamala, A. Wu, B. Spector, A. Singhal, K. Ramesh, C. Ré
Just read twice: closing the recall gap for recurrent language modelsS. Arora, A. Timalsina, A. Singhal, B. Spector, S. Eyuboglu, X. Zhao, A. Rao, A. Rudra, C. RéICML ES-FoMo 2024
Simple linear attention language models balance the recall-throughput tradeoffS. Arora*, S. Eyuboglu*, M. Zhang*, A. Timalsina, S. Alberti, J. Zou, A. Rudra, C. RéICML 2024 (Spotlight Award) and ICML ES-FoMo 2024 (Oral Award, Best Paper Award)
Zoology 🐘🦁🦒: Measuring and Improving Recall in Efficient Language ModelsS. Arora*, S. Eyuboglu*, A. Timalsina, I. Johnson, M. Poli, J. Zou, A. Rudra, C. RéICLR 2024 and NeurIPS ENSLP 2023
Monarch Mixer 🦋: A Simple Sub-Quadratic GEMM-Based ArchitectureD. Y. Fu, S. Arora*, J. Grogan*, I. Johnson*, S. Eyuboglu*, A. W. Thomas*, B. F. Spector, M. Poli, A. Rudra, C. RéNeurIPS 2023 (Oral Award)
Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data LakesS. Arora, B. Yang, S. Eyuboglu, A. Narayan, A. Hojel, I. Trummer, C. RéVLDB Journal 2024
Ask Me Anything: A simple strategy for prompting language modelsS. Arora*, A. Narayan*, M. Chen, L. Orr, N. Guha, K. Bhatia, I. Chami, F. Sala, C. RéICLR 2023 (Spotlight Award)

Notes

Recent Talks

ThunderKittens goes live: AMA and library walkthroughStanford MLSys Seminars (Winter 2024)
Understanding and Improving Efficient Language ModelsSimons Institute of Computing (Fall 2024), UC Berkeley NLP Group (Summer 2024), Stanford NLP Lunch (Summer 2024), Cornell Tech (Winter 2024), Princeton PLI (Winter 2024), Microsoft AI Research (Winter 2024), 56th ACM Symposium on Theory of Computing (STOC) Workshop Keynote Speaker (Summer 2024)
Building High Throughput Data Management SystemsNeurIPS TRL Keynote Speaker (Fall 2023)
Ask Me Anything: How are Foundation Models Changing the Way We Build Data Management Systems?Snorkel Foundation Model Summit (Winter 2023), Apple Machine Learning Research Team Reading Group (Winter 2023), ICLR Spotlight Presentation (Spring 2023), CRFM Research Spotlight Talks (Fall 2023)

Teaching

CS 229s Systems for Machine Learning (Fall '23) - Co-Creator and Co-InstructorStanford University, Department of Computer Science
CS 528 Machine Learning Systems Seminar (Fall '23) - Course InstructorStanford University, Department of Computer Science
MCIT 595 Computer Systems Coursera course (Spring '19) - Co-Creator & Course AssistantUniversity of Pennsylvania, Department of Computer Science
CIS 380 Operating Systems (Fall '18) - Course AssistantUniversity of Pennsylvania, Department of Computer Science
CIS 160 Discrete Math (Fall '17, Spring '18) - Course AssistantUniversity of Pennsylvania, Department of Computer Science