I am Heejun Lee. I am a Ph.D. student at the Korea Advanced Institute of Science and Technology AI. Hello, World! 🐱

My research interest includes:

• Neural Network Compression and Computation Optimization (Currently, I am focusing on an attention mechanism)
• Large-scale Machine Learning System Design, Development, and Serving (This is why I am working with DeepAuto.AI)
• General Purpose AI (NLP/CV)

• KAIST | Sep. 2024 - Aug. 2030 (expected) | Ph.D. in Artificial Intelligence
• KAIST | Mar. 2020 - Aug. 2024 | B.S. in Computer Science

[A2] HiP Attention: Sparse Sub-Quadratic Attention with Hierarchical Attention Pruning
[paper][code] Heejun Lee*, Geon Park*, Youngwan Lee*, Jaduk Suh*, Jina Kim, Wonyoung Jeong, Bumsik Kim, Hyemin Lee, Myeongjae Jeon, Sung Ju Hwang. Arxiv 2024

[A1] Training-Free Exponential Extension of Sliding Window Context with Cascading KV Cache
[paper][code] Jeffrey Willette, Heejun Lee, Youngwan Lee, Myeongjae Jeon, Sung Ju Hwang. Arxiv 2024

[C2] SEA: Sparse Linear Attention with Estimated Attention Mask
[paper][code] Heejun Lee, Jina Kim, Jeffrey Willette, Sung Ju Hwang. ICLR 2024

[C1] Sparse Token Transformer with Attention Back Tracking
[paper][code] Heejun Lee, Minki Kang, Youngwan Lee, Sung Ju Hwang. ICLR 2023

(*: equal contribution)

• KAIST MLAI. Jul 2021 - Aug 2024. Undergraduate Intern (I am continuing my Ph.D. program in this lab)
• DeepAuto.ai. Jan 2023 - May 2023. Research Engineer
• Transverse. Oct 2020 - Mar 2021. AI Lead Developer

• 23'F KAIST College of Engineering Leadership Award on Research Excellence
• 22'F KAIST College of Engineering Leadership Award on Research Excellence
• 22'S KAIST College of Engineering Dean's List
• 18' Intel International Science and Engineering Fair Finalist & 4th Grand Award in Robotics and Intelligent Machines
• 18' 제 9회 한국청소년과학창의대회(ISEF-K) 1등 (1st Place)
• 17' 제 34회 한국정보올림피아드(Korea Olympiad in Informatics; KOI) 공모부분 금상 (Gold Medal; 2nd Place)
• 16' 제 33회 한국정보올림피아드(Korea Olympiad in Informatics; KOI) 공모부분 금상 (Gold Medal; 2nd Place)