# Fanyi Pu 濮凡轶 [Homepage](https://pufanyi.com/) · [Curriculum Vitae](https://pufanyi.com/cv) Incoming Ph.D. Student in Computer Science University of Wisconsin–Madison - [+86 133-3671-7687](tel:+8613336717687) - * [Homepage](https://pufanyi.com/) * [GitHub](https://github.com/pufanyi) * [Google Scholar](https://scholar.google.com/citations?user=58tv6skAAAAJ) * [LinkedIn](https://linkedin.com/in/pufanyi) * [X (Twitter)](https://x.com/pufanyi) * [Blog](https://pufanyi.com/blog) ## About I am currently a final-year undergraduate student at [Nanyang Technological University](https://www.ntu.edu.sg/), specializing in [Data Science and Artificial Intelligence](https://www.ntu.edu.sg/education/undergraduate-programme/bachelor-of-science-in-data-science-artificial-intelligence). I am currently conducting research at [SenseTime](https://www.sensetime.com/en) and [MMLab@NTU](https://www.mmlab-ntu.com/). I am fortunate to be supervised by [Prof. Ziwei Liu](https://liuziwei7.github.io/), I am also grateful for the extensive help I have received from [Bo Li](https://brianboli.com/), [Yuanhan Zhang](https://zhangyuanhan-ai.github.io/), [Zhongang Cai](https://caizhongang.com/) and [Lei Yang](https://scholar.google.com/citations?user=jZH2IPYAAAAJ). My research interests center on multimodal models. During my time at [Shaoxing No.1 High School](http://www.sxyz.net/), I participated in the [Olympiad in Informatics](https://www.noi.cn/). After graduating, I spent a rewarding year at the [International School of Information Science & Engineering](https://drise.dlut.edu.cn/) at [Dalian University of Technology](https://www.dlut.edu.cn/). During the COVID-19 pandemic, I chose to withdraw and begin my undergraduate studies anew at Nanyang Technological University. I competed in the [ICPC](https://icpc.global/) at both universities. I also contribute to the [Nanyang Programming Contest](https://ntu-icpc.github.io/npc/). I am grateful to [my teammates and friends](https://pufanyi.com/icpc) for their support! 🙏 I enjoy playing chess. My Lichess username is [@pufanyi](https://lichess.org/@/pufanyi), and my current rapid rating is 2086. ## Education ### University of Wisconsin–Madison Sep 2026 – Present Ph.D. in Computer Science (Incoming) Madison, WI ### Nanyang Technological University Aug 2022 – Jul 2026 Bachelor of Science in Data Science and Artificial Intelligence Singapore - Honours (Highest Distinction), CGPA: 4.67 / 5.00 - Research interests: Multimodal and Video Generation. 1,800+ Google Scholar citations - Selected coursework: Reinforcement Learning, Deep Learning, Data Structures and Algorithms, Probability and Statistics, Computational Economics, Stochastic Processes, Numerical Analysis, Cryptography, Electricity & Magnetism ### University of California, Berkeley Jun 2024 – Aug 2024 Summer Session Berkeley, CA - Studied Computer Security and Game Theory, GPA: 4.00 / 4.00 ## Publications & Research ### LMMs-Engine: A Simple, Unified Multimodal Models Training Engine Aug 2025 – Present Research Project, Core Developer - [GitHub](https://github.com/EvolvingLMMs-Lab/lmms-engine) * A lean and flexible training framework designed for both rapid research prototyping and large-scale production * Optimized the Bagel training pipeline by integrating FSDP2 and Liger Kernel for efficient distributed training * Designing asynchronous reinforcement-learning infrastructure for the project ### LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models Jan 2024 – Present NAACL 2025 (Findings), Co-first author - [Paper](https://arxiv.org/abs/2407.12772) - [Project Page](https://lmms-lab.github.io/lmms-eval-blog/lmms-eval-0.1/) * An open-source multimodal evaluation framework with **4.3K GitHub stars** * Built the evaluation pipeline and developed a low-cost automatic data-generation pipeline for Multi-modal LiveBench, which leverages continuously updated news and online forums to evaluate models' generalization in the wild ### Mage-Flow: An Efficient Native-Resolution Foundation Model for Image Generation and Editing Jan 2026 – Apr 2026 Technical Report (Microsoft Research), Contributor - [Paper](https://arxiv.org/abs/2607.19064) - [Project Page](https://microsoft.github.io/Mage/flow/) * An efficient native-resolution foundation model for high-quality image generation and editing * Extended LMMs-Engine for Mage-Flow pre-training by integrating FSDP2, Liger Kernel, and sequence packing, enabling efficient native multi-resolution training across heterogeneous image resolutions and aspect ratios ### Demystifying Video Reasoning Jan 2026 – Mar 2026 ECCV 2026, Third Author - [Paper](https://arxiv.org/abs/2603.16870) - [Project Page](https://www.wruisi.com/demystifying_video_reasoning/) * Revisited video reasoning and proposed Chain-of-Step, in which models reason through diffusion steps * Curated a large collection of examples to validate this hypothesis and systematically analyzed models' reasoning behaviors ### SenseNova-SI: Scaling Spatial Intelligence with Multimodal Foundation Models Sep 2025 – Jan 2026 CVPR 2026, Co-first author - [Paper](https://openaccess.thecvf.com/content/CVPR2026/papers/Cai_Scaling_Spatial_Intelligence_with_Multimodal_Foundation_Models_CVPR_2026_paper.pdf) - [GitHub](https://github.com/OpenSenseNova/SenseNova-SI) * Developed a foundation model for scaling spatial intelligence, achieving state-of-the-art performance on key spatial benchmarks * Conducted rigorous ablation studies to establish that spatial intelligence is primarily driven by data scaling rather than reasoning heuristics (CoT), guiding the project's strategic focus on constructing the SenseNova-SI-8M dataset * Executed the full-stack training pipeline for Qwen-based variants with LMMs-Engine on 128 GPUs ### Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videos Sep 2024 – Feb 2025 ACL 2026, Third Author - [Paper](https://arxiv.org/abs/2410.13764) - [Project Page](https://video-mmmu.github.io/) * Built an evaluation set with 300 expert-level videos and 900 human-annotated questions across six disciplines, cited by [Gemini 3 Pro](https://deepmind.google/models/gemini/pro/) (Google) and [GPT-5](https://openai.com/index/introducing-gpt-5/) (OpenAI) * Proposed a knowledge-gain metric to quantify performance improvements after watching video lectures; validated expert-level data in economics and medicine to ensure benchmark quality ### Multi-Modal In-Context Instruction Tuning (Otter, MIMIC-IT) Jun 2023 – Dec 2023 IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Co-first author - [Paper](https://arxiv.org/abs/2306.05425) - [Project Page](https://github.com/Luodian/Otter) * Developed an early vision-language-agent (VLA) model with **3.3K GitHub stars** * Developed a language-model-based pipeline that generated 2.8M multimodal instruction-tuning samples; created an evaluation framework that later evolved into LMMs-Eval ## Research & Professional Experience ### LMMs-Lab Jan 2024 – Present Core Member Singapore - Core member of a non-profit initiative dedicated to democratizing Large Multimodal Models (LMMs); instrumental in the lab's 0-to-1 establishment, from initial ideation and naming to its current operations - Spearheaded the development of high-impact open-source projects: the evaluation framework LMMs-Eval, the efficient training library LMMs-Engine, and the video understanding benchmark Video-MMMU ### SenseTime Research Jul 2025 – Jun 2026 Research Intern (Spatial Intelligence and Video Generation) Singapore - Conducted research on SenseNova-SI and video reasoning, with a focus on spatial intelligence and video generation ### Synvo AI Jan 2025 – Jul 2025 Core Contributor Singapore - Architected and implemented the Synvo File System, the company's inaugural product, serving as the storage backbone for unstructured multimodal data - Empowered the core contextualization engine by structuring data for efficient retrieval, enabling the development of AI systems that can remember, learn, and adapt ### MMLab\@NTU Jan 2025 – May 2025 Research Intern (Multimodal Models) Singapore - Supervised by [Prof. Liu Ziwei](https://liuziwei7.github.io/); focused on multimodal language models and unified multimodal models ## Competitions ### The International Collegiate Programming Contest (ICPC) 2021 – 2024 Nanyang Technological University / Dalian University of Technology Singapore / China - **Ranked 22nd** in the 2024 ICPC Asia Pacific Championship - **Ranked 13th** in the 2023 ICPC Asia Jakarta Regional Contest - **Ranked 6th** in the 2022 ICPC Asia Manila Regional Contest - **Gold Medal** in the 2021 ICPC Asia Kunming Regional Contest - **Silver Medal** in the 2021 ICPC Asia Nanjing Regional Contest ### Simon Marais Mathematics Competition Dec 2022 Nanyang Technological University Singapore - Received the Best-in-University Prize at Nanyang Technological University - Solved problems in number theory, game theory, and calculus ## Teaching & Activities ### Tutorial Lecturer Jan 2026 – Jun 2026 NTU SC1008 C & C++ Programming Singapore - Instructed a cohort of 50 beginners in core C/C++ syntax and methodologies ### Nanyang Programming Contest Jan 2025 – Jun 2026 Nanyang Technological University Singapore - Organized five competitions and tutorials for non-ICPC students - Designed activities to help students strengthen algorithmic problem-solving skills for technical interviews ### NTU Students' Computing and Data Science Club Nov 2024 – Jun 2026 Nanyang Technological University Singapore - Managed a Telegram group with 130+ members - Answered AI/ML questions and shared resources on deep learning research ### Teaching Assistant Aug 2024 – Dec 2024 NTU SC1003 Introduction of Computational Thinking Singapore - Instructed a cohort of 20 beginners in core Python syntax and data manipulation methodologies ## Interests Physics, Chess ([Lichess](https://lichess.org/@/pufanyi) rating: 2084), Calligraphy, Table Tennis