Ho Kei (Rex) Cheng
I am a Ph.D. candidate at the University of Illinois Urbana-Champaign, advised by
My research focuses on two directions: building long-term memory architectures to model temporal dynamics in videos, and developing generative models using flow matching. I have interned at Adobe Research, Kaiber, Sony AI, and FAIR/Meta MSL PAR.
Selected Research (hover over videos to play)
ICLR 2025
A unified model for detection, segmentation, and tracking of objects in images and video using text, exemplar, and visual prompts.
ICCV 2025
Provides straighter flows through condition-aware coupling of samples from the prior and data distributions, without the test-time degradation induced by naïve optimal transport.
CVPR 2025
Generates high-quality synchronized audio from video or text inputs, with an architecture that enables training on data from multiple sources even when some modalities are missing.
Click here to watch a fun video!
CVPR 2024 Highlight
ICCV 2023
Achieves open-world video segmentation by combining universal image segmentation with temporal propagation. Easy to extend.
ECCV 2022
Approaches video object segmentation from a memory perspective with a pipeline that effectively models both short-term and long-term dependencies.
Used by supervisely and Track-Anything.
CVPR 2021
Decouples interactive video segmentation into two components: single-frame interaction and temporal propagation, demonstrating significantly improved performance.
Used by Sieve.
CVPR 2020
An iterative refinement network that achieves high-quality 4K+ segmentation using only low-resolution training data (less than 500 pixels per side).
Work Experience
Contributed to video perception in SAM 3, focusing on efficient memory designs for object tracking.
Developed MMAudio, a multimodal flow-matching generative model, and trained it from scratch.
Researched controllable video generation and editing pipelines using diffusion models.
Invited Talks
- Object-Level Reasoning in Video Object Segmentation and Its Multimodal Applications, Twelve Labs 2024
- Segmenting Videos in the Open World, IBM Zurich, Accelerated Discovery 2023
- Large-Scale Decoupled Video Segmentation, Apple 2023
Open Source Tools
- av-benchmark: Evaluation suite for text-to-audio and video-to-audio generative models.
- nitrous-ema: Post-hoc EMA implementation for PyTorch with minimal overhead.
- vos-benchmark: Fast evaluation library for video object segmentation.
- shared-memory-tensor-dataset: Simple demo for shared-memory data loading with DDP processes.
Professional Activities
- Co-organized the 1st Workshop on Generative AI for Audio-Visual Content Creation (Gen4AVC) at ICCV 2025.
- Conference Reviewer: CVPR, ICCV, ECCV, NeurIPS, ICLR, ICML (Outstanding Reviewer '22), AAAI.
- Journal Reviewer: TMLR, IEEE TPAMI, IEEE TIP, IEEE TCSVT, Pattern Recognition.
Misc
- I was a proud member of the HKUST Robotics Team. A short clip.
-
I am generally interested in artificial intelligence. I believe in AGIs and has high hope for their potential to transform human civilization for the better.
- "Man is condemned to be free. Condemned, because he did not create himself, in other respect is free; because, once thrown into the world, he is responsible for everything he does."
- Look at this cat in HKUST. Another picture. Or this cat.
- "Ho Kei" (with the space) is my first name and "Cheng" is my last name. "Rex" is the commonly used "english name" that is not part of my legal name. This is common in Hong Kong.
