Rex Cheng

Ho Kei (Rex) Cheng

I am a Ph.D. candidate at the University of Illinois Urbana-Champaign, advised by Alexander Schwing. Before that, I was at The Hong Kong University of Science and Technology, advised by Yu-Wing Tai and Chi Keung Tang.

I am recently working on algorithms for videos, including segmentation, tracking, editing, and generation. I have interned at Adobe Research (open-world video segmentation) and Kaiber (diffusion models for videos). I am interning at Sony AI (multimodal generative models with flow matching) in Tokyo this summer.

[GitHub] | [Google Scholar] | [CV]

Research (hover over videos to play)

Putting the Object Back into Video Object Segmentation

Ho Kei Cheng, Seoung Wug Oh, Brian Price, Joon-Young Lee, Alexander Schwing.

CVPR 2024 Highlight

Project page / code / arXiv / pdf

Equipped with an object transformer that integrates pixel-level features and object-level features for efficient and robust video object segmentation in challenging scenarios. Used by iMotions and Annolid.

Tracking Anything with Decoupled Video Segmentation

Ho Kei Cheng, Seoung Wug Oh, Brian Price, Alexander Schwing, Joon-Young Lee.

ICCV 2023

Project page / code / arXiv / pdf

Open-world video segmentation achieved by combining universal image segmentation with temporal propagation. Easy to extend.

XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Model

Ho Kei Cheng, Alexander Schwing.

ECCV 2022

Project page / code / arXiv / pdf

We look at video object segmentation from a memory perspective and design a pipeline that models both short-term and long-term dependencies effectively. Used by supervisely and Track-Anything.

Rethinking Space-Time Networks with Improved Memory Coverage for Efficient Video Object Segmentation

Ho Kei Cheng, Yu-Wing Tai, Chi Keung Tang.

NeurIPS 2021

Project page / code / arXiv / pdf

We devise a new, simple, and effective way of modeling correspondences between pixels from different frames. Used by Trioscope and BURST.

Modular Interactive Video Object Segmentation: Interaction-to-Mask, Propagation and Difference-Aware Fusion

Ho Kei Cheng, Yu-Wing Tai, Chi Keung Tang.

CVPR 2021

Project page / code / arXiv / pdf

We decouple the problem of interactive video segmentation into single-frame interaction and temporal propagation, showing that this works better by a large margin. Used by Sieve.

CascadePSP: Toward Class-Agnostic and Very High-Resolution Segmentation via Global and Local Refinement

Ho Kei Cheng*, Jihoon Chung*, Yu-Wing Tai, Chi Keung Tang.

CVPR 2020

Project page / code / arXiv / pdf / pypi

We train an iterative refinement network that generalizes to high-quality/high-resolution (4K+) segmentation with just low-resolution (<500 pixels per side) data.

Invited Talks

Object-Level Reasoning in Video Object Segmentation and Its Multimodal Applications @ Twelve Labs, September 2024
Segmenting Videos in the Open World @ IBM Zurich, Accelerated Discovery, September 2023
Large-Scale Decoupled Video Segmentation @ Apple, September 2023

Tools

nitrous-ema Low overhead post-hoc EMA for PyTorch.

vos-benchmark Fast and simple benchmarking for video object segmentation.

shared-memory-tensor-dataset A simple demo for sharing an in-memory dataset among DDP processes.

Professional Activities

Reviewer: CVPR, ICCV, ECCV, NeurIPS, ICLR, ICML, AAAI, IEEE TIP, IEEE PR, IEEE TPAMI, IEEE TCSVT.
Outstanding reviewer in ICML 2022.

Misc

I was a proud member of the HKUST Robotics Team. A short clip.
I am generally interested in artificial intelligence. I believe in AGIs and has high hope for their potential to transform human civilization for the better.
"Man is condemned to be free. Condemned, because he did not create himself, in other respect is free; because, once thrown into the world, he is responsible for everything he does."
Look at this cat in HKUST. Another picture. Or this cat.
"Ho Kei" (with the space) is my first name and "Cheng" is my last name. "Rex" is the commonly used "english name" that is not part of my legal name. This is common in Hong Kong.