Pixel Cube: Diffusion-based Portrait Video Relighting Through Realistic Lighting Reproduction

We present a diffusion-based method for photorealistic and temporally consistent portrait video relighting. Our model is trained on a hybrid dataset of real-captured and rendered dynamic portraits with diverse appearances, motions, poses, and known lighting conditions. To acquire realistic training data, we build an LED-based lighting system for high-speed video capture under controllable illumination. Using HDR environment maps and synthesized background images as controls, our model relights in-the-wild portrait videos while preserving identity, expressions, skin tone, wrinkles, and facial hair. Experiments show that our method generalizes well to unseen subjects, motions, and lighting, achieving high-quality results in realism, lighting harmony, and temporal consistency.