Ruofan Liu

Ruofan Liu

Ph.D. candidate in Computer Science, Institute of Science Tokyo

RUOFAN LIU - About Me

I am a Ph.D. 3rd year student (2023.10~) at the Deparment of Computer Science, Institute of Science Tokyo, supervised by Prof. Hideki Koike. I am currently a visiting researcher at Stanford Computational Imaging Lab, Stanford University, supervised by Prof. Gordon Wetzstein, and a research assistant at Sony Computer Science Laboratories, supervised by Prof. Shinichi Furuya.

I obtained my M.E. in Computer Science at the Institute of Science Tokyo with Prof. Hideki Koike, and obtained my B.E. in Computer Science and Technology at Shanghai Jiao Tong University with Prof. Baoliang Lu.

My research lies at the intersection of computer vision, multimodal generation for content creation, foundation models, and interactive systems. My ultimate research goal is to create a next-generation deep learning framework and a generative interaction model. I believe that the critical path toward achieving this involves developing cross-modal synthesis and large-scale learning methods that are more generalizable and explainable. To this end, I have been focusing on the following topics:

  • Generative AI, Foundation Model, and Content Creation: Video generation model post-training for content creation (coming soon), Audio-to-motion generation based on DiT (under review for CVPR '26), VLM-based garment generation from a single image (under review for CVPR '26)
  • Cross-modal Synthesis and Computational Sensing: Multimodal learning for pose-to-EMG estimation (NeurIPS '25), VQ-VAE-based cross-modal synthesis (CHI '25), Hand muscle electromyography inference (SA '24)
  • Interative System and XR Prototype: Embodied and detached golf muscle training in AR (under review for IEEE VR '26), Aligment-based piano AR prototype (ISMAR '23), Alignment-based hand pose discrepancy visualization (CHI '23)

Beyond research, I enjoy go-karting, piano playing, snowboarding, indoor bouldering, traveling, and walking friends’ dogs.

Research Interests

Generative AI, Multimodal Learning for Content Creation, Large Foundation Model, Cross-Modal Synthesis, Human-Computer Interaction

For more info

More detailed info can be found in CV and Publications. Feel free to DM me using LinkedIn! Happy to have any chats!