Rao FU

Computer Vision/Graphics/Robotics

 Hello,

a bit about me:

I'm a PhD student in Computer Science at Brown University. My research interests are in 3D Vision, Graphics, and Robotics. My research primarily explores:

  1. Effective representations and methods for controllable generative simulation.
  2. Representation and control techniques for hand-object interactions.
  3. 3D vision understanding and reasoning.
I'm fortunate to work with Prof. Srinath Sridhar and Prof. Daniel Ritchie at Brown IVL & Brown VC Group.

Previously, I have enriched my research experience as an intern at Microsoft Research Asia, Autodesk AI Research, and Meta GenAI.

Prior to this, I earned my Bachelor of Engineering degree from the University of the Chinese Academy of Sciences, advised by Prof. Lin Gao and Prof. Xilin Chen. I also had a wonderful time at the University of Southern California and the University of California San Diego, mentored by Prof. Hao Su.

Your Image

勿戳！
Don't poke my cheeks!
¡no me toques las mejillas!
मेरे गाल मत दबाओ!
আমার গালে খোঁচা দিও না!
não cutuque minhas bochechas!
не тыкай меня в щеки!
私の頬をつつかないで！
ਮੇਰੀਆਂ ਗੱਲ੍ਹਾਂ ਨੂੰ ਨਾ ਮਾਰੋ!
唔好戳我嘅脸颊！
đừng chọc má tôi!
నా బుగ్గలు కుట్టకు!
yanaklarımı dürtme!
내 뺨을 찌르지 마!
ne touchez pas mes joues!
stoß mir nicht in die Wangen!
என் கன்னங்களை குத்தாதே!
میرے گالوں کو مت مارو!
ojo nyocot pipiku!
non toccarmi le guance!
لا تكز خدي!
and more about me...
email_button

Email to
Say Hi

cv_button

CV/
Resume

google_button

Google
Scholar

lkin_icon

LinkedIn
Profile

ucas_button

B.E.@
UCAS

brown_button

Ph.D.@
Brown

cat_button

My Cat
Inocchi

art_button

Arts
Todo...
Welcome to Reach Out to Me!
If you are interested in working with me, please fill out this form or email me!

News

  • [Dec 2023] Oral Presentation at NECV 2023. See you at Dartmouth!
  • [Aug 2023] I'm attending Siggraph 2023 for fun: See you in Los Angeles!
  • [June 2023]I'm excited to join Meta Research as a research scientist intern this summer : See you in Menlo Park!
Research Paper
 AnyHome: Open-Vocabulary Generation of Structured and Textured 3D Homes.
R. Fu*†, Z.H. When*, Z.C. Liu*, S. Sridhar. (†Corresponding Author)
We introduce AnyHome, a framework that translates open-vocabulary descriptions, ranging from simple labels to elaborate paragraphs, into well-structured and textured 3D indoor scenes at a house-scale.
[paper] [project]
CLIP-Sculptor: Zero-Shot Generation of High-Fidelity and Diverse Shapes from Natural Language.
A. Sanghi, R. Fu, V. Liu, K. Willis, H. Shayani, A.H. Khasahmadi, S. Sridhar, D. Ritchie.
CVPR 2023
We introduce CLIP-Sculptor, a method to address these limitations by producing high-fidelity and diverse 3D shapes without the need for (text, shape) pairs for training.
[paper] [project]
ShapeCrafter: A Recursive Text-Conditioned 3D Shape Generation Model.
R. Fu, X. Zhan, Y.W. Chen, D. Ritchie, S. Sridhar.
NeurIPS 2022
We present ShapeCrafter, a neural network for recursive text-conditioned 3D shape generation. Our method supports shape editing, extrapolation, and can enable new applications in human-machine collaboration for creative design.
[paper] [project]
HRFormer: High-Resolution Transformer for Dense Prediction.
Y.H. Yuan, R. Fu, L. Huang, W.H. Lin, C. Zhang, X.L. Chen, J.D. Wang.
NeurIPS 2021
We present a High-Resolution Transformer (HRFormer) that learns high-resolution representations for dense prediction tasks. We demonstrate the effectiveness of the HRFormer on both human pose estimation and semantic segmentation tasks.
[paper] [project]
NeuralODF: Learning Omnidirectional Distance Fields for 3D Shape Representation.
T. Houchens, C.Y. Lu, S. Duggal, R. Fu, S. Srinath.
Technical Report
We propose Omnidirectional Distance Fields (ODFs), a new 3D shape representation that encodes geometry by storing the distance to the object’s surface from any 3D position in any viewing direction. We also introduce efficient forward mapping algorithms for transforming ODFs to and from common 3D representations.
[paper]
ROSAN: Rotation-Robust Structure-Aware Network for Fine-Grained 3D Shape Retrieval.
R. Fu, Y.C. Zhang, J. Yang, J.W. Sun, F.L. Zhang, Y.K. Lai, L. Gao.
CVM 2024
We introduce ROSAN, which learns rotation robust 3D shape descriptors and achieve accurate results on the task of fine-grained 3D object retrieval. We also build and publish a new 3D shape dataset for fine-grained 3D shape retrieval.
[paper] [project] [dataset]
Education
BROWN UNIVERSITY
Sept.2021 – Present
  • Title: Computer Science Ph.D. Student
  • Concentration: Vision, Graphics and Robotics
UNIVERSITY OF CHINESE ACADEMY OF SCIENCES
Sept. 2017-Jul. 2021
  • Degree: Bachelor Engineering in Computer Science.
  • Outstanding Thesis Awards.
  • National Inspirational Scholarship(5%).
UNIVERSITY OF CALIFORNIA, SAN DIEGO
May. 2020-Aug. 2020
  • Title: Visiting Scholar.
UNIVERSITY OF SOUTHERN CALIFORNIA
Jan. 2020-May. 2020
  • Title: Visiting Student.
BEIJING NATIONAL DAY SCHOOL
Sept. 2011-Jul. 2017
  • Title: Student.
Employment
Research intern: Meta
May. 2023 – Sept. 2023
Group: GenAI
Menlo Park, CA, US
  • Research on Large 3D Visual-Language Model.
Research intern: Autodesk Inc.
May. 2022 – Dec. 2022
Group: AI Lab
Toronto, Canada
  • Research on Lauguage-guided 3D Shape Generation.
Research intern: Microsoft Research, Asia
Mar. 2021 – July. 2021
Group: Speech Group, Visual Computing Group
Beijing, China
  • Research on High-Resolution Machine Perception Backbone
Research intern: University of California, San Diego
May. 2020 – Nov. 2020
Advisor: Prof. Hao Su
San Diego, CA, USA
  • Research on Robot Grasping with Two-Finger Grippers and Humanoid Robot Grippers.
Research intern: Institute of Computing Technology, CAS.
Jul. 2019 – Nov. 2020
Advisor: Prof. Lin Gao
Beijing, China
  • Research on Fine-grained 3D Shape Retrieval.
  • Research on Talking Head Generation.

Buy me a Boba:

If you like my homepage and think it's useful, welcome to buy me a Boba via Wechat or Paypal down here:
© 2023 by Rao FU