Jinwei Gu

SenseBrain -- AI for Mobile Photography

Email: LastnameFirstname AT
CV   [Google Scholar]
SenseBrain aims to bring mobile photography to a new level with latest computational imaging and machine learning advances. We work closely with major image sensor manufacturer and smartphone OEMs on AI camera system integration. We provide turn-key solutions for all your mobile imaging needs for still photo and video quality enhancement. Our R&D capabilities spans customized ISP designs for novel 2D image sensor, iToF and dToF 3D imaging, and system-level joint optics-electronics-algorithm optimization.

Recent Research


Deep Camera Obscura: An Image Restoration Pipeline for Lenless Pinhole Photography[Paper]

Joshua, Huaijin, Shuai, Jinwei, and Suren. Arxiv 2021


Low-light Image and Video Enhancement Using Deep Learning: A Survey[Paper][Data+Code]

Chongyi, Chunle, Linghao, Jun, Ming-Ming, Jinwei and Chen Change. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021. (to appear)


iToF2dToF: A Robust and Flexible Representation for Data-driven Time-of-Flight Imaging[Project(Paper+Data+Code)]

Felipe, Huaijin, Mohit, Andreas, and Jinwei. IEEE Transactions on Computational Imaging, 2021. (to appear)


ReconfigISP: Reconfigurable Camera Image Processing Pipeline[Project (Paper+Code)]

Ke, Zexian, Yue, Cavan, and Jinwei. ICCV, 2021.


STAR: A Structure-Aware Lightweight Transformer for Real-time Image Enhancement [Paper]

Zhaoyang, Yitong, Jun, Xiaogang, Ping, and Jinwei ICCV, 2021.


Dynamic Quantization with Mixed Precision and Adaptive Resolution [Paper]

Zhaoyang, Wenqi, Jinwei, Xiaogang, and Ping ICML, 2021. (spotlight)


Removing Diffraction Image Artifacts in Under-Display Camera via Dynamic Skip Connection Network [Project/Code]

Ruicheng, Chongyi, Huaijin, Shuai, Cavan, and Jinwei CVPR, 2021.


GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolution [Project/Code]

Kelvin, Xintao, Xiangyu, Jinwei, and Cavan CVPR, 2021. (Oral)


Video Stitching for Linear Camera Arrays [Project]

Wei-Sheng, Orazio, Jinwei, Deqing, Ming-hsuan, and Jan. BMVC, 2019. (to appear)


NRMVS: Non-Rigid Multi-View Stereo [Arxiv][PDF][Supplementary Video][Project (coming soon)]

Matthias, Kihwan, Jinwei, Matthias, Charles, Marc, and Jan. Arxiv, 2019.


Neural RGB->D Sensing: Depth and Uncertainty from a Video Camera [Arxiv][Code]

Chao, Jinwei, Kihwan, Srinivas, and Jan. CVPR 2019 (Oral, Best Paper Finalist).


PlaneRCNN: 3D Plane Detection and Reconstruction from a Single Image[Arxiv] [Code]

Chen, Kihwan, Jinwei, Yasutaka, and Jan. CVPR 2019 (Oral).


Neural Inverse Rendering of an Indoor Scene from a Single Image[Arxiv][Project and Dataset]

Soumyadip, Jinwei, Kihwan, Guilin, David, and Jan. Arxiv 2019. ICCV 2019 (to appear)


Very Power Efficient Neural Time-of-Flight[Arxiv][PDF][Supplementary Video][Project (coming soon)]

Yan, Jimmy, Xuanye, Keyuan, and Jinwei. Arxiv 2019.


Context-Aware Synthesis and Placement of Object Instances[Arxiv][PDF][Supplementary Video][Code]

Donghoon, Sifei, Jinwei, Ming-Yu, Ming-Hsuan, and Jan. NIPS 2018.


Switchable Temporal Propagation Network [Arxiv][PDF][Supplementary Video][Applications]

Sifei, Jinwei, Shalini, Guangyu, Varun, Ming-Hsuan, and Jan. ECCV 2018.


Separating Reflection and Transmission Images in the Wild [Arxiv][PDF][Supplementary Video][Code]

Patrick, Orazio, Jinwei, and Jan. ECCV 2018.


MapNet: Geometry-Aware Learning of Maps for Camera Localization[PDF][Code][Project]

Samarth, Jinwei, Kihwan, James, and Jan. CVPR 2018 (spotlight).


Space-Time-Brightness Sampling Using an Adaptive Pixel-Wise Coded Exposure [PDF]

Hajime, Toshiki, Dengyu, and Jinwei. CVPR 2018, Computational Camera and Display Workshop.


Reblur2Deblur: Deblurring Videos via Self-Supervised Learning[PDF][Project]

Huaijin, Jinwei, Orazio, Ming-Yu, Ashok, and Jan. ICCP 2018.


Learning Affinity via Spatial Propagation Networks [PDF][Supplementary Video]

Sifei, Shalini, Jinwei, Guangyu, Ming-Hsuan,and Jan. NIPS 2017 (Accepted).


A Lightweight Approach for On-the-Fly Reflectance Estimation[PDF][Supplementary Video]

Kihwan, Jinwei, Stephen, Pavlo, Matthias, and Jan. ICCV 2017 Oral. arXiv:1705.07162


Dynamic Facial Analysis: From Bayesian Filtering to Recurrent Neural Networks [Project][PDF][Supplementary Video]

Jinwei, Xiaodong, Shalini and Jan. CVPR 2017


Polarimetric Multi-view Stereo [PDF][Supplementary Document][Video]

Zhaopeng, Jinwei, Boxin, Ping and Jan. CVPR 2017


Reconstructing Intensity Images from Binary Spatial Gradient Cameras [PDF][Supplementary Document][Video]

Suren, Orazio, Jinwei, Timo and Jan. CVPR EVW 2017 (Oral)


Deep Learning with Energy-efficient Binary Gradient Cameras [PDF]

Suren, Orazio, and Jinwei. arXiv:1612.00986, 2016


Video Stitching with Spatial-Temporal Content-Preserving Warping[PDF][Video Results]

Wei and Jinwei. CVPR 2015 Workshop on Computational Cameras and Displays


Previous Research


Exemplar-Based Method for Automatic Visual Editing and Retouching of Fine Art Reproduction [project]

Jun and Jinwei. CIC 2013


Learning Discriminative Illumination and Filters for BTF Classification[project] [database](with code)

Chao, Jinwei, and Gefei. CVPR 2013


What is the Space of Camera Spectral Sensitivities?[project] [database](with code)

Jun, Dengyu, Jinwei, and Sabine. IEEE WACV 2013


Classifying Raw Materials with Discriminative Illumination [project] [database]

Chao and Jinwei. CVPR 2012 (Oral)


Spectral Imaging Under Commonly Available Lighting [project]

Jun and Jinwei. CVPR 2012 Workshop on Computational Camera and Displays


Multiplexed Illumination for Scene Recovery in the Presence of Global Illumination[project]

Toshi, Jinwei, Mohit, and Shree. ICCV 2011 (Oral)


Video from a Single Exposure Coded Photograph using a Learned Over-Complete Dictionary[project]

Yas, Jinwei, Mohit, Tomoo, and Shree. ICCV 2011


Coded Rolling Shutter Photography: Flexible Space-Time Sampling [project]

Yas, Jinwei, Tomoo, and Shree. ICCP 2010


Measurement, Modeling, and Synthesis of Time-Varying Appearance of Natural Phenomena [PDF]

Jinwei Gu. PhD Thesis. Columbia University. March 2010


Removing Image Artifacts Due to Dirty Camera Lenses and Thin Occluders[project]

Jinwei, Ravi, Peter, and Shree. SIGGRAPH Asia 2009


Printing Spatially-Varying Reflectance [project]

Jinwei, Wojciech Matusik, Boris Ajdin, Jason Lawrence, Hendrik P. A. Lensch, Fabio Pellacini, and Szymon Rusinkiewicz. SIGGRAPH Asia 2009


Compressive Structured Light for Recovering Inhomogenous Participating Media [project](with code)

Jinwei, Ravi, Peter, Shree, and Eitan. ECCV 2008 (Oral)


Dirty Glass: Modeling and Rendering Contamination on Transparent Surfaces [project] [database] (with code)

Jinwei, Ravi, Peter, and Shree. EGSR 2007


Time-Varying Surface Appearance [project][database] (with code)

Jinwei, Chien-i Tu, Ravi, Peter, Wojciech, and Shree. SIGGRAPH 2006


Radiometric Calibration from a Single Image [project]

Jinwei, Stephen Lin, Shuntaro Yamazaki, and Harry Shum, CVPR 2004 (Oral)


Fingerprint Orientation Field Modeling and Its Applications [project]

with Jie Zhou, Chunyu Yang. CVPR 2003


Selected Publications (to be updated ...)

Computer Vision and Machine Learning

Computational Photography

Computer Graphics and Physics-based Computer Vision

Image Processing and Biometrics

Other Areas


A Short Bio

Since November 2018, I joined SenseBrain and built a R&D team focusing on AI-powered computational photography solutions on mobile phones. Before that, I was a senior research scientist in NVIDIA Research. From August 2010 to June 2013, I was an assistant professor at the Munsell Color Science Laboratory in Center for Imaging Science in Rochester Institute of Technology. From November 2013 to October 2015, I was a senior researcher in the media lab (New Jersey Office) at Futurewei Technologies (aka, Huawei R&D USA) in Bridgewater, NJ. I obtained a Ph.D. in Spring 2010 in the Computer Science Department of Columbia University with Prof. Shree Nayar, Prof. Peter Belhumeur, and Prof. Ravi Ramamoorthi, and I got a B.S. and M.S. (supervised by Prof. Jie Zhou) in the Department of Automation in Tsinghua University, China, in 2002 and 2005 respectively. My general research interests are computer vision, machine learning, and computational imaging. Here is a copy of my CV.


Valid HTML 4.01 Transitional