[Back to Home]
IEEE TNNLS 2015

Multiple Representations-Based Face Sketch-Photo Synthesis

Chunlei Peng¹ Xinbo Gao¹ Nannan Wang¹ Dacheng Tao² Xuelong Li³ Jie Li¹

¹Xidian University ²University of Technology, Sydney ³Xi'an Institute of Optics and Precision Mechanics

TNNLS2015_MrFSPS

Abstract

Face sketch-photo synthesis plays an important role in law enforcement and digital entertainment. Most of the existing methods only use pixel intensities as the feature. Since face images can be described using features from multiple aspects, this paper presents a novel multiple representations based face sketch-photo synthesis method that adaptively combines multiple representations to represent an image patch. Specifically, it combines multiple features from face images processed using multiple filters and deploys Markov networks to exploit the interacting relationships between neighboring image patches. The proposed framework could be solved using an alternating optimization strategy and it normally converges in only five outer iterations in the experiments. Our experimental results on the Chinese University of Hong Kong face sketch database, celebrity photos, CUHK Face Sketch FERET Database, IIIT-D Viewed Sketch Database, and forensic sketches demonstrate the effectiveness of our method for face sketch-photo synthesis. In addition, cross-database and database-dependent style synthesis evaluations demonstrate the generalizability of this novel method and suggest promising solutions for face identification in forensic science.

Contribution Highlights

Multiple representations are used to describe a face image patch in the face sketch-photo synthesis problem

An effective and efficient framework based on Markov networks is proposed to adaptively learn the combination weights of multiple representations

A database-dependent style synthesis strategy is designed, which achieves promising performance on forensic sketch-photo synthesis and recognition

Perceptive and quantitative experiments are used to illustrate the effectiveness of the proposed method

Experimental Results

Results of All [Link];

Results on Sketch Synthesis [Link];

Results on Photo Synthesis [Link];

Results on Celebrities and Photos under Different Lighting [Link];

Results on Forensic Sketches and Mugshot Photos [Link];

Results on Cross Database Evaluation (Available by request);

Citation

C. Peng, X. Gao, N. Wang, D. Tao, X. Li, and J. Li. Multiple Representations-Based Face Sketch-Photo Synthesis. IEEE Transactions on Neural Networks and Learning Systems, 2015. [PDF]