[Back to Home]
IEEE TNNLS 2015

Multiple Representations-Based Face Sketch-Photo Synthesis


Chunlei Peng1      Xinbo Gao1      Nannan Wang1      Dacheng Tao2      Xuelong Li3      Jie Li1


1Xidian University
2University of Technology, Sydney
3Xi'an Institute of Optics and Precision Mechanics


TNNLS2015_MrFSPS


Abstract


      Face sketch-photo synthesis plays an important role in law enforcement and digital entertainment. Most of the existing methods only use pixel intensities as the feature. Since face images can be described using features from multiple aspects, this paper presents a novel multiple representations based face sketch-photo synthesis method that adaptively combines multiple representations to represent an image patch. Specifically, it combines multiple features from face images processed using multiple filters and deploys Markov networks to exploit the interacting relationships between neighboring image patches. The proposed framework could be solved using an alternating optimization strategy and it normally converges in only five outer iterations in the experiments. Our experimental results on the Chinese University of Hong Kong face sketch database, celebrity photos, CUHK Face Sketch FERET Database, IIIT-D Viewed Sketch Database, and forensic sketches demonstrate the effectiveness of our method for face sketch-photo synthesis. In addition, cross-database and database-dependent style synthesis evaluations demonstrate the generalizability of this novel method and suggest promising solutions for face identification in forensic science.



Contribution Highlights


  • Multiple representations are used to describe a face image patch in the face sketch-photo synthesis problem
  • An effective and efficient framework based on Markov networks is proposed to adaptively learn the combination weights of multiple representations
  • A database-dependent style synthesis strategy is designed, which achieves promising performance on forensic sketch-photo synthesis and recognition
  • Perceptive and quantitative experiments are used to illustrate the effectiveness of the proposed method



Experimental Results


  • Results of All [Link];
  • Results on Sketch Synthesis [Link];
  • Results on Photo Synthesis [Link];
  • Results on Celebrities and Photos under Different Lighting [Link];
  • Results on Forensic Sketches and Mugshot Photos [Link];
  • Results on Cross Database Evaluation (Available by request);

TNNLS2015_MrFSPS



Citation


  • C. Peng, X. Gao, N. Wang, D. Tao, X. Li, and J. Li. Multiple Representations-Based Face Sketch-Photo Synthesis. IEEE Transactions on Neural Networks and Learning Systems, 2015. [PDF]