All Smiles : Automatic Photo Enhancement by Facial Expression Analysis
Rajvi Shah and Vivek Kwatra
1CVIT, IIIT Hyderabad, India,  2Google Research, USA 
CVMP 2012  [ Best Paper Award ] 
Now a Part of Google+ Auto Awesome Smile Feature
We propose a framework for automatic enhancement of group photographs by facial expression analysis. We are motivated by the observation that group photographs are seldom perfect. Subjects may have inadvertently closed their eyes, may be looking away, or may not be smiling at that moment. Given a set of photographs of the same group of people, our algorithm uses facial analysis to determine a goodness score for each face instance in those photos. This scoring function is based on classifiers for facial expressions such as smiles and eye-closure, trained over a large set of annotated photos. Given these scores, a best composite for the set is synthesized by (a) selecting the photo with the best overall score, and (b) replacing any low-scoring faces in that photo with high-scoring faces of the same person from other photos, using alignment and seamless composition. For more details please read our paper.
Publications
Rajvi Shah and Vivek Kwatra. "All smiles: automatic photo enhancement by facial expression analysis". In Proceedings of the 9th European Conference on Visual Media Production (CVMP '12). ACM, New York, NY, USA, 1-10.
| PDF | PPTBibTex |