Dataset: 3581 images from Photo [Datta et al., 2006] Aesthetics scoring scale 1-7

1. Apply N. Bayes Pick K images from top where H = +1, and Apply N. Bayes Pick K images from top where H = -1, and Learning the Consensus on Visual Quality for Next-Generation Image Management Ritendra Datta, Jia Li, and James Z. Wang The Pennsylvania State University, University Park, PA, USA PROBLEM SPECIFICATION METHODOLOGY EXPERIMENTS • Dataset: 3581 images from Photo.net [Datta et al., 2006] • Aesthetics scoring scale 1-7 • We have averages scores and count nkof ratings for Ik • Characteristics of Dataset: • Results: • Formally, define consensus on quality for image Ik as • where qk,i is the ith user rating received • Actually, estimated over a finite set of ratings  • Useful for the average user – Our target audience • May not be useful for the outlier user – Personalized recommendations? • Let us have D visual features that correlate with quality • In the literature – [Datta et al., ECCV 2006] – 56 features + higher-order terms • Brightness, Contrast, DOF, Saturation, Region Composition, etc. • (A) Learn a Weighted Least Squares Linear Regressor: • Directly learn a mapping from features to consensus scores • Weights related to trust associated with score: • Greater the no. of samples n, better estimate is of • Formulation: • Convenient parameter estimation: • Here, and denotes pseudoinverse. • Inferencing based on • (B) Learn a Naïve Bayes Classifier: • Motivation: Non-overlapping performance advantages over regressor • By some threshold HIGH we can map  (binary) • Parameters to estimate: Pr(H = h) and Pr( Xk(d) | H = h) • Former estimated by counting, latter fitted with single-component Gaussian p.d.f. • Inferencing based on • Photo-sharing is getting popular • Dedicated Websites: • Flickr, Photobucket, Photo.net, etc. • Bundled with social networking sites: • Facebook, Orkut, MySpace, etc. • Personal Photo Collections are growing • Cheap digital cameras and storage • Organizing them can be time-consuming • Many methods for topic-based management • Image classification • Content-based image search/annotation • Collaborative tagging architectures • How about quality-based*photo management? • Can we distinguish these photos • (Rated over 6 out of 7 on average by many Photo.net users) • from these ones? • (Rated below 4 out of 7 on average by many Photo.net users) • Specifically, for a photo collection, can we • Select high-quality pictures? • As representatives, for front-page display • Enhanced quality-aware image search/browsing • Eliminate low-quality ones? • Cleaning up, limited space/slots (Orkut - 12 photos) • * What is Quality Here? • A measure of visual appeal (referred to as aesthetics • by our data-source Photo.net), as decided by consensus • over a population. On an average, picks 20 high-quality (HIGH=5.5) photos with ~82% precision, while SVM-based method gets less than 50% On an average, eliminates 50 low-quality (LOW=4.5) photos with ~65% precision, while wrongly eliminating only ~9% high-quality photos. The SVM figures are 43% and 28% resp. Selection/Elimination Sort Descending HIGH QUALITY CONCLUDING REMARKS Apply Regressor: Predict N scores • Encouraging results, moving toward real-world applicability. • Proposed features in [Datta et al. 2006] show more promise. • Weighting training data by confidence improves performance. Sort Ascending Collection of N images LOW QUALITY