Visual Search: examining relationship between cropping and image size for limited-screen devices

1. Visual Search: examining relationship between cropping and image size for limited-screen devices Paul Lin Andres Odio

2. Idea • For browsers with limited screen space, how you display images matter • Browsing by small multiple thumbnails is a good start, but what happens when the screen space is extremely limited (such as on a PDA or mobile phone)? • How do you create effective visual search tools on these devices? What variables are involved? What are the relationships between the variables? How do you optimize these variables to create the fastest and most enjoyable browsing experience?

3. Cropping • Automatic thumbnail cropping and its effectiveness (Suh, Ling, Bederson, 2003) • As long as the crop is “salient” • cropping is more effective regardless of technique and cropping size • cropping is more effective regardless of content (18% for animal images, 24% for completely random Corbis images) • So to create the most effective display method, we must optimize the relationship between cropping, image size and the number of images per page • But... • How does cropping relate to the size of the images, and the number of images shown for each page? • What is the relationship between cropped, resized and number of images on the screen? • How does cropping affect the user experience, and their level of enjoyability?

4. Hypothesis • Rate of searching through items should be consistent for increasing number of items • However, we believe that for a large number of images, cropping is more effective. As the number of images decreases, cropping becomes more effective, because you can use extraneous details considered “noise” for comparison • Thus, we expect results to be:

5. Experiment Design (1) • Implementation • Created in C# • Image search program • 320 x 480, to simulate the fixed screen size of a typical Palm PDA

6. Experiment Design (2) • Content • domain consistency - women’s clothing only • size consistency - upper body items (jackets, shirts, sweaters) only • image quality consistency - from Banana Republic only • Cropping mechanism • “salient” cropping • 33% of original area crop

7. Experiment Design (3)

8. Experiment Design (4) • 12 users • 6 sets of searches/user • 3x3 (9 images) per page, resized • 3x3 (9 images) per page, cropped • 5x5 (25 images) per page, resized • 5x5 (25 images) per page, cropped • 7x7 (49 images) per page, resized • 7x7 (49 images) per page, cropped • 15 searches/set • 90 searches/user • 190 searches/type of set • 1080 data points total • 148 images different used, with images randomized after each set • Sequence of sets randomized for each user

9. Results • Analysed using SigmaPlot 9.0 • Organized results by sets • Reject outliers by mean  2 standard deviations (95% confidence interval) • Plotted results to histogram to check for distribution, using SigmaPlot 9.0’s automatic binning • Distribution appears to be exponential decay (y=ax-bx) • Best representation of the “mean” for exponential decay is 1/decay constant, or 1/b • Lines were fitted to histograms to obtain equation, with all equations satisfying > 90% confidence

10. 3×3 Full 5×5 Full 7×7 Full 3×3 Cropped 5×5 Cropped 7×7 Cropped 21.93s 17.24s 23.53s 19.16s 12.51s 22.52s

11. Results(2) • 5x5 is the best performing size, for both cropped and resized • Cropping is always faster and more efficient than resized • On average, cropping is 18.92% faster than resized images, which confirms Suh, Ling and Bederson’s results of 18-24% performance improvement

12. Quantitative Analysis • Questions: • Why is the graph V shaped? • Why is 3x3 almost as slow as 7x7, for both cropped and resized? • Why is 5x5 the fastest? • If it’s the number of images, then it 7x7 should be faster… • If it’s the size of the images, then 3x3 should be faster…

13. Quantitative Analysis(2) • New hypothesis: • There is a third variable – number of pages • For fixed screen size devices, with screen size kept constant… • number of pages is inversely proportional to number of images • number of pages is proportional to the size of image • The number of pages, and the time spent travelling through them, is as significant as the image size and number of images for overall result

14. Quantitative Analysis(3) • So if… • overall result = f(g), • number of page transitions = f(h), • time constant for time spent on each page transition = k, • ideal graph where you have limitless screen area, where page transition is not a factor = f(i), • Then f(i) = f(g) – f(h).k

15. Quantitative Analysis(4) • Graphed the number of page transitions for each image search (forwards, backwards) • Again, organized results by sets, and rejected outliers by mean  2 standard deviations (95% confidence interval) • The graph is f(h), and for the constant k, we chose a k of 2 seconds

16. Quantitative Analysis(5) • f(i) = f(g) – f(h).k • Result = linear graph, where difficulty of finding image increases with the number of images • Shows that if no page transitions are involved, then 3x3 should be a lot more efficient than 7x7 • Similar to our hypothesis, except that cropped is always better than resized • Shows that page transitions has a huge effect on the efficiency of visual search

17. Qualitative Analysis of User Experience • User survey conducted for each user • Interesting results: • 75% of users thought that they performed better for resized images, when numbers showed that cropped were almost always faster • Only 1 user thought that 5x5 cropped was the fastest • 67% of the users enjoyed full sized images more than cropped

18. Conclusion • Assuming that the design of the system doesn’t require page transitions, then increasing number of images while decreasing in size will also increase difficulty • However, in real life, with screens being of fixed sizes, multiple pages are necessary for large numbers of images • Page transitions severely affects the overall image search times for larger images, and less for smaller images. • Thus, to calculate optimal display methods, one must find the ideal equation, then modify it with the predicted amount of time spent on page transitions. This amount of time will depend on the physical screen size. • Cropping is always faster than resized images, but it is also a less enjoyable experience.

19. Questions?