grocery shopping assistant l.
Skip this Video
Loading SlideShow in 5 Seconds..
Grocery Shopping Assistant PowerPoint Presentation
Download Presentation
Grocery Shopping Assistant

Loading in 2 Seconds...

  share
play fullscreen
1 / 20
Download Presentation

Grocery Shopping Assistant - PowerPoint PPT Presentation

emily
389 Views
Download Presentation

Grocery Shopping Assistant

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Grocery Shopping Assistant Carolina Galleguillos Pixel-café / June 2 2006

  2. Description GroZi project (grocery shopping assistant) • Increase independence of people with low vision (specially blind) to perform grocery shopping in a supermarket or store. • Help to plan shopping list, walking path to the store and grocery shopping.

  3. Motivation • 1.3 million legally blind people in the U.S • Grocery store are underselling to this market. • Blind people are “high cost” customers. • Advance research on object recognition for mobile robotics with constrained computing resources.

  4. Motivation Characteristics Grocery Store: • Structured Environment (+). • Controlled Lightening (+). • Maintained by staff (+). • Well indexed (+). • People moving around aisles (-). • Huge amount of products (30K) (-).

  5. Motivation Possible existing solutions: • Seeing-eye dog trained. • RFID tags (aisle, shelf, product). • Barcode scanning (shelf). • Help of sighted guide/customer service. • Memorize store layout. • Home delivery.

  6. Why computer vision? • Limited ability of dogs. • RFID tags bring privacy concerns and heavy infrastructure. • Eye safety and mislabeling. • Independence. • Store layout changes constantly. • Autonomy.

  7. Our Solution • Develop a handheld device that performs visual object recognition with haptic feedback. • Avail of complementary resources (RFID, Barcode scan, sighted guide) We are focusing on the computer vision aspects of this problem.

  8. MoZi Box General purpose low-cost mobile system geared for computer vision applications. MoZi is a combination of the Mobile Vision System (MoVs) and ZigZag • Finite memory : Compact Flash (CF) cards ranging from 256 MB to 4 GB. • Processor speed: in the neighborhood of 60-400MHz • Frame rate: enough snapshots to cover the shelf with some overlap (as in panoramic stitching) (15fps instead of 30fps?). • Color Calibration: Macbeth color chart to calibrate the color space.

  9. Use of the System • Creating a Shopping List. • Getting to the Grocery Store. • Navigating the Store.

  10. Shopping List • Online Website: • Website stores data and images of different products. • Feedback from users. • Provides walking path. • Prepare shopping list. • Download information into Mozi Box.

  11. On the way Separate project. • Mozi Box with GPS. • Visual waypoints. • Traffic/Street sign reading. • Use in addition to cane and asking sighted bystanders.

  12. Inside the Store • Finding aisle (OCR, RFID, ask). • Avoiding obstacles (cane). • Finding products (sweep of aisle, spot product, barcode check). • Checking out (coupon and cash).

  13. Obtaining training data • Online Images (Web). • Collecting from MoZi box (in situ). • Collecting from embedded camera near the barcode scanner (in situ). • Known databases (COIL-100,ETH-80, etc.)(more research oriented) • Synthetic examples. • Active learning.

  14. Obtaining training data • Active learning problem: Find UPC for the corresponding image (labeling). • Semi-supervised. • Weakly labeled.

  15. Obtaining training data • Sunshine Store @ UCSD. • Venue for pilot study. • 4K items in stock. • 1749 sq. ft. (assignable) • We want to scale to a bigger number of products (30K). • No bakery or vegetables.

  16. Object Recognition 2 types of recognition (m:n, m<<n): • Detection (of objects). • Verification (objects detected are in that list). Algorithms: SIFT, AdaBoost cascade, Multiclass Adaboost, Probabilistic Boosting tree, Color histogram matching, etc.

  17. Text Detection • Standard OCR is unlikely to be sufficient. • Low resolution and distortion are main problems. • Reading aisle signs, text on shelves.

  18. Considerations • Occlusion and clutter of products (caused by people and shopping carts). • Multiple images of same shelf to perform “hole-fill-in”. • Cannot fit dominant plane to the front of product shelves. • Large number of items.

  19. Acknowledgements People (UCSD/Calit2): • Serge Belongie • John Miller • Stephan Steinbach • Michele Merler • Tom Duerig Captions: Dennis Metz/ D. Stein [X. Chen and A. Yuille ]

  20. Questions? Comments?