grocery shopping assistant l.
Skip this Video
Loading SlideShow in 5 Seconds..
Grocery Shopping Assistant PowerPoint Presentation
Download Presentation
Grocery Shopping Assistant

Loading in 2 Seconds...

play fullscreen
1 / 20

Grocery Shopping Assistant - PowerPoint PPT Presentation

  • Uploaded on

Grocery Shopping Assistant Carolina Galleguillos Pixel-café / June 2 2006 Description GroZi project (grocery shopping assistant) Increase independence of people with low vision (specially blind) to perform grocery shopping in a supermarket or store.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

Grocery Shopping Assistant

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
grocery shopping assistant

Grocery Shopping Assistant

Carolina Galleguillos

Pixel-café / June 2 2006


GroZi project (grocery shopping assistant)

  • Increase independence of people with low vision (specially blind) to perform grocery shopping in a supermarket or store.
  • Help to plan shopping list, walking path to the store and grocery shopping.
  • 1.3 million legally blind people in the U.S
  • Grocery store are underselling to this market.
  • Blind people are “high cost” customers.
  • Advance research on object recognition for mobile robotics with constrained computing resources.

Characteristics Grocery Store:

  • Structured Environment (+).
  • Controlled Lightening (+).
  • Maintained by staff (+).
  • Well indexed (+).
  • People moving around aisles (-).
  • Huge amount of products (30K) (-).

Possible existing solutions:

  • Seeing-eye dog trained.
  • RFID tags (aisle, shelf, product).
  • Barcode scanning (shelf).
  • Help of sighted guide/customer service.
  • Memorize store layout.
  • Home delivery.
why computer vision
Why computer vision?
  • Limited ability of dogs.
  • RFID tags bring privacy concerns and heavy infrastructure.
  • Eye safety and mislabeling.
  • Independence.
  • Store layout changes constantly.
  • Autonomy.
our solution
Our Solution
  • Develop a handheld device that performs visual object recognition with haptic feedback.
  • Avail of complementary resources (RFID, Barcode scan, sighted guide)

We are focusing on the computer vision aspects of this problem.

mozi box
MoZi Box

General purpose low-cost mobile system geared for computer vision applications. MoZi is a combination of the Mobile Vision System (MoVs) and ZigZag

  • Finite memory : Compact Flash (CF) cards ranging from 256 MB to 4 GB.
  • Processor speed: in the neighborhood of 60-400MHz
  • Frame rate: enough snapshots to cover the shelf with some overlap (as in panoramic stitching) (15fps instead of 30fps?).
  • Color Calibration: Macbeth color chart to calibrate the color space.
use of the system
Use of the System
  • Creating a Shopping List.
  • Getting to the Grocery Store.
  • Navigating the Store.
shopping list
Shopping List
  • Online Website:
    • Website stores data and images of different products.
    • Feedback from users.
    • Provides walking path.
  • Prepare shopping list.
  • Download information into Mozi Box.
on the way
On the way

Separate project.

  • Mozi Box with GPS.
  • Visual waypoints.
  • Traffic/Street sign reading.
  • Use in addition to cane and asking sighted bystanders.
inside the store
Inside the Store
  • Finding aisle (OCR, RFID, ask).
  • Avoiding obstacles (cane).
  • Finding products (sweep of aisle, spot product, barcode check).
  • Checking out (coupon and cash).
obtaining training data
Obtaining training data
  • Online Images (Web).
  • Collecting from MoZi box (in situ).
  • Collecting from embedded camera near the barcode scanner (in situ).
  • Known databases (COIL-100,ETH-80, etc.)(more research oriented)
  • Synthetic examples.
  • Active learning.
obtaining training data14
Obtaining training data
  • Active learning problem:

Find UPC for the corresponding image (labeling).

  • Semi-supervised.
  • Weakly labeled.
obtaining training data15
Obtaining training data
  • Sunshine Store @ UCSD.
    • Venue for pilot study.
    • 4K items in stock.
    • 1749 sq. ft. (assignable)
    • We want to scale to a bigger number of products (30K).
    • No bakery or vegetables.
object recognition
Object Recognition

2 types of recognition (m:n, m<<n):

  • Detection (of objects).
  • Verification (objects detected are in that list).


SIFT, AdaBoost cascade, Multiclass Adaboost, Probabilistic Boosting tree, Color histogram matching, etc.

text detection
Text Detection
  • Standard OCR is unlikely to be sufficient.
  • Low resolution and distortion are main problems.
  • Reading aisle signs, text on shelves.
  • Occlusion and clutter of products (caused by people and shopping carts).
  • Multiple images of same shelf to perform “hole-fill-in”.
  • Cannot fit dominant plane to the front of product shelves.
  • Large number of items.

People (UCSD/Calit2):

  • Serge Belongie
  • John Miller
  • Stephan Steinbach
  • Michele Merler
  • Tom Duerig


Dennis Metz/ D. Stein

[X. Chen and A. Yuille ]