1 / 9

Mining the Web for Multimedia Question Answering

This research focuses on leveraging web mining techniques for multimedia question answering (Q/A) to extract valuable information from diverse sources such as text, images, and tables. Our team at Carnegie Mellon University, led by Yiming Yang, explores various applications including student demographics, macro-economics, and international trends. We employ advanced tools like web crawlers, information extraction methods, and template-based reasoning to enhance the Q/A process. Our findings suggest that integrating multimedia forms significantly improves the quality and depth of answers to complex questions.

cindy
Download Presentation

Mining the Web for Multimedia Question Answering

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Mining the Web forMultimedia Question Answering Yiming Yang Language Technologies Institute & Computer Science Department Carnegie Mellon University

  2. Research Team • Yiming Yang (PI) • Jaime Carbonell (Co-PI) • Yan Liu (PhD student) • Bora Cenk Gazen (PhD student) • Fan Li (PhD student) • Shoubin Dong (Visiting researcher)

  3. A picture is worth a thousand of words Example 1. Student demographics • What are the race distribution of the international students in university X? • Is there an unexpected trend in the admission? • Do other universities have a similar trend recently?

  4. Tends in international student admission

  5. Example 2. Macro-economics • What is the overall trend in the balance of trade between the US and mainland China? • Do similar cases exist in recent history, and if so, between which countries? • Is there any unexpected trend in trading agricultural products in Asian/Pacific region? • …

  6. Technical Components • Templates defined for Q/A of certain types • Web crawling tools • Text categorization for relevant pages • HTML/XML parsing for tables & images • Information extraction techniques • Named Entities, tables, graphs & text • Templates connecting multi-media objects • Q/A mapping and reasoning • Collaborative filtering of frequent Q’s and A’s • Case-based reasoning across text, tables & graphs • Numerical analysis of graphical similarity (lines)

  7. Hypotheses • Answers combining text and pictures are beneficial for certain types of questions. • Valuable information in multi-media forms can be gathered from the Web. • Information extraction & comparative image analysis (graphs, curves) can be jointly used for cross-media data mining. • Template-based reasoning can serve as a vehicle for multi-media Q/A.

More Related