1 / 12

Global Discovery: Turning Vision into Reality

Global Discovery: Turning Vision into Reality. Presented by Abe Lederman, President and CTO Deep Web Technologies, LLC Symposium: Global Discovery on the Internet: A Grand Challenge.

johnsonb
Download Presentation

Global Discovery: Turning Vision into Reality

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Global Discovery: Turning Vision into Reality Presented by Abe Lederman, President and CTO Deep Web Technologies, LLC Symposium: Global Discovery on the Internet: A Grand Challenge AAAS Annual Meeting 16-20 February 2006 St. Louis, MO

  2. Global Discovery: Speeding up the Diffusion of New Scientific Knowledge What is the goal of Global Discovery? Greatly increase the contact rate between distant communities – through a virtual aggregation or federation of diverse deep web databases. How will we achieve it? Through multiple simultaneous deep web searches with integrated ranking of results. 2

  3. Math Databases: • Research Papers • Correspondence • Conferences Global Discovery Search Portal Knowledge Diffusion in Action Mathematician’s Scientific Discovery Math Community • Biology Databases: • Research Papers • Correspondence • Conferences Biology Researcher’s Scientific Discovery Biology Community • Physics Databases: • Research Papers • Correspondence • Conferences Physics Scientific Discovery Physics Community 3

  4. Challenges in Working with Thousands of Data Sources Locate Reliable Sources Categorize Sources by Content Configure Sources for Searching Maintain Sources 4

  5. Challenges in Searching Thousands of Sources Automatically Select Sources to Search Perform Many Searches in Parallel Analyze and Organize Results Relevance Rank Cluster/ Visualize 5

  6. ResearchAssistant The State-of-the-art Federated Search Engine Behind Science.gov • Scalable, grid-computing based federated search engine • Sophisticated Search Conductor • Multi-tier relevance ranking • Framework accepts integration of advanced linguistic, analyses, and visualization modules Sponsored by DOE and Science.gov Alliance 6

  7. Grid Computing: Distributing the Workload 7

  8. Select sources to search Perform search Enough good results? YES Deliver results to user NO Can I get more results from “good” sources? YES NO Search Conductor 8

  9. Multi-tier Relevance Ranking • QuickRank – Ranks results based on occurrence of search terms in title and snippet • MetaRank– Ranks results utilizing custom algorithms applied to metadata • DeepRank – Downloads and indexes full-text documents 9

  10. Search Conductor Sample Document Database Source Selection Optimizer Previous Results Database Source Selection Optimizer 10

  11. Summary It is unknown at this time how many data sources would be searched through a comprehensive Global Discovery portal. Although there are significant challenges, DOE, and the Science.gov alliance with DWT as a partner are well on the road to turning the vision of global discovery into reality 11

  12. Turning Vision into Reality Abe Lederman 122 Longview Drive Los Alamos, NM 87544 abe@deepwebtech.com www.deepwebtech.com 12

More Related