Open Source Intelligence:
This presentation is the property of its rightful owner.
Sponsored Links
1 / 17

Open Source Intelligence: PowerPoint PPT Presentation


  • 71 Views
  • Uploaded on
  • Presentation posted in: General

Open Source Intelligence:. Access All Intelligence, in All Languages, All the Time. Presented by Abe Lederman, President and CTO Deep Web Technologies, LLC.

Download Presentation

Open Source Intelligence:

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Open source intelligence

Open Source Intelligence:

Access All Intelligence, in All Languages, All the Time

Presented by

Abe Lederman, President and CTO

Deep Web Technologies, LLC

IOP ’06 Sheraton Premier, Tysons Corner, Virginia January 16-20


About deep web technologies dwt

About Deep Web Technologies (DWT)

DWT is a New Mexico based company focused on providing state-of-the-art software solutions which search, retrieve, aggregate, and analyze content.

  • Deployed first “federated search” portal in the Federal Government, 1999

  • Major clients include:

    • DOE Office of Scientific & Technical Information

    • Defense Technical Information Center

    • Science.gov Alliance

    • DOE Office of Science

    • National Agricultural Library


Open source intelligence

Open Source Intelligence

The Problem:

  • Collecting and analyzing enormous quantities of information in any language, in myriad formats, located anywhere, accessible through a large variety of means, with a majority not accessible through the Internet


Shared challenge osint and knowledge discovery diffusion

Shared Challenge: OSINT and Knowledge Discovery/Diffusion

OSINT Challenges

Knowledge Discovery/ Diffusion Challenges

DWT for the past six years has been the lead technical organization addressing these challenges in collaboration with DOE Office of Scientific & Technical Information


The dwt proposition

The DWT Proposition

To apply DWT’s technology, expertise and ongoing innovations* to address the challenges of OSINT

*Developed in partnership with DOE/OSTI


Challenges in working with thousands of data sources

Challenges in Working with Thousands of Data Sources

Locate Reliable Sources

Categorize Sources by Content

Configure Sources for Searching

Maintain Sources


Challenges in searching thousands of sources

Challenges in Searching Thousands of Sources

Automatically Select Sources to Search

Perform Many Searches in Parallel

Translate, Analyze and Organize Results

Relevance Rank

Extract Key Information

Cluster/ Visualize


Dwt s state of the art federated search engine

ResearchAssistantTM

DWT’s State-of-the-art Federated Search Engine

  • Scalable, grid-computing based federated search engine

  • Sophisticated Search Conductor

  • Supports custom connectors

  • Multi-tier relevance ranking

  • Framework accepts integration of advanced linguistic, analyses, and visualization modules


Open source intelligence

Grid Computing: Distributing the Workload


Search conductor

Perform search

Enough good results?

Deliver results to user

Can I get more results from “good” sources?

Search Conductor

Select sources to search

YES

NO

YES

NO


Multi tier relevance ranking

Multi-tier Relevance Ranking

  • QuickRankTM – Ranks results based on occurrence of search terms in title and snippet

  • MetaRankTM – Ranks results utilizing custom algorithms applied to metadata

  • DeepRankTM – Downloads and indexes full-text documents


Open source intelligence

Science.gov Alliance Consortium of 12 Federal Government Agencies

Dept of Agriculture

Dept of Commerce

Dept of Defense

Dept of Education

Dept of Energy

Dept of Health/Human Services

Dept of Interior

Environmental Protection Agency

NASA

National Science Foundation

US Government Printing Office

National Archives & Records Administration

Sponsoring

Science.gov Portal

(Access to most of Federal Government R&D


Open source intelligence

Science.gov Advanced Search Page


Open source intelligence

Science.gov Results Page


A science gov document

A Science.gov Document


Next steps

Next Steps

Identify Sponsors and development partners that can collaborate on the development of a pilot that integrates best-of-breed technologies of value to OSINT.

This pilot will result in a portal that aggregates content of different types, generating actionable intelligence.


Open source intelligence

Contact Us

Abe Lederman

122 Longview Drive

Los Alamos, NM 87544

[email protected]

www.deepwebtech.com

http://www.deepwebtech.com/talks/IOP.ppt


  • Login