Querying the Web for Genealogical Information

1. Querying the Web for Genealogical Information Troy Walker Spring Research Conference 2003 Research funded by NSF

2. Genealogical Information on the Web • Hundreds of thousands of sites • Some professional (Ancestry.com, Familysearch.org) • Mostly hobbyist (Cyndislist.com) • Search engines • “Walker genealogy” on Google: 199,000 results • 1 page/minute = 5 months to go through • Why not enlist the help of a computer?

3. Problems • No standard way of presenting data • Text formatted with HTML tags • Tables • Forms to access information • Each site has its own idea of what genealogical information is—differing schemas

4. Proposed solution • Based on Ontos and other work done at the BYU Data Extraction Group • Able to extract from: • Semi-structured or unstructured text • Tables • Forms • Scalable and robust to changes in pages • Built for genealogy but easily adaptable to other domains

5. Text

6. Tables

7. Forms

8. Forms

9. System Overview URL Database Unstructured or Semi-Structured Text Engine User Query Document Retriever Document Structure Recognizer Table Engine Data Extraction Engine Result Filter Form Engine Mapping Information To be implemented To be improved To be integrated

10. User Query • Form generated from ontology • Query by example

11. URL Databaseand Document Retriever • Contains Genealogy URLs • Search each URL—too much time • Filter likely URLs

12. Method Selector • Analyze page • Select appropriate method

13. Preprocessing Engines • Text • Improved record-separation • Ability to handle single-record pages • Table • Forms

14. Extraction Engine • Ontos • Cache schema matches

15. Result Filter • Filters objects relevant to query • Presents to user

16. Conclusion • Integrates, builds on previous DEG work • Extracts from: • Semi-structured or unstructured text • Tables • Forms • Scalable—only searches probable pages • Robust to changes in pages • Ontology based—easily adapted to other domains