Practical Approaches to Sharing Information at Raytheon Taxonomies, Metadata and Beyond

1. Practical Approaches to Sharing Information at Raytheon Taxonomies, Metadata and Beyond Presented by Christine JM. Connors Christine_Connors@raytheon.com KMPro / KMForum, Bentley College, Waltham, MA June 30, 2004

2. Level of information management varies 85% of our information is unstructured Over 90% of information is not tagged High proportion of tagged documents result of templates, and therefore relay bad data Ethan Frome – over 200 documents Automatically extracted data imprecise “The flight to Dallas is cancelled” Greater than 13% of information is exactly duplicated “Near” duplication harder to determine but potentially more costly Worst duplication in File Servers / Shared Drives Difficult to determine true age of document due to web scripting, date of publication to public drive Over 23% of sample data not modified in previous 5 years Over 68% of sample data not modified in previous 2 years Data Discovery – What we Learned

3. 16 multiple choice questions, 1 optional free-text comment field 516 surveys started (clicked into) 199 responses over 3 weeks 39% completion rate 101 comments on “How can we improve the intranet search and browse capabilities?” 51% comment rate by survey participants Intranet Search and Browse SurveyJune 2003

4. 101 user comments frequently included: Qualify searches by function, organization, and business Qualify searches by date Qualify searches by document type (especially web pages) Provide sorting of results by date, document type Provide category search Do not change URLs of pages (users bookmarked) Reduce number of search results Google (mentioned 32 times in comments) How can we improve the intranet search and browse capabilities?

5. Search/browse About ¼ (26%) of respondents find the current capabilities “Good” or “Excellent” About ¼ (24%) of respondents consistently locate helpful information About ¼ (22%) of respondents indicate they are consistently successful using keyword searching About ¼ (26%) of respondents find it consistently easy to browse The advanced search page is used sparingly Categories 62% of respondents would find categories consistently useful Bookmarks 36% of respondents consistently return to previously found information 71% of respondents use bookmarks “Most of the time” or “Always” 65% of respondents consistently organize their bookmarks Results Summary

6. “The search engine is poor to inadequate. I needed to find an appropriations data sheet and was returned 366 entries, none which had anything to do with appropriations. I spend far too much time looking through the search results for this engine to be effective. If I could find this document on the INTERNET I would do so, but this is an internal Raytheon document that is successfully hidden somewhere in the archives with the Ark of the Covenant.” Unidentified search and browse survey participant, June, 2003 “Who gets more hits: www.amazon.com or www.thequaintbookstoredownthestreet.com? Listen up people: Our intranet is a wasteland of information. We need to unify - we need to standardize. Information is power - but only if it is on my desktop, not hidden away in some server waiting for a lucky adventurer to uncover it like some lost continent.” Another unidentified search and browse survey participant, June, 2003 OUCH!

7. Bentley College’s Design and Usability Testing Center 4 Focus Groups of 8-10 people each They told us: Want to filter searches Didn’t want long list of items to select from Liked “Suggested…” boxes Didn’t understand the taxonomy when presented like Yahoo! Liked taxonomy as file folder metaphor Liked thesaurus Were confused by relational taxonomies Liked “Categories” as the tab label, over Topics, Taxonomy, Thesaurus, Subjects or Browse. Usability Testing

8. The Dream Team Information Scientists Cognitive Scientists Linguists Programmers Database Experts Network Specialists Verity Administrators Human Computer Interaction / Usability Experts Subject Matter Experts Organizational Change Management Taxonomies – Who? • What we got • Information Scientists (in-house) • Cognitive Scientist / Linguist (from Verity) • Programmers, Database Experts, Network Specialists, and Verity Administrators (both in-house and from Verity) • HCI / UI Experts (in-house, Verity and Bentley College) • Subject Matter Experts (in-house)

9. Browse & Navigation Relational Taxonomies Refine Search Parametric Search Federated Search Dynamic taxonomies Profiling Compliance Engine Classification & Categorization Provide controlled vocabularies to use with Metadata Schema(s) Easy selection to minimize angst over having to fill out file properties Taxonomies – What for?

10. Deployed 5 taxonomies Defense Technologies (based on DTIC) Purchased DTIC taxonomy Revised to fit Raytheon’s data Removed several categories including agriculture that are not needed Raytheon Products Revised our products listing into a hierarchical approach Enlisted Raytheon Communicators as Subject Matter Experts IPDS Built using data from the IPDS web site Enlisted IPDS experts as Subject Matter Experts Engineering Implemented taxonomy built by Raytheon’s Engineering Technology Network – Needs revising and enhancing Information Technology Purchased from Verity Revised to fit Raytheon’s data Enlisted members of Corporate IT as Subject Matter Experts Taxonomies 2003

11. Human Resources taxonomy Deployed March 2004 Purchased from Verity Cross-functional team of HR representatives reviewed as Subject Matter Experts Six Sigma Will be deployed June 2004 Built based on Raytheon Six Sigma data Legal Taxonomy Will be deployed June 2004 Purchased from Verity Able to create additional taxonomies for Ethics, Environmental Health & Safety, and Export/Import Compliance from the purchase of this ONE taxonomy Will be restructuring our top level categories : Business Units – domestic & international Functions Processes Products Relationships Taxonomies 2004

12. Card sorting EZ Sort 3x5 cards Review search engine logs Internal logs Webtrends Review organic systems Web and file share navigation Review existing taxonomies/thesauri Concept Mapping Linguistic algorithm Intelligent Classifier – lots of query building behind each node MultiTes Mind Manager TextPad BUY! Taxonomies – How?

13. Build to suit users Reflect corporate vernacular Internal acronyms Corporate culture How is the business structured and portrayed? Can take a long time Time estimates depend on type/use of taxonomy/tools available Simple = 5 minutes to build term, 5 minutes to build category and map the topic to the taxonomy Complex = 75 minutes to build term PLUS Quality Assurance testing! Buy Industry standard Rapid implementation Need customization Both decisions require maintenance Build vs. Buy

14. Benefits - Increased Productivity • Delphi Group 2003 – as reported by Gartner • Business professionals spend more than 2 hours per day searching for information • Half of that time – 1 hour per day is wasted by failure to find what they seek • The single factor most attributed to the large amount of time wasted was • data changes (location 35%) and • bad tools (ineffective search and lack of labeling 28%) • If we conservatively assume only 1 hour per year would be saved per general employee and 1 hour per month per engineer, then:

15. TB TB TB TB TB Benefits – Reduce Storage Costs Relative of starting point, growth curves represent storage acquisition cost increases over time.

16. “New” search launched September 29 Latest survey results show improvement Neutral rating upgraded to Good Metrics show increased usage of search 17% increase in unique users per day 25% increase in searches per day Metrics show increasing use of categories in search Since launch, the categories have been used 50,000 times ITLT approved project funding Knowledge Representation team recipients of 2003 IT Excellence in Collaboration and Knowledge Management Award Is it working?