Challenges and Solutions in Enterprise Search Experience

1. The Enterprise Search Experience Mindmetre Research, October 2011

2. Smartlogic • Hugely varied clients • Recently acquired Schemalogic • Based in UK and USA

3. MindMetre • Independent Research • Since late 1990s • Arms-length fieldwork • Business issues • Global • Web technology, marketing technology, financial services, healthcare, governmental policy

4. Business Pressures • More from information asset • Less information effort • Information explosion • User dissatisfaction • Browse versus find

5. Information Overload • When Bill Clinton left office, the volume of information and documents concerning his presidency that had to be archived amounted to some 4 terabytes • When George Bush Jr left office, the equivalent volume of information and documents amounted to around 80 terabytes

6. The situation ten years ago • IDC study - "Quantifying Enterprise Search," May 2002 • Knowledge workers spend from 15% to 35% of their time searching for information. • Searchers are successful in finding what they seek 50% of the time or less • Only 21% of respondents said they found the information they needed 85% to100% of the time • 40% of corporate users reported that they can not find the information they need to do their jobs on their intranets • 90% of the time that knowledge workers spend in creating new reports or other products is spent in recreating information that already exists

7. The situation today • Over 2,000 senior company managers surveyed in US, UK, France and Germany • Derived expectation from a low quality requirement (external web search) • Threshold of acceptability – “a good search experience” – is two minutes • More than half of respondents said they cannot find the information they are seeking using their own organisation’s enterprise search facilities in two minutes or less • 2001 -2011 – FLATLINING?

8. Why Flatlining? • Information overload has increased faster than the massive efforts • External has more commercial imperatives than internal (and therefore has received massive effort in metadata) • Also highly visible measures (SE ranking, transactions) • Some exceptions – data loss prevention; regulatory compliance and investigation management • Irony – enterprise search queries likely to be of higher quality than external search queries • Similar/same technology requirements for enterprise search as external search

9. Enterprise search - commercial imperatives • Reduce waste of time and task repetition (e.g. product development, research, marketing, analysis) • More rapid reactions for competitive edge • Turn information assets into earnings • Be ready for regulatory inspections • A March 2009 IDC study found that reducing the time wasted by dealing with information overload by just 15% could save a 500+ employee company over $2m per year.

10. Enterprise search issues/ requirements • We expect google yet we fail to invest in it- • Search not connected to the site -Natural History Museum • Benchmark established as two minutes or less Blackberry • ‘Anarchy of the algorithm’ not acceptable -UNICEF • Enterprises and industries have context, language, standards, authority lists, parameters -NASA • Content intelligence therefore required so that search (& find!) understand the query in context and lead to the relevant information as quickly as possible – NHS Choices • Content made intelligent through accurate metadata • So how are we applying metadata?

11. Search An ExampleBeautiful web page…

12. Search An Example…tunnel-vision search

13. Search Algorithm anarchy

14. Connecting user language to content

15. Metadata driven

16. Connecting to navigation

17. Enterprise search - bottlenecks • 67% of enterprises surveyed say it is very difficult to find information in their organisation, other than key financial or sales data • 62% believe that information overload means that only a small proportion of documents are made easily available to management and staff • 61% say the vast majority of documents in organisations such as theirs are never properly categorised for accurate and rapid retrieval • Two thirds believe that cost is the main obstacle to effective categorisation

18. Attitudes to automation • Manual categorisation is unaffordable in a world of information explosion • Yet 7 in 10 respondents believe that automated systems can perform effective categorisation • Tagging challenge • In fact, a level of manual analysis is always required to implement contextual categorisation • People know best, machines implement best • Human intelligence, automated consistency

19. The tagging challenge

20. Conclusions • Despite the efforts of information architects, enterprise search is not yet satisfying its users, both at the individual or corporate level • Enterprise search has compelling commercial imperatives, but they are not widely enough recognised by general management • Content intelligence is the key to making all the investment in content management and BI translate into enterprise search that finds what people need ‘in a couple of minutes’

21. Enterprise Search Content Management Portal Infrastructure Document Management Content Intelligence Records Management Social collaboration Publishing Systems Digital Asset Management Process Management & Workflow eDiscovery

22. http://www.smartlogic.com/home/knowledge-zone/white-papers/1129-mind-the-enterprise-search-gap-white-paperhttp://www.smartlogic.com/home/knowledge-zone/white-papers/1129-mind-the-enterprise-search-gap-white-paper