Collection & Service Description and the NISO Metasearch Initiative

1. Collection & Service Description and the NISO Metasearch Initiative Juha Hakala, Director (IT), Helsinki University Library Chair, NISO Metasearch Initiative Task Group 2 Pete Johnston, UKOLN, University of Bath Member, NISO Metasearch Initiative Task Group 2 Special Session, DC-2004, Shanghai, China, Wednesday 13 October 2004 http://www.ukoln.ac.uk/

2. Collection & Service Description for the NISO Metasearch Initiative • The Metasearch problem • The NISO Metasearch Initiative • Collections and Services • Collection Description & Service Description

3. The Metasearch problem

4. The problem • Content providers make their collections available through their own separate “presentation services” • User wants to access/use items from multiple content providers • User has to discover, access and interact with multiple presentation services • But • Each service has different user interface for discovery • Results human-readable (HTML), but difficult to merge, reuse, manipulate • Different authentication/access requirements

5. The solution (ideally…) • The provision of "Metasearch" services that • enable user to search across the metadata databases of multiple content providers from a single interface • manage multiple result sets and present to user • manage authentication/access • (etc!) • Technologies exist e.g. • (Real-time) cross-searching (Z39.50, SRW/U, service-specific APIs) • Harvesting (OAI-PMH) • Seamless (to the user) discovery of and access to heterogeneous, distributed resources! • However…..

6. The problems with Metasearch today • User requires/expects resources from increasing range of content providers • Many content providers have not implemented standards-based search interfaces • Many proprietary APIs • Some "screen scraping" (parsing of HTML) • Metasearch services do work, but • fragile, susceptible to changes by content provider • labour-intensive, scalability issues • duplication of effort • Also content provider concerns about • efficiency/effectiveness of search • access management, logging etc • branding/IPR/presentation of results

7. What is needed • For effective Metasearch services, content providers and service providers need agreement on (at least…) • Transport protocol(s) • Query language(s) • syntax and semantics • Metadata schemas • syntax and semantics • Intellectual property rights issues • how metadata records and resources are presented, used • Authorisation / authentication • Disclosure / discovery of collections and services

8. The NISO Metasearch Initiative http://www.lib.ncsu.edu/niso-mi/

9. The NISO Metasearch Initiative • Response to content provider/service provider concerns • Bring together • Content providers • System vendors • Library service providers • Standards developers • "To identify, develop, and frame the standards and other common understandings that are needed to enable an efficient and robust information environment"

10. The NISO Metasearch Initiative • Aims to enable • metasearch service providers to offer more effective and responsive services • content providers to deliver enhanced content and protect their intellectual property • libraries to deliver services that distinguish their offerings from other free web services

11. The NISO Metasearch Initiative • Standardisation of metasearch applications (portals) must be accomplished • Traditional integrated library systems are fairly well standardised • ISO 2709 Exchange format (since early 1970s) • MARC formats & AACR2 cataloguing rules • Z39.50 Information retrieval protocol • ISO ILL, NCIP • For metasearch applications, many relevant standards will be developed in the NISO MI • This will e.g. enable libraries and other users to exchange metadata between them

12. The NISO Metasearch Initiative: Current Activity • Task Group 1: Access Management • Gather requirements for access/authentication • Describe existing processes • Develop use cases • Task Group 2: Collection Description • Establish metasearch services' requirements for description of • Collections • Services which provide access to Collections ("Informational Services") • Select/develop metadata schemas • Recommend syntax for representation & data exchange • Task Group 3: Search & Retrieval • Describe existing practice • Metadata to describe result sets • Metadata to describe article-level citations

13. Collections and Services

14. Collections and Services • Item • A physical or digital entity • Collection • An aggregation of one or more items • Service • The provision of, or system of supplying, one or more functions of interest to an end-user or software application. • Physical or digital • Digital services may be "structured" or "unstructured" • Informational services • Services that provide access to, or metadata about, items and/or collections • JISC Information Environment Architecture: Glossary http://www.ukoln.ac.uk/distributed-systems/jisc-ie/arch/

15. Website "Screen-scrape" OAI repository Harvestvia OAI-PMH Collection of digital orphysicalitems Collection of digitalmetadata records Z39.50 target Search/retrievevia Z39.50 RSS channel Alert via RSS/HTTP unstructured network service structured network service Informational services

16. Website "Screen-scrape" OAI repository Harvest Z39.50 target Search Alert RSS channel

17. Functional Model: “Surveying the landscape” • Agent • "Enters" information landscape • Views a default set of collections, based on information about the agent • "Surveys" landscape • Modifies landscape by adding/removing collections, based on information about the collections • "Discovers" items of interest within collections • "Drills down" into selected collections • N.B. Agent may be • Human researcher • Human administrator of presentation service • Software application acting on behalf of human researcher http://www.ukoln.ac.uk/distributed-systems/jisc-ie/arch/functional-model/ http://ccinterop.cdlr.strath.ac.uk/

18. My default landscape Coll B Coll C Coll D Coll E Surveying the information landscape

19. My default landscape My default landscape My default landscape Coll B Coll B Coll A Coll C Coll C Coll B Coll D Coll D Coll C Coll E Coll E Coll D modified

20. Functional Requirements • Allow an agent to • Discover collections of potential interest • Identify a collection • Select one or more collections from amongst a number of discovered collections • Identify the informational services that provide access to the collection • Select a service with which to interact • Interact with service • Subject to "knowledge" of interface semantics Collectiondescription Servicedescription

21. Relations between Collections and Services • Relationships exist • Between collections and services • Between collections • In NISO MI conceptual model • A collectionis-made-available-by zero or more services • A servicemakes-available exactly one collection • A collectionis-part-of zero or more (super-) collections (parent) • A collectionhas-parts zero or more (sub-) collections (child)

22. Service is-Made-Available-By Service is-Made-Available-By is-Part-Of is-Part-Of is-Made-Available-By Collection Collection Service is-Made-Available-By Service is-Part-Of is-Part-Of Collection Collection Collection

23. Collection Description &Service Description

24. Collection Description & Service Description • NISO MI TG2 specifying metadata for collections & services • Data model • Metadata semantics • Syntax(es) for representation and data exchange • Guidelines for use • Should build on/reuse existing work where possible • Make recommendations for future work • N.B. TG2 is not • building a service; or • specifying the architecture within which a service might operate • specifying the protocols for the exchange of collection/service metadata

25. Collection Description

26. Collection Description • Collection as “an aggregation of one or more items” • "functional granularity" • Collection-level description • Description of the collection as a whole • Unitary finding-aid • Considerable recent work on collection-level description • Research Support Libraries Programme (UK, 1999-2002) • support for academic research • improve disclosure/discovery of library/archive collections • also collaborative collection management • recognition of CLD as important mechanism for disclosure/discovery

27. RSLP Collection Description Project, 1999-2000 • Funded by RSLP, OCLC • RSLP CD Model • Entity-Relation model (Michael Heaney, University of Oxford) • Implementation independent • Intended to be applicable to wide range of collections • Informed by IFLA FRBR approach as well as existing descriptive standards • RSLP CD Schema • DC-based metadata schema (Andy Powell, UKOLN) • Expresses subset of RSLP model • Simplification of model • Significant influence on other initiatives • But concernsover status, ownership, visibility, persistence, maintenance, etc http://www.ukoln.ac.uk/metadata/rslp/

28. creates Creator Content is-embodied-in produces Producer Item is-gathered-into Collector collects Collection owns Owner is-located-in administers Administrator Location RSLP CD Schema v Model

29. DC Collection Description Working Group • Active 2001 (really 2003!) - • Provide forum for sharing information about CLD activity • Develop a DC Application Profile for collection-level description • Specification of how DC (and other) properties are used for describing collections • Develop supporting materials for use of AP • Informed by experience of RSLP CD implementers and other CLD initiatives • RSLP projects, TEL, JISC IESR, IMLS, others http://dublincore.org/groups/collections/

30. DC Collection Description Application Profile (DC CD AP) • A "core" set of collection description properties • For simple collection-level descriptions • Suitable for a broad range of collections • Primarily to support discovery of collections • Examine collection attributes (only) of RSLP CD Schema as starting point • DC CD AP building on Heaney E-R model • introduces Service as entity-type • describes Collection-Location, Collection-Service, Collection-Agent relationships • but excludes Location, Service, Agent description http://www.ukoln.ac.uk/metadata/dcmi/collection-ap-summary/

31. m is-gathered-into n m 1 is-described-by n Collection CollectionDescription n m 1 collects owns is-located-in is-Made-Available-By m n Service Location m m n n m provides administers Agent m Item

32. DC Collection Description Application Profile (DC CD AP) • Draft 2004-08-20 covers • Identification of collection • Content of items in collection • Form of items in collection • Process by which items gathered into collection • Ownership of collection • Rights of access to/use of collection • Location of collection • Services that provide access to collection • Relationships between collections • Instances can be represented using DC guidelines (RDF/XML, DC-in-XML)

33. NISO MI TG2 & DC CD AP • DC CD AP still work in progress • Some issues • data model (Location/Service) • one-to-one rule • Some terms still to be assigned URIrefs • Scope and specificity of DC CD AP • NISO MI addressing (primarily) library service providers • Some library-specific requirements • e.g. completeness of collection • NISO MI may require superset of DC CD AP • May require non-DCMI naming authority for some metadata terms

34. Service Description

35. Service Description • Led by Larry Dixson, Library of Congress • (Informational) Service • Means of accessing collection • Service description must provide • Indication of protocol used • Access point for service • Authentication/authorisation information • Operations/queries supported • N.B. does not describe the "syntax" of service • Assumption that protocols described elsewhere • NISO MI evaluating use of Zeerex

36. Zeerex background • Z39.50-based specification • Based on earlier work, including Z39.50 Explain Service and Explain Lite, developed in the ONE2 project • Relatively easy to implement, yet allows detailed description of services (e.g. Z39.50 servers) • Sufficiently expressive/flexible to describe similar types of service • Services that provide access to a “database”

37. ServerInfo protocol host/IP port database/service authentication* DatabaseInfo access restrictions Zeerex • IndexInfo • search • scan (browse) • sort • RecordInfo • record syntax • element set name

38. Zeerex and access protocols • Access protocols “in scope” for NISO MI • Z39.50 • SRW/SRU • OAI-PMH • HTTP • LDAP/X.500? • GRID metasearch? • GIS search facility? • Can Zeerex support description of services using all of these protocols? • Tests currently in progress

39. Summary • NISO MI bringing together different stakeholders to develop shared approaches to common problems • Disclosure/discovery of collections and informational services critical to effective metasearch services

40. Acknowledgements • UKOLN is funded by the UK Museums, Libraries and Archives Council (MLA), the Joint Information Systems Committee (JISC) of the UK higher and further education funding councils, as well as by project funding from the JISC and the European Union.UKOLN also receives support from the University of Bath where it is based. • http://www.ukoln.ac.uk/

41. Collection & Service Description and the NISO Metasearch Initiative Juha Hakala, Director (IT), Helsinki University Library Chair, NISO Metasearch Initiative Task Group 2 Pete Johnston, UKOLN, University of Bath Member, NISO Metasearch Initiative Task Group 2 Special Session, DC-2004, Shanghai, China, Wednesday 13 October 2004 http://www.ukoln.ac.uk/