1 / 10

Harvesting Process

Harvesting Process. Isovera Consulting Feb. 2008. Quarterly Harvesting Process. Harvesting Overview. Goal: To collect decentralized metadata for learning objects, located at various collections, in a central “portal” database. Requirements:

kasper-charles
Download Presentation

Harvesting Process

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Harvesting Process Isovera Consulting Feb. 2008

  2. Quarterly Harvesting Process Internet consulting for non-profits

  3. Harvesting Overview • Goal: To collect decentralized metadata for learning objects, located at various collections, in a central “portal” database. • Requirements: • Portal and Collections must agree upon the metadata format, i.e., the BEN Metadata Specification • Collections must provide metadata which adheres to Portal’s specifications, i.e., is of sufficient quality, includes all required fields, etc. • Portal and Collections must agree upon a protocol (communication procedure), i.e., the Protocol for Metadata Harvesting • Portal must implement an OAI “Harvester” program which requests data from Collections • Collection must implement a OAI “Provider” program which responds to thresher requests. Internet consulting for non-profits

  4. Quarterly Harvesting Process, Part 1 • BEN Project Manager asks each collaborator to determine whether a significant body of resources has been added or modified in the past quarter, and have been properly peer reviewed. • Collaborator responds, indicating readiness or lack of readiness for harvesting. • BEN Technical Staff use OAI Harvester administrative tools to harvest resources from Collaborator to the BEN staging site. • BEN Technical Staff prepare a report of all resources harvested from the Collaborator. Internet consulting for non-profits

  5. Quarterly Harvesting Process, Part 2 • BEN Project Manager and Collaborator Project Manager review the resources on the staging site. • If some resources have technical problems, BEN Technical Staff re-harvest to the staging site, and another round of review begins. • If resources appear to be technically sound, BEN Technical staff use OAI Harvester administrative tools to harvest resources from Collaborator to the BEN production site. • Once resources have been harvested to the production site, BEN Portal end users may view them. Internet consulting for non-profits

  6. Requirements for Harvested Resources • Metadata includes required metadata fields (see Metadata presentation.) • Resource is peer-reviewed for scientific accuracy and educational value. • URL is well-formed and resolves to a proper digital resource. • Metadata is accurate, and lacks spelling and grammatical errors. Internet consulting for non-profits

  7. Process Technical Overview • BEN OAI Harvester issues “Identify” request. • Collaborator OAI Provider responds with identifying information. • BEN OAI Harvester issues “ListIdentifiers” request. • Collaborator OAI Provider responds with a list of identifiers for all resources created or modified since the last harvest. • Usually, an identifier is just a number automatically generated by the database as a primary key. • BEN OAI Harvester issues a “GetRecord” request for each resource listed in Step 4. • Collaborator OAI Provider responds with a BEN-LOM XML document for the requested resource. Internet consulting for non-profits

  8. Process Technical Overview BEN OAI Harvester Collaborator OAI Provider Internet consulting for non-profits

  9. How the Harvester/Reaper Works • BEN OAI Harvester issues HTTP Request • e.g., http://www.collaborator.org/OAI-provider?verb=GetRecord&identifier=123&metadataPrefix=oai_BEN • Collaborator web server executes OAI Provider program (Perl script or PHP module) • Collaborator OAI Provider parses HTTP request • OAI Provider requests BEN LOM metadata records from library system. • Library system returns BEN-LOM XML. • OAI Provider wraps BEN-LOM document in OAI-PMH envelope. • BEN OAI Harvester opens OAI-PMH envelope, reads BEN-LOM document, and inserts metadata into BEN Portal database. Internet consulting for non-profits

  10. References • Open Archives Initiative Protocol for Metadata Harvesting - http://www.openarchives.org/OAI/openarchivesprotocol.html • BEN Collaborators Peer Review Policies - http://www.biosciednet.org/project_site/PeerReviewProcessOfBENPartners.pdf Internet consulting for non-profits

More Related