Converting local portal data to myspace
Download
1 / 26

Converting Local Portal Data to MySpace - PowerPoint PPT Presentation


  • 90 Views
  • Uploaded on
  • Presentation posted in: General

Converting Local Portal Data to MySpace. Anne L. Highsmith, Consortia Systems Coordinator & Prabha Acharya, Programmer Analyst II Texas A&M University. Points to Cover. Background Data Conversion Issues Testing on a Cloned Database Conversion Process Results and Future Plans.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha

Download Presentation

Converting Local Portal Data to MySpace

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Converting Local Portal Data to MySpace

Anne L. Highsmith, Consortia Systems Coordinator &

Prabha Acharya, Programmer Analyst II

Texas A&M University


Points to Cover

  • Background

  • Data Conversion Issues

  • Testing on a Cloned Database

  • Conversion Process

  • Results and Future Plans

Converting Local Portal Data to MySpace ELUNA 2007


Background (1)

  • TAMU had a local Electronic Resources Locater (ERL) function to list ejournals and databases.

  • Users searched ERL and clicked on links to access eresources

  • Users could store the links in their MyPortal record on the library’s website

Converting Local Portal Data to MySpace ELUNA 2007


Background (2)

  • Metalib went into production at TAMU in January 2006 as Search Now.

  • TAMU had discussions with Ex Libris staff as to how MyPortal links could be converted to Metalib

    • First proposal, for Ex Libris staff to do a major conversion, not feasible

    • Decided to wait for Metalib 4.0 and X-Portal APIs to store data to MySpace

Converting Local Portal Data to MySpace ELUNA 2007


Rude awakening!

  • Fall of 2006, Ex Libris staff informed TAMU there would be no API to store to MySpace

  • Alternatives

    • Margaritas?

    • Convert them ourselves?

Converting Local Portal Data to MySpace ELUNA 2007


Conversion options

  • Assumed conversion would have to be:

    • User-initiated

    • Title by title

  • Discovered a way do the conversion directly, without involving the end user

Converting Local Portal Data to MySpace ELUNA 2007


Data Conversion Issues

  • Patrons

  • Journals

  • Databases

Converting Local Portal Data to MySpace ELUNA 2007


Data Conversion Issues - Patrons

  • How to match the MyPortal patron with the Metalib user?

    • MyPortal patrons identified by uin (University ID Number)

    • Our Metalib implementation uses uin as user login

    • Needed to match the uin with the z312_source_id

Converting Local Portal Data to MySpace ELUNA 2007


Data Conversion Issues - Journals

  • How to match MyPortal journals with Metalib?

    • Basic journal data stored in z11 table

      • Wrote Perl script (journals.pl) to extract journal identifier and issn from z11

    • Used MS Access to match MyPortal data, keyed by uin & issn, with Metalib data

    • User journal data stored in z312_jnl_001

      • Wrote Perl script (journals_by_patron.pl) to extract z312_source_id (uin) and journal identifier from Metalib user record in z312 table

Converting Local Portal Data to MySpace ELUNA 2007


Data Conversion Issues - Databases

  • How to match MyPortal databases with Metalib?

    • During creation of Metalib IRDS, stored the ERL key in the IRD cataloger note

      • Loaded IRD extract into MS Access

    • Matched Metalib user extract with IRD extract

    • Basic database data stored in z122 table

      • Wrote Perl script (databases_by_patron.pl) to extract z312_source_id (uin) and Metalib identifier (TEX #)

Converting Local Portal Data to MySpace ELUNA 2007


Data Conversion Issues - Final

  • As a result of matching ERL data and Metalib data, created a file that contained 1 or 2 entries for each user whose data had to be converted:

    • Tab-separated record beginning with uin and list of database id numbers

    • Tab-separated record beginning with uin and list of journal id numbers

Converting Local Portal Data to MySpace ELUNA 2007


Testing on a Cloned Database

Converting Local Portal Data to MySpace ELUNA 2007


Testing – Cloning the database (1)

  • TAMU has a test Metalib DB on separate server

  • Cloned production DB to test DB so could test in a realistic environment

    • mpsync - documented in Appendix D in the System Administrators Guide & Knowledge Base entry #1240.

    • exp_current_library & imp_current_library, documented in 7.8 in the SysAdmin Guide

  • Further doc – “Working with Test/Failover Installations of MetaLib”

Converting Local Portal Data to MySpace ELUNA 2007


Cloning – Export from Production

  • Created files on prod for export to test

    • Run mpsync to copy IRDs

      • TAMU has approx. 4500 IRDS

      • Elapsed time – 58 sec.

    • Run export current library for vir00

      • TAMU has 32,000+ user records

      • Elapsed time – 1 hr.

    • Run export current library for jnl01

      • TAMU has approx. 44,000 journal records

      • Elapsed time – 2:25 min.

Converting Local Portal Data to MySpace ELUNA 2007


Cloning – Import into Test

  • Import IRDS created by mpsync

    • Elapsed time – 3 hrs. 688 new records added

    • Matched existing keys. Appeared to create some duplicates

  • Import User Data

    • Elapsed time – 40 min.

    • Drops & re-adds tables

  • Import Journal Data

    • Elapsed time – 3 min.

    • Drops & re-adds tables

Converting Local Portal Data to MySpace ELUNA 2007


Conversion Process

  • Quality control

  • File preparation

  • Loading new patrons

  • Convert databases and ejournals

Converting Local Portal Data to MySpace ELUNA 2007


Conversion – Quality Control

  • Wrote a Perl script (pre_migration.pl) to collect stats and set up QC measures. For each user it reports:

    • Users who don’t have Metalib record

    • # of databases and journals already in Metalib

    • # of databases and journals to convert

    • # of users that have more than maximum number of databases or ejournals

Converting Local Portal Data to MySpace ELUNA 2007


Conversion – File Preparation

  • Testing demonstrated that It takes about 3 seconds per resource to be migrated.

  • With about 13000 resources that would be about 10 hours

  • So, wrote Perl script (breakup_myportal_file.pl) to break the input file into several files of about 250 records each.

Converting Local Portal Data to MySpace ELUNA 2007


Conversion – Loading new patrons

  • Since TAMU uses external authentication rather than patron load, some older MyPortal patrons not in Metalib database

  • Using file created during QC stage, create_xml_patron_load.pl formats user records for Metalib patron load utility

    • Set expiry date to 1 year in future

    • Based on a review of records created on that date, approx. 65 have since logged in.

Converting Local Portal Data to MySpace ELUNA 2007


Conversion - Convert databases and ejournals (1)

  • Run databases_by_patron.pl to get current lists from user records for comparison

Converting Local Portal Data to MySpace ELUNA 2007


Conversion - Convert databases and ejournals (2)

  • Using save_to_bookbag.cgi, for each user

    • Log in to Metalib

    • Send request to Metalib pds to get pds handle

    • Send request to Metalib, using just obtained pds handle, to get session id

    • Construct url of the form

      • http://metalibhost.edu/V/sessionid?pds_handle=pdshandle&func=find-db-add-res&resource=resourceid

Converting Local Portal Data to MySpace ELUNA 2007


Conversion - Convert databases and ejournals (3)

  • Set session id in cookie

  • For each database

    • Check to see if it duplicates db already stored

    • If not, store db to user’s clipboard

  • For each journal

    • Store to user’s record

Converting Local Portal Data to MySpace ELUNA 2007


Conversion - Convert databases and ejournals (4)

  • Do QC to make sure that all of the appropriate databases and ejournals are stored

  • Exit the session

  • Error logging revealed that for some users, 1 database or ejournal was not stored – these were re-run as an added input file

Converting Local Portal Data to MySpace ELUNA 2007


Final results

  • Processing all files took most of the week, off and on

  • Each file, limited to 250 records, took about 1 hour to process

  • Only post-production problem had nothing to do with conversion – ezproxy error.

  • Once users have logged into library portal, can now go directly to My Databases or My Ejournals list within Metalib

Converting Local Portal Data to MySpace ELUNA 2007


Future plans

  • Conversion of Voyager My Bookbag to e-shelf

Converting Local Portal Data to MySpace ELUNA 2007


Contact & download info

Anne L. Highsmith

Consortia Systems Coordinator

Texas A&M University

hismith@tamu.edu

URL to download perl code, cgi, etc.

http://libraryasp.tamu.edu/users/ahighsmi/presentations/save_to_bookbag.zip

Converting Local Portal Data to MySpace ELUNA 2007


ad
  • Login