1 / 53

From LDAP to Patron Load Generating a patron update SIF file from an LDAP directory dump

Learn how to create a Patron SIF file from an LDAP directory dump. This session covers the process of querying the LDAP directory, filtering for valid patrons, processing the records, formatting the data, and loading the SIF file into Voyager.

jsane
Download Presentation

From LDAP to Patron Load Generating a patron update SIF file from an LDAP directory dump

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. From LDAP to Patron LoadGenerating a patron update SIF file from an LDAP directory dump Michael Doran, Systems Librarian Endeavor Users Group Meeting, Chicago, IL Session 8, Thursday, April 26, 2007

  2. Cataloging OPAC Circulation Acquisitions Integrated library system patron database Michael Doran 123 Elm St. Arlington, TX PATRON BLOCK patron DB Michael Doran, Systems Librarian

  3. Getting patron data into Voyager D’oh! • Manual data entry via the Circulation client Michael Doran, Systems Librarian

  4. Getting patron data into Voyager • Manual data entry via the Circulation client • Batch loading via the Pptrnupdt job • Technical User’s Guide • 6 : Patron Update • 19 : Patron Record Standard Interface File Michael Doran, Systems Librarian

  5. The Patron SIF specification • Each record will consist of • 1 base segment - fixed length • 52 fixed length fields • certain fields are required • 1-9 address segments - fixed length • 19 fixed length fields • one address is required • 1 optional notes segment - variable length • 1 variable length field • 1 end-of-record mark Michael Doran, Systems Librarian

  6. Patron SIF – one record Michael Doran, Systems Librarian

  7. Patron SIF – one record Michael Doran, Systems Librarian

  8. Patron SIF documentation Michael Doran, Systems Librarian

  9. Patron SIF documentation Item (field) number – 72 total Item (field) name Offset – starting point Item (field) description Format n - numeric s - string d - date Required field? Length of field Michael Doran, Systems Librarian

  10. Patron SIF files the easy way… You send the Patron SIF documentation to your University IT department… Here’s your Patron SIF file. Enjoy! Patron SIF programmer … and voila Michael Doran, Systems Librarian

  11. Patron SIF files the hard way… I want you to write a program that generates Patron SIF files. Enjoy! Voyager Manuals your boss Michael Doran, Systems Librarian

  12. So we have our marching orders Michael Doran, Systems Librarian

  13. Somewhere on your campus is… • One or more directories of information on people registered to attend classes • One or more directories of information on people who work for the university students faculty and staff “patrons” Michael Doran, Systems Librarian

  14. Directories, databases, protocols, APIs • directory an alphabetical or classified list (as of names or addresses) • database a collection of data organized for rapid retrieval (as by a computer) • protocol a standard that controls the connection, communication, and data transfer between two computing endpoints • application programming interface (API) an abstraction layer between the gory details of an application and the world outside that software Michael Doran, Systems Librarian

  15. Directories, databases, protocols, APIs LDAP directory • directory an alphabetical or classified list (as of names or addresses) • database a collection of data organized for rapid retrieval (as by a computer) • protocol a standard that controls the connection, communication, and data transfer between two computing endpoints • application programming interface (API) an abstraction layer between the gory details of an application and the world outside that software Lightweight Directory Access Protocol Michael Doran, Systems Librarian

  16. Directories, databases, protocols, APIs Voyager patron load • directory an alphabetical or classified list (as of names or addresses) • database a collection of data organized for rapid retrieval (as by a computer) • protocol a standard that controls the connection, communication, and data transfer between two computing endpoints • application programming interface (API) an abstraction layer between the gory details of an application and the world outside that software “Standard Interface File” (SIF) Pptrnupdt Michael Doran, Systems Librarian

  17. Voyager patron database tables and relationships Michael Doran, Systems Librarian

  18. The pieces and players Directory Voyager LDAP Protocol LDAP server LDAP client Net::LDAP ? Perl program patron SIF Pptrnupdt patron info patron info Michael Doran, Systems Librarian

  19. The pieces and players Directory Voyager LDAP Protocol LDAP server LDAP client Net::LDAP ? Perl program patron SIF Pptrnupdt patron info patron info patron info Michael Doran, Systems Librarian

  20. Three easy(?) steps… • Query the LDAP directory • filter for valid patrons • Process the records • minor data formatting • decision algorithms • output records in SIF format • Load SIF file into Voyager • Pptrnupdt Michael Doran, Systems Librarian

  21. Three easy(?) steps… • Query the LDAP directory • filter for valid patrons • Process the records • minor data formatting • decision algorithms • output records in SIF format • Load SIF file into Voyager • Pptrnupdt Perl program Michael Doran, Systems Librarian

  22. Before we start Perl programming Directory Voyager LDAP Protocol LDAP server LDAP client Net::LDAP ldapsearch ? Perl program patron SIF Pptrnupdt patron info patron info Michael Doran, Systems Librarian

  23. What you need to get started • command line client • ldapsearch comes standard on Solaris • LDAP server info • hostname • port numbers • 389 – default ldap port • 636 – default ldaps port • LDAP Schema • bind credentials • Distinguished Name (DN) • anonymous • your own (e.g. NetID username & password) • “superuser” (provided by LDAP administrator) Michael Doran, Systems Librarian

  24. ldapsearch search ldapsearch \ -h "ldap.uta.edu" \ -D "uid=doran,cn=accounts,dc=uta,dc=edu" \ -w "<password>" \ -b "cn=people, dc=uta, dc=edu" \ "utaAccountName=doran" Michael Doran, Systems Librarian

  25. ldapsearch search ldapsearch \ -h "ldap.uta.edu" \ -D "uid=doran,cn=accounts,dc=uta,dc=edu" \ -w "<password>" \ -b "cn=people, dc=uta, dc=edu" \ "utaAccountName=doran" <uid> LDAP server hostname Bind Distinguished Name Search Base Search Filter Bind Credentials Michael Doran, Systems Librarian

  26. ldapsearch output cedarid=1085691573581181000,cn=People,dc=uta,dc=edu utaAccountName=doran utaPrevAccountName=doran utaPrimaryAccount=doran utaPrimaryEmail=doran@uta.edu utaHomeStreet1=123 Elm St. utaEmployeeTitle=Librarian utaDeptCode=6800000 utaEmployeeStatus=active utaEmployeeCampusBox=19497 eduPersonPrimaryAffiliation=staff utaHomeCity=Arlington utaEmployeeTitleCode=0100 sn=Doran cedarid=1085691573581181000 utaHomeState=TX homePhone=+1 817 555 1234 utaEmployeeJobCategory=prof_non_faculty cn=Michael D Doran utaEmployeeType=monthly utaHomeZip=76013 utaEmployeeBldg=LIB givenName=Michael search filter match displayName=doran, michael d utaEmployeeExemption=exempt utaEmployeePhone=+1 817 272 5326 homePostalAddress=123 Elm St.$Arlington$TX$76013 utaID=1000068572 utaEmployeePercentTime=100.00 utaEmployeeRetirementCode=9 utaEmplID=1000068572 utaMiddleName=D objectClass=top objectClass=person objectClass=inetOrgPerson objectClass=utaPerson objectClass=utaEmployee eduPersonScopedAffiliation=employee@uta.edu eduPersonScopedAffiliation=staff@uta.edu utaPersonAffiliation=employee utaPersonAffiliation=staff utaPersonAffiliation=administrative eduPersonAffiliation=employee eduPersonAffiliation=staff “patron-worthy” search filter Michael Doran, Systems Librarian

  27. LDAP search with Perl Net::LDAP #!/usr/local/bin/perl -w use Net::LDAP; $ldap = Net::LDAP->new("ldap.uta.edu"); $mesg = $ldap->bind("uid=doran,cn=accounts,dc=uta,dc=edu", password => "<password>"); $mesg = $ldap->search( base => 'cn=people, dc=uta, dc=edu', filter => "(utaAccountName=doran)" ); foreach $entry ($mesg->all_entries) { $entry->dump; }; $ldap->unbind(); exit(0); Michael Doran, Systems Librarian

  28. LDAP search with Perl Net::LDAP #!/usr/local/bin/perl -w use Net::LDAP; $ldap = Net::LDAP->new("ldap.uta.edu"); $mesg = $ldap->bind("uid=doran,cn=accounts,dc=uta,dc=edu", password => "<password>"); $mesg = $ldap->search( base => "cn=people, dc=uta, dc=edu", filter => "(utaAccountName=doran)" ); foreach $entry ($mesg->all_entries) { $entry->dump; }; $ldap->unbind(); exit(0); this Perl module will need to be installed LDAP server hostname Bind Distinguished Name Search Base Search Filter Bind Credentials Michael Doran, Systems Librarian

  29. Perl Net::LDAP output dn:cedarid=1085691573581181000,cn=People,dc=uta,dc=edu utaAccountName: doran utaPrimaryAccount: doran utaPrimaryEmail: doran@uta.edu utaHomeStreet1: 123 Elm St. utaEmployeeTitle: Librarian utaDeptCode: 6800000 utaEmployeeStatus: active utaEmployeeCampusBox: 19497 eduPersonPrimaryAffiliation: staff utaHomeCity: Arlington utaEmployeeTitleCode: 0100 sn: Doran cedarid: 1085691573581181000 utaHomeState: TX homePhone: +1 817 555 1234 utaEmployeeJobCategory: prof_non_faculty cn: Michael D Doran utaEmployeeType: monthly utaHomeZip: 76013 utaEmployeeBldg: LIB givenName: Michael displayName: doran, michael d ...: ... Michael Doran, Systems Librarian

  30. Superuser & valid patron filtering #!/usr/local/bin/perl -w use Net::LDAP; $ldap = Net::LDAP->new("ldap.uta.edu"); $mesg = $ldap->bind("cn=library,cn=app,dc=uta,dc=edu", password => "<password>"); $mesg = $ldap->search( base => 'cn=people, dc=uta, dc=edu', filter => "(|(utaStudentStatus=enrolled) (utaEmployeeStatus=active) (utaEmployeeStatus=retired))" ); foreach $entry ($mesg->all_entries) { $entry->dump; }; ... a superuser Bind DN patron-worthy LDAP records Michael Doran, Systems Librarian

  31. Three easy(?) steps… • Query the LDAP directory • filter for valid patrons • Process the records • minor data formatting • decision algorithms • output records in SIF format • Load SIF file into Voyager • Pptrnupdt Michael Doran, Systems Librarian

  32. LDAP records simplified dn:cedarid=1085691573581181000,cn=People,dc=uta,dc=edu utaAccountName: doran utaPrimaryAccount: doran utaPrimaryEmail: doran@uta.edu utaHomeStreet1: 123 Elm St. utaEmployeeTitle: Librarian utaDeptCode: 6800000 utaEmployeeStatus: active utaEmployeeCampusBox: 19497 eduPersonPrimaryAffiliation: staff utaHomeCity: Arlington utaEmployeeTitleCode: 0100 sn: Doran cedarid: 1085691573581181000 utaHomeState: TX homePhone: +1 817 555 1234 utaEmployeeJobCategory: prof_non_faculty cn: Michael D Doran utaEmployeeType: monthly utaHomeZip: 76013 utaEmployeeBldg: LIB givenName: Michael displayName: doran, michael d ...: ... attributes values attribute-value pair Michael Doran, Systems Librarian

  33. LDAP records made more complex • Not all records have all the same attributes additional attributes for employees generic attributes for all people additional attributes for students let’s consider “people” records Michael Doran, Systems Librarian

  34. Attributes continued • Some attributes are required, but many (most) are optional • Some attributes have a constrained set of possible values • Some attributes are “multi-valued” • Attributes will not necessarily be sorted in any order LDAP Schema for information… Michael Doran, Systems Librarian

  35. Sample schema page (UTA LDAP) Michael Doran, Systems Librarian

  36. Sample schema page (UTA LDAP) Michael Doran, Systems Librarian

  37. Processing records $mesg = $ldap->search( base => 'cn=people, dc=uta, dc=edu', filter => "(|(utaStudentStatus=enrolled) (utaEmployeeStatus=active) (utaEmployeeStatus=retired))", callback => \&ProcessRecord ); foreach $entry ($mesg->all_entries) { $entry->dump; }; dumps records we want to process records The “callback” option allows you to process each record as it is received Michael Doran, Systems Librarian

  38. ProcessRecord subroutine sub ProcessRecord { my ($mesg,$entry) = @_; if (!$entry) {&Finish(0);} $surname = $entry->get_value('sn'); $first_name = $entry->get_value('givenName'); $middle_name = $entry->get_value('utaMiddleName'); $uta_id = $entry->get_value('utaID'); @uta_affil = $entry->get_value('utaPersonAffiliation'); $email_addr = $entry->get_value('utaPrimaryEmail'); $home_phone = $entry->get_value('homePhone'); if ($home_phone) { $home_phone =~ s/\+1 //; $home_phone =~ s/ /\-/g; } # ...yada, yada, yada Michael Doran, Systems Librarian

  39. ProcessRecord subroutine sub ProcessRecord { my ($mesg,$entry) = @_; if (!$entry) {&Finish(0);} $surname = $entry->get_value('sn'); $first_name = $entry->get_value('givenName'); $middle_name = $entry->get_value('utaMiddleName'); $uta_id = $entry->get_value('utaID'); @uta_affil = $entry->get_value('utaPersonAffiliation'); $email_addr = $entry->get_value('utaPrimaryEmail'); $home_phone = $entry->get_value('homePhone'); if ($home_phone) { $home_phone =~ s/\+1 //; $home_phone =~ s/ /\-/g; } # ...yada, yada, yada single-valued attributes => assigned to a scalar assigning LDAP attribute values to program variables multi-valued attributes => assigned to an array minor reformatting of data Michael Doran, Systems Librarian

  40. LDAP record Find the Voyager patron group dn:cedarid=1085691573581181000,cn=People,dc=uta,dc=edu utaAccountName: doran utaPrimaryAccount: doran utaPrimaryEmail: doran@uta.edu utaHomeStreet1: 123 Elm St. utaEmployeeTitle: Librarian utaDeptCode: 6800000 utaEmployeeStatus: active utaEmployeeCampusBox: 19497 eduPersonPrimaryAffiliation: staff utaHomeCity: Arlington utaEmployeeTitleCode: 0100 sn: Doran cedarid: 1085691573581181000 utaHomeState: TX homePhone: +1 817 555 1234 utaEmployeeJobCategory: prof_non_faculty cn: Michael D Doran utaEmployeeType: monthly utaHomeZip: 76013 utaEmployeeBldg: LIB givenName: Michael displayName: doran, michael d ...: ... The LDAP record doesn’t have a patron group Michael Doran, Systems Librarian

  41. Decision algorithms program looks at student attributes first patron group assignment example if ($stud_status eq "enrolled" && $stud_class) { if ($stud_class =~ /freshman/i || $stud_class =~ /sophomore/i || $stud_class =~ /junior/i || $stud_class =~ /senior/i ) { $patron_group = "und"; } elsif ($stud_class =~ /masters/i || $stud_class =~ /doctoral/i || $stud_class =~ /graduate/i) { $patron_group = "grad"; $expire_days = "180"; } } elsif ($stud_status eq "enrolled") { $patron_group = "und"; } student classification not a required attribute in LDAP record, so need a default Michael Doran, Systems Librarian

  42. Decision algorithms program looks at employee attributes second if ($empl_status eq "active") { if (! $patron_group) { $patron_group = "staff"; } foreach my $affil (@uta_affil) { if (($affil =~ /faculty/i) || ($affil =~ /admin/i) || ($affil =~ /assistant/i)) { $patron_group = "fac"; $expire_days = "400"; } } } default only if no student patron group assigned Michael Doran, Systems Librarian

  43. Decision algorithms continued multiple sets of address attributes work address & phone home address & phone student address & phone employees students all people Michael Doran, Systems Librarian

  44. Three easy(?) steps… • Query the LDAP directory • filter for valid patrons • Process the records • minor data formatting • decision algorithms • output records in SIF format • Load SIF file into Voyager • Pptrnupdt Michael Doran, Systems Librarian

  45. A poetry break… “Our time is fixed, and all our days are number'd; How long, how short, we know not.” Robert Blair, The Grave “The SIF fields are fixed, and all the fields are number'd; How long, how short, we better know.” anonymous programmer Michael Doran, Systems Librarian

  46. Patron SIF formatting simplified • For each patron SIF record • Format each field • s string – left-justified / blank-filled • n numeric – right-justified / zero-filled • n* “special” numeric – if no value, then all blanks • d date – YYYY.MM.DD format • Concatenate all fields into one long string • Slap an end-of-record marker on string • Write string to file Michael Doran, Systems Librarian

  47. Remember this? Michael Doran, Systems Librarian

  48. Formatting with sprintf* $first_name = sprintf("%-20s",Michael); $patron_id = sprintf("%010d",123456); $concatenation = $first_name . $patron_id; print $concatenation; Michael 0000123456 *sprintf help Michael Doran, Systems Librarian

  49. Three easy steps… • Query the LDAP directory • filter for valid patrons • Process the records • minor data formatting • decision algorithms • output records in SIF format • Load SIF file into Voyager • Pptrnupdt Michael Doran, Systems Librarian

  50. Things I didn’t cover • Encryption • Data validation • Log files • Session file • Cron job setup • yada, yada, yada… Michael Doran, Systems Librarian

More Related