from ldap to patron load generating a patron update sif file from an ldap directory dump l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
From LDAP to Patron Load Generating a patron update SIF file from an LDAP directory dump PowerPoint Presentation
Download Presentation
From LDAP to Patron Load Generating a patron update SIF file from an LDAP directory dump

Loading in 2 Seconds...

play fullscreen
1 / 53

From LDAP to Patron Load Generating a patron update SIF file from an LDAP directory dump - PowerPoint PPT Presentation


  • 280 Views
  • Uploaded on

From LDAP to Patron Load Generating a patron update SIF file from an LDAP directory dump. Michael Doran, Systems Librarian. Endeavor Users Group Meeting, Chicago, IL Session 8, Thursday, April 26, 2007. Cataloging. OPAC. Circulation. Acquisitions. Integrated library system.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'From LDAP to Patron Load Generating a patron update SIF file from an LDAP directory dump' - Gabriel


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
from ldap to patron load generating a patron update sif file from an ldap directory dump

From LDAP to Patron LoadGenerating a patron update SIF file from an LDAP directory dump

Michael Doran, Systems Librarian

Endeavor Users Group Meeting, Chicago, IL

Session 8, Thursday, April 26, 2007

integrated library system

Cataloging

OPAC

Circulation

Acquisitions

Integrated library system

patron database

Michael Doran

123 Elm St.

Arlington, TX

PATRON BLOCK

patron DB

Michael Doran, Systems Librarian

getting patron data into voyager
Getting patron data into Voyager

D’oh!

  • Manual data entry via the Circulation client

Michael Doran, Systems Librarian

getting patron data into voyager4
Getting patron data into Voyager
  • Manual data entry via the Circulation client
  • Batch loading via the Pptrnupdt job
    • Technical User’s Guide
      • 6 : Patron Update
      • 19 : Patron Record

Standard Interface File

Michael Doran, Systems Librarian

the patron sif specification
The Patron SIF specification
  • Each record will consist of
    • 1 base segment - fixed length
      • 52 fixed length fields
      • certain fields are required
    • 1-9 address segments - fixed length
      • 19 fixed length fields
      • one address is required
    • 1 optional notes segment - variable length
      • 1 variable length field
    • 1 end-of-record mark

Michael Doran, Systems Librarian

patron sif one record
Patron SIF – one record

Michael Doran, Systems Librarian

patron sif one record7
Patron SIF – one record

Michael Doran, Systems Librarian

patron sif documentation
Patron SIF documentation

Michael Doran, Systems Librarian

patron sif documentation9
Patron SIF documentation

Item (field) number – 72 total

Item (field) name

Offset – starting point

Item (field) description

Format

n - numeric

s - string

d - date

Required field?

Length of field

Michael Doran, Systems Librarian

patron sif files the easy way
Patron SIF files the easy way…

You send the Patron SIF documentation

to your University IT department…

Here’s your Patron SIF

file. Enjoy!

Patron SIF

programmer

… and voila

Michael Doran, Systems Librarian

patron sif files the hard way
Patron SIF files the hard way…

I want you to write a program that generates Patron SIF files. Enjoy!

Voyager

Manuals

your boss

Michael Doran, Systems Librarian

so we have our marching orders
So we have our marching orders

Michael Doran, Systems Librarian

somewhere on your campus is
Somewhere on your campus is…
  • One or more directories of information on people registered to attend classes
  • One or more directories of information on people who work for the university

students

faculty

and staff

“patrons”

Michael Doran, Systems Librarian

directories databases protocols apis
Directories, databases, protocols, APIs
  • directory

an alphabetical or classified list (as of names or addresses)

  • database

a collection of data organized for rapid retrieval (as by a computer)

  • protocol

a standard that controls the connection, communication, and data transfer between two computing endpoints

  • application programming interface (API)

an abstraction layer between the gory details of an application and the world outside that software

Michael Doran, Systems Librarian

directories databases protocols apis15
Directories, databases, protocols, APIs

LDAP directory

  • directory

an alphabetical or classified list (as of names or addresses)

  • database

a collection of data organized for rapid retrieval (as by a computer)

  • protocol

a standard that controls the connection, communication, and data transfer between two computing endpoints

  • application programming interface (API)

an abstraction layer between the gory details of an application and the world outside that software

Lightweight Directory Access Protocol

Michael Doran, Systems Librarian

directories databases protocols apis16
Directories, databases, protocols, APIs

Voyager

patron load

  • directory

an alphabetical or classified list (as of names or addresses)

  • database

a collection of data organized for rapid retrieval (as by a computer)

  • protocol

a standard that controls the connection, communication, and data transfer between two computing endpoints

  • application programming interface (API)

an abstraction layer between the gory details of an application and the world outside that software

“Standard Interface File” (SIF)

Pptrnupdt

Michael Doran, Systems Librarian

slide17

Voyager patron database tables and relationships

Michael Doran, Systems Librarian

the pieces and players
The pieces and players

Directory

Voyager

LDAP Protocol

LDAP server

LDAP client

Net::LDAP

?

Perl program

patron SIF

Pptrnupdt

patron info

patron info

Michael Doran, Systems Librarian

the pieces and players19
The pieces and players

Directory

Voyager

LDAP Protocol

LDAP server

LDAP client

Net::LDAP

?

Perl program

patron SIF

Pptrnupdt

patron info

patron info

patron info

Michael Doran, Systems Librarian

three easy steps
Three easy(?) steps…
  • Query the LDAP directory
    • filter for valid patrons
  • Process the records
    • minor data formatting
    • decision algorithms
    • output records in SIF format
  • Load SIF file into Voyager
    • Pptrnupdt

Michael Doran, Systems Librarian

three easy steps21
Three easy(?) steps…
  • Query the LDAP directory
    • filter for valid patrons
  • Process the records
    • minor data formatting
    • decision algorithms
    • output records in SIF format
  • Load SIF file into Voyager
    • Pptrnupdt

Perl program

Michael Doran, Systems Librarian

before we start perl programming
Before we start Perl programming

Directory

Voyager

LDAP Protocol

LDAP server

LDAP client

Net::LDAP

ldapsearch

?

Perl program

patron SIF

Pptrnupdt

patron info

patron info

Michael Doran, Systems Librarian

what you need to get started
What you need to get started
  • command line client
    • ldapsearch comes standard on Solaris
  • LDAP server info
    • hostname
    • port numbers
      • 389 – default ldap port
      • 636 – default ldaps port
    • LDAP Schema
  • bind credentials
    • Distinguished Name (DN)
    • anonymous
    • your own (e.g. NetID username & password)
    • “superuser” (provided by LDAP administrator)

Michael Doran, Systems Librarian

ldapsearch search
ldapsearch search

ldapsearch \

-h "ldap.uta.edu" \

-D "uid=doran,cn=accounts,dc=uta,dc=edu" \

-w "<password>" \

-b "cn=people, dc=uta, dc=edu" \

"utaAccountName=doran"

Michael Doran, Systems Librarian

ldapsearch search25
ldapsearch search

ldapsearch \

-h "ldap.uta.edu" \

-D "uid=doran,cn=accounts,dc=uta,dc=edu" \

-w "<password>" \

-b "cn=people, dc=uta, dc=edu" \

"utaAccountName=doran"

<uid>

LDAP server hostname

Bind Distinguished Name

Search Base

Search Filter

Bind Credentials

Michael Doran, Systems Librarian

ldapsearch output
ldapsearch output

cedarid=1085691573581181000,cn=People,dc=uta,dc=edu

utaAccountName=doran

utaPrevAccountName=doran

utaPrimaryAccount=doran

utaPrimaryEmail=doran@uta.edu

utaHomeStreet1=123 Elm St.

utaEmployeeTitle=Librarian

utaDeptCode=6800000

utaEmployeeStatus=active

utaEmployeeCampusBox=19497

eduPersonPrimaryAffiliation=staff

utaHomeCity=Arlington

utaEmployeeTitleCode=0100

sn=Doran

cedarid=1085691573581181000

utaHomeState=TX

homePhone=+1 817 555 1234

utaEmployeeJobCategory=prof_non_faculty

cn=Michael D Doran

utaEmployeeType=monthly

utaHomeZip=76013

utaEmployeeBldg=LIB

givenName=Michael

search filter match

displayName=doran, michael d

utaEmployeeExemption=exempt

utaEmployeePhone=+1 817 272 5326

homePostalAddress=123 Elm St.$Arlington$TX$76013

utaID=1000068572

utaEmployeePercentTime=100.00

utaEmployeeRetirementCode=9

utaEmplID=1000068572

utaMiddleName=D

objectClass=top

objectClass=person

objectClass=inetOrgPerson

objectClass=utaPerson

objectClass=utaEmployee

eduPersonScopedAffiliation=employee@uta.edu

eduPersonScopedAffiliation=staff@uta.edu

utaPersonAffiliation=employee

utaPersonAffiliation=staff

utaPersonAffiliation=administrative

eduPersonAffiliation=employee

eduPersonAffiliation=staff

“patron-worthy” search filter

Michael Doran, Systems Librarian

ldap search with perl net ldap
LDAP search with Perl Net::LDAP

#!/usr/local/bin/perl -w

use Net::LDAP;

$ldap = Net::LDAP->new("ldap.uta.edu");

$mesg = $ldap->bind("uid=doran,cn=accounts,dc=uta,dc=edu",

password => "<password>");

$mesg = $ldap->search(

base => 'cn=people, dc=uta, dc=edu',

filter => "(utaAccountName=doran)"

);

foreach $entry ($mesg->all_entries) {

$entry->dump;

};

$ldap->unbind();

exit(0);

Michael Doran, Systems Librarian

ldap search with perl net ldap28
LDAP search with Perl Net::LDAP

#!/usr/local/bin/perl -w

use Net::LDAP;

$ldap = Net::LDAP->new("ldap.uta.edu");

$mesg = $ldap->bind("uid=doran,cn=accounts,dc=uta,dc=edu",

password => "<password>");

$mesg = $ldap->search(

base => "cn=people, dc=uta, dc=edu",

filter => "(utaAccountName=doran)"

);

foreach $entry ($mesg->all_entries) {

$entry->dump;

};

$ldap->unbind();

exit(0);

this Perl module will need to be installed

LDAP server hostname

Bind Distinguished Name

Search Base

Search Filter

Bind Credentials

Michael Doran, Systems Librarian

perl net ldap output
Perl Net::LDAP output

dn:cedarid=1085691573581181000,cn=People,dc=uta,dc=edu

utaAccountName: doran

utaPrimaryAccount: doran

utaPrimaryEmail: doran@uta.edu

utaHomeStreet1: 123 Elm St.

utaEmployeeTitle: Librarian

utaDeptCode: 6800000

utaEmployeeStatus: active

utaEmployeeCampusBox: 19497

eduPersonPrimaryAffiliation: staff

utaHomeCity: Arlington

utaEmployeeTitleCode: 0100

sn: Doran

cedarid: 1085691573581181000

utaHomeState: TX

homePhone: +1 817 555 1234

utaEmployeeJobCategory: prof_non_faculty

cn: Michael D Doran

utaEmployeeType: monthly

utaHomeZip: 76013

utaEmployeeBldg: LIB

givenName: Michael

displayName: doran, michael d

...: ...

Michael Doran, Systems Librarian

superuser valid patron filtering
Superuser & valid patron filtering

#!/usr/local/bin/perl -w

use Net::LDAP;

$ldap = Net::LDAP->new("ldap.uta.edu");

$mesg = $ldap->bind("cn=library,cn=app,dc=uta,dc=edu",

password => "<password>");

$mesg = $ldap->search(

base => 'cn=people, dc=uta, dc=edu',

filter => "(|(utaStudentStatus=enrolled)

(utaEmployeeStatus=active)

(utaEmployeeStatus=retired))"

);

foreach $entry ($mesg->all_entries) {

$entry->dump;

};

...

a superuser Bind DN

patron-worthy LDAP records

Michael Doran, Systems Librarian

three easy steps31
Three easy(?) steps…
  • Query the LDAP directory
    • filter for valid patrons
  • Process the records
    • minor data formatting
    • decision algorithms
    • output records in SIF format
  • Load SIF file into Voyager
    • Pptrnupdt

Michael Doran, Systems Librarian

ldap records simplified
LDAP records simplified

dn:cedarid=1085691573581181000,cn=People,dc=uta,dc=edu

utaAccountName: doran

utaPrimaryAccount: doran

utaPrimaryEmail: doran@uta.edu

utaHomeStreet1: 123 Elm St.

utaEmployeeTitle: Librarian

utaDeptCode: 6800000

utaEmployeeStatus: active

utaEmployeeCampusBox: 19497

eduPersonPrimaryAffiliation: staff

utaHomeCity: Arlington

utaEmployeeTitleCode: 0100

sn: Doran

cedarid: 1085691573581181000

utaHomeState: TX

homePhone: +1 817 555 1234

utaEmployeeJobCategory: prof_non_faculty

cn: Michael D Doran

utaEmployeeType: monthly

utaHomeZip: 76013

utaEmployeeBldg: LIB

givenName: Michael

displayName: doran, michael d

...: ...

attributes

values

attribute-value pair

Michael Doran, Systems Librarian

ldap records made more complex
LDAP records made more complex
  • Not all records have all the same attributes

additional

attributes for

employees

generic

attributes for

all people

additional

attributes for

students

let’s consider “people” records

Michael Doran, Systems Librarian

attributes continued
Attributes continued
  • Some attributes are required, but many (most) are optional
  • Some attributes have a constrained set of possible values
  • Some attributes are “multi-valued”
  • Attributes will not necessarily

be sorted in any order

LDAP

Schema

for information…

Michael Doran, Systems Librarian

sample schema page uta ldap
Sample schema page (UTA LDAP)

Michael Doran, Systems Librarian

sample schema page uta ldap36
Sample schema page (UTA LDAP)

Michael Doran, Systems Librarian

processing records
Processing records

$mesg = $ldap->search(

base => 'cn=people, dc=uta, dc=edu',

filter => "(|(utaStudentStatus=enrolled)

(utaEmployeeStatus=active)

(utaEmployeeStatus=retired))",

callback => \&ProcessRecord

);

foreach $entry ($mesg->all_entries) {

$entry->dump;

};

dumps records

we want to process records

The “callback” option allows you to process

each record as it is received

Michael Doran, Systems Librarian

processrecord subroutine
ProcessRecord subroutine

sub ProcessRecord {

my ($mesg,$entry) = @_;

if (!$entry) {&Finish(0);}

$surname = $entry->get_value('sn');

$first_name = $entry->get_value('givenName');

$middle_name = $entry->get_value('utaMiddleName');

$uta_id = $entry->get_value('utaID');

@uta_affil = $entry->get_value('utaPersonAffiliation');

$email_addr = $entry->get_value('utaPrimaryEmail');

$home_phone = $entry->get_value('homePhone');

if ($home_phone) {

$home_phone =~ s/\+1 //;

$home_phone =~ s/ /\-/g;

}

# ...yada, yada, yada

Michael Doran, Systems Librarian

processrecord subroutine39
ProcessRecord subroutine

sub ProcessRecord {

my ($mesg,$entry) = @_;

if (!$entry) {&Finish(0);}

$surname = $entry->get_value('sn');

$first_name = $entry->get_value('givenName');

$middle_name = $entry->get_value('utaMiddleName');

$uta_id = $entry->get_value('utaID');

@uta_affil = $entry->get_value('utaPersonAffiliation');

$email_addr = $entry->get_value('utaPrimaryEmail');

$home_phone = $entry->get_value('homePhone');

if ($home_phone) {

$home_phone =~ s/\+1 //;

$home_phone =~ s/ /\-/g;

}

# ...yada, yada, yada

single-valued attributes => assigned to a scalar

assigning LDAP attribute values to program variables

multi-valued attributes => assigned to an array

minor reformatting of data

Michael Doran, Systems Librarian

ldap record
LDAP record

Find the Voyager patron group

dn:cedarid=1085691573581181000,cn=People,dc=uta,dc=edu

utaAccountName: doran

utaPrimaryAccount: doran

utaPrimaryEmail: doran@uta.edu

utaHomeStreet1: 123 Elm St.

utaEmployeeTitle: Librarian

utaDeptCode: 6800000

utaEmployeeStatus: active

utaEmployeeCampusBox: 19497

eduPersonPrimaryAffiliation: staff

utaHomeCity: Arlington

utaEmployeeTitleCode: 0100

sn: Doran

cedarid: 1085691573581181000

utaHomeState: TX

homePhone: +1 817 555 1234

utaEmployeeJobCategory: prof_non_faculty

cn: Michael D Doran

utaEmployeeType: monthly

utaHomeZip: 76013

utaEmployeeBldg: LIB

givenName: Michael

displayName: doran, michael d

...: ...

The LDAP record doesn’t have a patron group

Michael Doran, Systems Librarian

decision algorithms
Decision algorithms

program looks at student attributes first

patron group assignment example

if ($stud_status eq "enrolled" && $stud_class) {

if ($stud_class =~ /freshman/i ||

$stud_class =~ /sophomore/i ||

$stud_class =~ /junior/i ||

$stud_class =~ /senior/i ) {

$patron_group = "und";

} elsif ($stud_class =~ /masters/i ||

$stud_class =~ /doctoral/i ||

$stud_class =~ /graduate/i) {

$patron_group = "grad";

$expire_days = "180";

}

} elsif ($stud_status eq "enrolled") {

$patron_group = "und";

}

student classification not a required attribute

in LDAP record, so need a default

Michael Doran, Systems Librarian

decision algorithms42
Decision algorithms

program looks at employee attributes second

if ($empl_status eq "active") {

if (! $patron_group) {

$patron_group = "staff";

}

foreach my $affil (@uta_affil) {

if (($affil =~ /faculty/i) ||

($affil =~ /admin/i) ||

($affil =~ /assistant/i)) {

$patron_group = "fac";

$expire_days = "400";

}

}

}

default only if no student

patron group assigned

Michael Doran, Systems Librarian

decision algorithms continued
Decision algorithms continued

multiple sets of address attributes

work

address

& phone

home address

& phone

student

address

& phone

employees

students

all people

Michael Doran, Systems Librarian

three easy steps44
Three easy(?) steps…
  • Query the LDAP directory
    • filter for valid patrons
  • Process the records
    • minor data formatting
    • decision algorithms
    • output records in SIF format
  • Load SIF file into Voyager
    • Pptrnupdt

Michael Doran, Systems Librarian

a poetry break
A poetry break…

“Our time is fixed,

and all our days are number'd;

How long, how short, we know not.”

Robert Blair, The Grave

“The SIF fields are fixed, and all the fields are number'd;

How long, how short, we better know.”

anonymous programmer

Michael Doran, Systems Librarian

patron sif formatting simplified
Patron SIF formatting simplified
  • For each patron SIF record
    • Format each field
      • s string – left-justified / blank-filled
      • n numeric – right-justified / zero-filled
      • n* “special” numeric – if no value, then all blanks
      • d date – YYYY.MM.DD format
    • Concatenate all fields into one long string
    • Slap an end-of-record marker on string
    • Write string to file

Michael Doran, Systems Librarian

remember this
Remember this?

Michael Doran, Systems Librarian

formatting with sprintf
Formatting with sprintf*

$first_name = sprintf("%-20s",Michael);

$patron_id = sprintf("%010d",123456);

$concatenation = $first_name . $patron_id;

print $concatenation;

Michael 0000123456

*sprintf help

Michael Doran, Systems Librarian

three easy steps49
Three easy steps…
  • Query the LDAP directory
    • filter for valid patrons
  • Process the records
    • minor data formatting
    • decision algorithms
    • output records in SIF format
  • Load SIF file into Voyager
    • Pptrnupdt

Michael Doran, Systems Librarian

things i didn t cover
Things I didn’t cover
  • Encryption
  • Data validation
  • Log files
  • Session file
  • Cron job setup
  • yada, yada, yada…

Michael Doran, Systems Librarian

you can do this
You* can do this!

Patron SIF

LDAP

protocol

*…you, or your

resident geek

Perl

programming

geek stigmata

Michael Doran, Systems Librarian

resources
Resources
  • Voyager Technical User’s Guide
    • Patron SIF specification
    • Patron Update (Pptrnupdt)
  • Perl Net::LDAP module
    • download from CPAN
    • documentation
  • Perl Date::Calc module
    • download from CPAN
    • documentation
  • LDAP to patron SIF webpage (ldump.pl)

Michael Doran, Systems Librarian

questions
Questions?

Please fill out the session evaluation!

Michael Doran, Systems Librarian