Benefits of NCIP

2. Agenda Overview of the NCIP Standard Example implementations What implementers should know and expect from NCIP What libraries should know and expect from NCIP Future direction of NCIP After reviewing the agenda: You know the rule where someone asks for a volunteer to step forward and everyone who’s smart steps back? Well in the summer of 1998 I failed to step back quickly enough and so I was selected to attend the meeting of the “Automation Vendors Industry Advisory Council” at ALA Annual in place the engineering VP of the company I worked for at the time – Ameritech Library Services. At that meeting the discussion turned to the slow rate of adoption of peer-to-peer ISO ILL systems and the consensus was that the lack of a standard for doing circulation services such as check out, or creating temporary bib and item records, was a large part of the problem. After reviewing the agenda: You know the rule where someone asks for a volunteer to step forward and everyone who’s smart steps back? Well in the summer of 1998 I failed to step back quickly enough and so I was selected to attend the meeting of the “Automation Vendors Industry Advisory Council” at ALA Annual in place the engineering VP of the company I worked for at the time – Ameritech Library Services. At that meeting the discussion turned to the slow rate of adoption of peer-to-peer ISO ILL systems and the consensus was that the lack of a standard for doing circulation services such as check out, or creating temporary bib and item records, was a large part of the problem.

3. Committee Charge Define transactions needed for circulation systems among independent “library” systems Facilitate direct patron borrowing remote patron authentication circulation/ILL interaction online payment controlled access to electronic resources

4. Scope of Standard Defines services, the messages that comprise those services and the structure and semantics of message elements. Does not define circulation functions or policies Does not define user interface

5. Applications Supported Circulation/InterLibrary Loan Interaction Direct Consortial Borrowing Self-service Circulation Access to Electronic Resources It had to support those, it may be able to support others

6. Object Classes Users Items Agencies (Libraries)

7. Service Types Lookup “Tell me these things about this object.” Update “Please take this action.” Notification “I have taken this action.” Service Types are comprised of Services. 3M SIP has Lookup and Update messaging; Notification is novel.3M SIP has Lookup and Update messaging; Notification is novel.

8. Service Definitions Every “Service” is a pair of messages: an “Initiation Message” and a “Response Message” Each message provides complete context for it to be understood The protocol is designed NOT to require any particular sequence of services. This doesn’t mean your application must support all possible sequences. It only means that the NCIP Standard doesn’t presume any sequence of service invocations.This doesn’t mean your application must support all possible sequences. It only means that the NCIP Standard doesn’t presume any sequence of service invocations.

9. Lookup Service Lookup Agency Lookup Item Lookup User Lookup Version Authenticate User Lookup Request (version 1.01) Remember a “Lookup” service type means “Tell me these things about this object.” So a Lookup User service means “Tell me these things about this User.” The Lookup User message initiates the Lookup User Service. Note the easy naming convention: Initiation message name is identical to Service name. The corresponding Response message is the Service name with the word “Response” appended to the end. Authenticate User does Not authorize the user. Authorization request is implicit in any update.Remember a “Lookup” service type means “Tell me these things about this object.” So a Lookup User service means “Tell me these things about this User.” The Lookup User message initiates the Lookup User Service. Note the easy naming convention: Initiation message name is identical to Service name. The corresponding Response message is the Service name with the word “Response” appended to the end. Authenticate User does Not authorize the user. Authorization request is implicit in any update.

10. Lookup Service Restrictions Lookups require a Unique Id They do not support discovery or searching Discovery, or search & retrieval, should be handled by Z39.50 or SRU (they’re not just for Bib/Holdings!)Discovery, or search & retrieval, should be handled by Z39.50 or SRU (they’re not just for Bib/Holdings!)

11. Update Services Typical Circulation Transactions: Request Item and Cancel Request Item Check Out Item and Undo Check Out Item Renew Item Recall Item and Cancel Recall Item Send User Notice Check In Item Accept Item Now we’re at the heart of any circulation protocol. If you scan the left side of that list you’ll see steps that may occur during any traditional circulation transaction; Request Item, Check Out Item, Renew Item, etc. down through Check In Item. Obviously those are necessary. And it’s easy to see the utility of “Cancel Request Item” and “Cancel Recall Item”; Requesting and Recalling leave an interval between when they’re invoked and when they’re fulfilled and thus the need to cancel them while they’re outstanding. If the remaining service is unfamiliar to you, perhaps it will help to consider a self-service checkout machine that has just sent the Check Out Item message to the circulation system. Before it gets the response message from the circulation system the user switches out the paperback edition of Moby Dick and replaces it with the original manuscript. Assuming you have security devices in your manuscripts, the self-service machine might desensitize the manuscript and let your user rob you of a priceless original. Unless of course it detects the switch and initiates a Cancel Check Out Item service to cancel the checkout of the paperback (and whatever other measures are necessary for handling the user).Now we’re at the heart of any circulation protocol. If you scan the left side of that list you’ll see steps that may occur during any traditional circulation transaction; Request Item, Check Out Item, Renew Item, etc. down through Check In Item. Obviously those are necessary. And it’s easy to see the utility of “Cancel Request Item” and “Cancel Recall Item”; Requesting and Recalling leave an interval between when they’re invoked and when they’re fulfilled and thus the need to cancel them while they’re outstanding. If the remaining service is unfamiliar to you, perhaps it will help to consider a self-service checkout machine that has just sent the Check Out Item message to the circulation system. Before it gets the response message from the circulation system the user switches out the paperback edition of Moby Dick and replaces it with the original manuscript. Assuming you have security devices in your manuscripts, the self-service machine might desensitize the manuscript and let your user rob you of a priceless original. Unless of course it detects the switch and initiates a Cancel Check Out Item service to cancel the checkout of the paperback (and whatever other measures are necessary for handling the user).

12. Update Services (cont.) Object maintenance: Create Agency and Update Agency Create Item, Update Item, Update Request Item, Update Circulation Status and Report Circulation Status Change Create User and Update User Create User Fiscal Transaction Create Services used for new objects Update Services include modify and delete The Update Services listed here aren’t the heart of circulation, but you could say they’re the skeleton: without services for creating and updating the objects involved in circulation it would only be possible to perform circulation transactions on existing objects. Because the protocol’s scope included Direct Consortial Borrowing it had to support creating and updating user records in the item owning agency’s system and item records in the user’s agency’s system. Because we wanted to support broker applications that might interact with discrete systems on behalf of some third system, we wanted to support creation and update of agencies as well. Other than Create and Update services for our three familiar object classes, the list includes only Update Request Item and Update User Fiscal Account. Update Request Item was included to handle updating the information associated with a request for an item, where that information isn’t part of the item or user objects (e.g., Shipping Address). Update User Fiscal Account was included to handle updating the information associated with a user’s account (e.g., to add a fine to a user’s account for trying to steal that manuscript copy of Moby Dick).The Update Services listed here aren’t the heart of circulation, but you could say they’re the skeleton: without services for creating and updating the objects involved in circulation it would only be possible to perform circulation transactions on existing objects. Because the protocol’s scope included Direct Consortial Borrowing it had to support creating and updating user records in the item owning agency’s system and item records in the user’s agency’s system. Because we wanted to support broker applications that might interact with discrete systems on behalf of some third system, we wanted to support creation and update of agencies as well. Other than Create and Update services for our three familiar object classes, the list includes only Update Request Item and Update User Fiscal Account. Update Request Item was included to handle updating the information associated with a request for an item, where that information isn’t part of the item or user objects (e.g., Shipping Address). Update User Fiscal Account was included to handle updating the information associated with a user’s account (e.g., to add a fine to a user’s account for trying to steal that manuscript copy of Moby Dick).

13. Notification Services Typical Circulation Transactions: Item Requested and Item Request Cancelled Item Checked Out Item Renewed Item Recalled and Item Recall Cancelled User Notice Sent Item Checked In Item Shipped and Item Received Notice that for every update service (but one) there is a Notification service; thus every Update service can be tracked by one or more other agencies (for example, perhaps a funding agency wants to track them for statistical purposes). The Item Shipped and Item Received Notification services are essentially the Notification analogues to the Accept Item Update service, permitting either or both of the two applications involved in the Accept Item service to perform the Notification service as may be needed in an application area. The Undo Check Out Item does not have a matching Notification service because the Undo Check Out Item service must be invoked immediately after the Check Out Item service; circ systems that notify third parties of item check-outs must hold off on initiating the Item Checked Out service until after the Undo Check Out Item is no longer permitted. Pay attention to the naming convention for Notification services: they are the passive voice (past tense) of the matching Update service.Notice that for every update service (but one) there is a Notification service; thus every Update service can be tracked by one or more other agencies (for example, perhaps a funding agency wants to track them for statistical purposes). The Item Shipped and Item Received Notification services are essentially the Notification analogues to the Accept Item Update service, permitting either or both of the two applications involved in the Accept Item service to perform the Notification service as may be needed in an application area. The Undo Check Out Item does not have a matching Notification service because the Undo Check Out Item service must be invoked immediately after the Check Out Item service; circ systems that notify third parties of item check-outs must hold off on initiating the Item Checked Out service until after the Undo Check Out Item is no longer permitted. Pay attention to the naming convention for Notification services: they are the passive voice (past tense) of the matching Update service.

14. Notification Services (cont.) Object maintenance: Agency Created and Agency Updated Item Created, Item Updated, Item Request Updated, Circulation Status Updated and Circulation Status Change Reported User Created and User Updated User Fiscal Transaction Created These are all pretty obvious analogues to services in the Update Service Type.These are all pretty obvious analogues to services in the Update Service Type.

15. Notification Response Notifications occur after the fact The only permitted responses are Did not understand message Understood message Important note: There’s no way to say anything significant in response to a Notification message. Only “I understood what you said” or “I didn’t understand what you said.” In particular there’s no way to respond with a statement such as “Please don’t bother me with this.”The committee quite intentionally meant the Notifications to presume no behavior on the part of the responder. In designing your systems you should think through the implications of a responder that (quite validly from the protocol’s point of view) always hands back a “I understood what you said” and then does nothing whatever with the content of the message. That’s not to say you can’t build your design around an assumption that the responder will do something very specific with the message; however you can’t ensure that via the protocol so you’ll have to do that via consortial agreeements, static profiling, really good documentation, etc. All “extra-protocol” stuff. In summary, looking at three Service Types, there are: 5 Lookup services, 20 Update services, and 20 Notification services. For a grand total of 45 services (which is 90 messages). Important note: There’s no way to say anything significant in response to a Notification message. Only “I understood what you said” or “I didn’t understand what you said.” In particular there’s no way to respond with a statement such as “Please don’t bother me with this.”The committee quite intentionally meant the Notifications to presume no behavior on the part of the responder. In designing your systems you should think through the implications of a responder that (quite validly from the protocol’s point of view) always hands back a “I understood what you said” and then does nothing whatever with the content of the message. That’s not to say you can’t build your design around an assumption that the responder will do something very specific with the message; however you can’t ensure that via the protocol so you’ll have to do that via consortial agreeements, static profiling, really good documentation, etc. All “extra-protocol” stuff. In summary, looking at three Service Types, there are: 5 Lookup services, 20 Update services, and 20 Notification services. For a grand total of 45 services (which is 90 messages).

16. Message Structure Syntax and Encoding Enumerated Types: Scheme/Value pairs Datatypes

17. Syntax and Encoding XML DTD UTF-8 encoding of Unicode ASCII character encoding is the same in this encoding, but … Other systems are NOT restricted to ASCII, and you should be prepared to receive such data.

18. Enumerated Types: Scheme/Value pairs Enumerated data types are represented by a pair of elements: Scheme and Value. Ensures that codes (the Value element) are in a context (the Scheme element). Provides for extensibility

19. Scheme/Values (cont.) Example enumerated types: Language Defined by ISO 639-2 Bibliographic Language Codes Currency Codes: Defined by ISO 4217:1995 Codes for the representation of currencies and funds.

20. Scheme/Values (cont.) Allows for extensibility The Standard provides a Bibliographic Record Identifier Code scheme including these values: ANBN, BGF, BNBN, CN, LCCN, NLM TCN, OCLC, RLIN If you need a different list you can define your own scheme

21. Scheme/Values (cont.) Three kinds of Schemes: Closed, Enumerated Those defined in the Standard must be supported in order to conform New schemes may NOT be defined Open, Enumerated Those defined in the Standard must be supported in order to conform New schemes may be defined Open, Not Enumerated None are defined in the Standard An example of a Closed, Enumerated scheme is the Messaging Error scheme – to guarantee that the initiator will always understand what the responder says went wrong. Another example is the Language scheme, as it was felt that there’s no need for an alternative scheme. Finally there are schemes used in Lookup initiation messages that tell the responder what data the initiator wants back. An example of an Open, Enumerated scheme is Agency User Privilege Type (which is for patron type or category). The committee’s list is barely an example here; it’s almost certain that actual implementations will require their own schemes for this data element. Another example is Authentication Prompt Type, which identifies the type of data in the prompt used to authenticate users – the committee chose the IANA registry of MIME Media types so your prompts could be text/plain, text/rtf, sgml, image/gif, etc. Probably that will serve most purposes! The Open, Not Enumerated schemes are: Application Profile Supported Type, Application Profile Type, Consortium Agreement, From System Id, To System Id, and Unique Agency Id – all of these are entirely application or environment specific and there was no sense developing an example.An example of a Closed, Enumerated scheme is the Messaging Error scheme – to guarantee that the initiator will always understand what the responder says went wrong. Another example is the Language scheme, as it was felt that there’s no need for an alternative scheme. Finally there are schemes used in Lookup initiation messages that tell the responder what data the initiator wants back. An example of an Open, Enumerated scheme is Agency User Privilege Type (which is for patron type or category). The committee’s list is barely an example here; it’s almost certain that actual implementations will require their own schemes for this data element. Another example is Authentication Prompt Type, which identifies the type of data in the prompt used to authenticate users – the committee chose the IANA registry of MIME Media types so your prompts could be text/plain, text/rtf, sgml, image/gif, etc. Probably that will serve most purposes! The Open, Not Enumerated schemes are: Application Profile Supported Type, Application Profile Type, Consortium Agreement, From System Id, To System Id, and Unique Agency Id – all of these are entirely application or environment specific and there was no sense developing an example.

22. Scheme Registration Scheme names conform to URI specification Values within any scheme must be unique Once published, the list of values must not change in any way NCIP maintenance agency will host a registration service.

23. Datatypes Taken from XML Datatypes http://www.w3.org/TR/xmlschema-2/ 6 datatypes: boolean “true”, “false”, “1”, “0” integer nonNegativeInteger positiveInteger As a point of clarification, presently everything in an XML document or message must be represented in characters (this is different from the BER encoding used in Z39.50 and ISO 10161).As a point of clarification, presently everything in an XML document or message must be represented in characters (this is different from the BER encoding used in Z39.50 and ISO 10161).

24. Datatypes (cont.) timeInstant Restricted to ISO 8601’s “Extended format” Expressed in Coordinated Universal Time (UTC). “CCYY-MM-DDThh:mm:ss.sssZ” string You can append “-hh:mm” or “+hh:mm” to indicate local time as a difference (plus or minus) from UTC. Expressed as fixed attributes in the DTD. In the non-normative XML Schema these are proper datatypes.

25. Technical Foundation Application Roles Messaging Required Behavior Rules Security

26. Application Roles For a given connection, there is: 1 and only 1 initiating application (e.g., self-service machine), and 1 and only 1 responding application (e.g., circ system). Initiators may NOT send a second message until the first is responded to. Responders may NOT send initiation messages on that connection.

27. Application Roles (cont.) Applications MAY establish multiple connections at the same time. The Standard is written in terms of “initiating application” and “responding application”; this is always in the context of a given connection, not in the broader context of the application as a whole.

28. Messaging State Tables Transport Requirements Transport Protocol(s)

29. Messaging State Tables Do NOT govern the state of the circulation transaction DO govern the state of the exchange of the initiation/response message pair Initiating application is in IDLE or WAITING state Responding application is in IDLE or PROCESSING state

30. Defined Transport Protocols Initiator chooses from these 3: TCP/IP HTTP HTTPS Responder must reply on same connection

31. Omission of Requested Elements Applies to entire Lookup Service Type and to “piggy-backed” lookups on Update Services. Permits omission of some of the data the initiator asked for. Permits omission of the “Electronic Resource” element if the responder would rather not supply it in the response message.

32. Update Processing Responding application will behave as if all deletions requested were performed before all additions requested in the same message If an update to one element causes an update to another element not specifically asked - a Notification message may be used to inform the other side Example - change of birthday causes user category to change

33. Messaging Errors Indicate lack of understanding of the message: Invalid XML XML not conformant to the DTD Unknown scheme

34. Processing Errors Indicate inability or unwillingness to perform the action requested User Delinquent Unknown item Item does not circulate (Checkout) Maximum renewals exceeded (Renewal)

35. Document Structure Protocol Definition Implementation Profile 1 XML DTD/Schema Application Profiles

36. Application Profiles Currently three application areas: Consortial borrowing Circulation / ILL Self-service May be multiple profiles per application area Define how to use NCIP within a given application context

37. Application Profiles (cont.) Profiles can define: Messages used Message sequencing Optional data elements that are mandatory Transport protocols required Schemes required or available Security requirements Use cases

38. Application Profiles (cont.) Some Application Profiles Written by NCIP Committee – meant as proof of concept for what Application Profiles should contain. Intent is that Application Profiles will be developed to define requirements of specific Applications/Implementations.

39. Example Implementations Interlibrary loan Patron self-service Direct Consortial Borrowing

40. Interlibrary Loan Example: Ship

41. Interlibrary Loan Example: Renew

42. Patron Self-Service Example: Registration

43. Direct Consortial Borrowing Example: Check Out Item Add data elements to message arrow text. Add patron graphic.Add data elements to message arrow text. Add patron graphic.

44. Direct Consortial Borrowing Example: Overdue Notice Add data elements to message arrow text. Add patron graphic.Add data elements to message arrow text. Add patron graphic.

45. What Implementers Should Know Designers: Protocol, particularly the services you will use. Programmers: Protocol and Implementation Profile 1 XML, Unicode UTF-8 There are tools to help with mapping XML to objects (e.g. for Java there’s Castor or JAXB). Internet transports (TCP/IP, HTTP, HTTPS) There are standard ways to pass messages over these transports; don’t “roll your own” if you can avoid it. The NCIP-IG has a Roadmap document to guide your staff through the learning process.

46. What Implementers Should Expect Like with all standards, there will be surprising differences of interpretation between implementers Expect early testing to be slow going Be mindful of what you expect from other standards (e.g. MARC, Z39.50) There are many differences that haven’t mattered because the results were being read by humans (e.g. contents of MARC 001 field) which you will now be using in an automated fashion so their contents must be “right.” It will help responders greatly if the initiators reach consensus if they’re doing similar things E.g., Circ/ILL and DCB-3 turned out to be virtually identical.

47. What Libraries Should Know NCIP can automate many routine tasks in many areas of the library. NCIP implementations will require two (or more) vendors to co-operate. Understand key NCIP enumerated types: E.g., how does NCIP’s “Agency” map to your circulation units (e.g. service desks, branches, etc.) – this will depend on how your circ vendor implements NCIP.

48. What Libraries Should Expect Initial deployment of NCIP for any vendor is likely to be rough going. Multiple updates to two or more systems will have to be co-ordinated. This is a big part of the future of library automation (not just NCIP) Interoperation between systems for “micro services” Using XML, Unicode, HTTPS

49. Future Directions Simplification Relationship to ISO ILL Library standards as a whole Web 2.0