NLETS Lessons Learned

1. NLETS Lessons Learned

2. Kate Silhol NLETS Senior Software Engineer ksilhol@nlets.org (602) 224-0744 www.nlets.org

3. Its sole purpose is to provide for the international, interstate and/or interagency exchange of criminal justice and criminal justice related information. Our Mission: …to provide, within a secure environment, an international criminal justice telecommunications capability NLETS will assist those national and international governmental agencies and other organizations with similar missions Its sole purpose is to provide for the international, interstate and/or interagency exchange of criminal justice and criminal justice related information. Our Mission: …to provide, within a secure environment, an international criminal justice telecommunications capability NLETS will assist those national and international governmental agencies and other organizations with similar missions

5. Facts and Figures 41 Million transactions per month! Only way to: State Criminal Histories Electronic “HIT” confirmation Canadian Files (CPIC) Homeland Alert Messages Screen Officers Flying Armed INS databases at LESC – IAQs Driver records and vehicle registrations Aircraft Registrations HAZMAT, GSA Federal registrations, NICB Push administrative messages to all CJ agencies

7. Requirements Represent precise data Reliability Performance Flexibility Maintainability

8. NLETS XML Operational Capabilities XML Message Router (XMR) since 2001 All NLETS transactions available in GJXDM 3.0 format plus full legacysystem translation capabilities since September 750,000 XML rap sheets alone per month All FBI, WI and KY rapsheets are in XML All NLETS states/agencies have access to XML and web services

9. XML Formats and Transport NLETS allows users to send XML via Web Services, MQ Series or TCP/IP Allowing XML over various protocols is crucial to users in our environment NLETS transforms between XML and legacy text to allow interoperability between users All new data providers will be connecting via Web Services

11. Web Services Within a secured network, NLETS exposes a web service to which users connect and send XML as a string States wishing to receive a response via web services host own service for NLETS to connect to NLETS specifies name and fingerprint of service, how data is handled is up to state

12. NLETS Standardized GJXDM XML Transactions All Inquiries Standard text format is easy to convert to/from XML Criminal History Responses Leveraging the JTF Criminal History Rapsheet Specification Hazardous Materials Responses Standardized single-source responses from Operation Respond Drivers and Registration Responses Standardized responses from state DMVs based on CANDLE grant products

13. So, how did we do it? Mapped elements to GJXDM Documented XML Instances Created applications to consume XML Created schemas Get users to start implementing Change management

14. Mapping NLETS Transactions First, created a generic message envelope, in NLETS namespace (prefix n) Header information remains in the payload instead of SOAP Multiple transports Legacy header data

15. Mapping NLETS Transactions Next, within the generic NLETS message envelope we defined each transaction in XML Going from a legacy system provided us a strictly defined set of information Used GJXDM elements where available Extended GJXDM elements and types when possible Example: Agency Participated in GJXDM development by utilizing feedback mechanisms (Bugzilla, XSTF)

16. NLETS GJXDM Example <n:NLETS> <n:NLETSMailHeader> <n:MessageKeyCode>RQ</n:MessageKeyCode> <j:DocumentSource.Organization> <j:OrganizationORIID> <j:ID>GA0250300</j:ID> </j:OrganizationORIID> </j:DocumentSource.Organization> <n:DocumentDestinationID> <j:ID>AZ</j:ID> </n:DocumentDestinationID> <n:DocumentControlFieldText>TERM000000</n:DocumentControlFieldText> </n:NLETSMailHeader> <n:NLETSInquiryData Key=“RQ”> <j:VehicleRegistrationPlateID> <j:ID>LJB934</j:ID> </j:VehicleRegistrationPlateID> <j:VehicleRegistrationPlateTypeCode>PC</j:VehicleRegistrationPlateTypeCode > <n:IDExpirationDateText>1983</n:IDExpirationDateText> </n:NLETSInquiryData> </n:NLETS>

17. NLETS GJXDM Header <n:NLETSMailHeader> <n:MessageKeyCode>RQ</n:MessageKeyCode> <j:DocumentSource.Organization> <j:OrganizationORIID> <j:ID>GA0250300</j:ID> </j:OrganizationORIID> </j:DocumentSource.Organization> <n:DocumentDestinationID> <j:ID>AZ</j:ID> </n:DocumentDestinationID> <n:DocumentControlFieldText>TERM000000</n:DocumentControlFieldText> </n:NLETSMailHeader>

18. NLETS GJXDM Data

19. Supporting legacy text values Legacy text may have supported nonconforming data values (example: Expiration Date) Newly standardized transactions Cannot find structure in older, non-standardized formats Must support two versions of XML “3.0” Multiple ways to represent same data GJXDM not meant to support certain types of data Messaging/Header information GJXDM Mapping Challenges

20. Schemas Each message type has its own schema “package” Combined Document/Extension schema Defines overall structure of message – referencing GJXDM elements and defining NLETS-specific elements Global JXDM constraint schema Defines GJXDM elements being used and specifies constraints (min/max occurrences) Shared supporting schemas NCIC, NIBRS code enumerations, etc

21. Schemas

22. Schemas NLETS does not currently perform schema validation on each transaction Table-based validation of data Validate users’ XML in a test environment first, then trust them Schemas are provided primarily as documentation for programmers Recommend use during development and testing Schemas cannot communicate all rules for XML conformance

23. Documentation of XML Specifications XML is defined in detail in Appendix A of the NLETS User Guide For each transaction, schemas, XML instances, stylesheets and business rules are provided

24. NLETS processes, switches data but does not create, store or manipulate data Must parse text and XML in order to route, validate and convert XPath for XML, regular expressions for text

25. Complies with GJXDM V3.0 Interstate Criminal History Transmission Specification V3.0 NLETS will support XML Rap Sheet V3.0 and V.2.2x but no transformations will occur between 3.0 and 2.2 - V2.2x will only be delivered as text Complies with GJXDM V3.0 Interstate Criminal History Transmission Specification V3.0 NLETS will support XML Rap Sheet V3.0 and V.2.2x but no transformations will occur between 3.0 and 2.2 - V2.2x will only be delivered as text

26. Mapping the Rapsheet Similar to the process followed in mapping NLETS transactions While we built from an existing specification, we had more flexibility Benefited from GJXDM data dictionary Data we had not previously thought to include Provides more clear consistency throughout rapsheet Found a greater need to extend elements to restrict allowable values

28. CANDLE Standardized GJXDM XML representation of DMV drivers license and registration responses Unlike NLETS Inquiries and Rapsheets, CANDLE specifications were built from scratch – not existing spec Cannot convert text driver and registration responses to XML

30. Operation Respond Not a criminal justice agency, though information is pertinent to CJ agencies GJXDM did not contain hazardous material elements Created elements in Operation Respond’s namespace, utilizing same naming conventions and GJXDM elements when available

31. Change Management Even small changes affect entire user base User concern about a “moving target” Only plan changes when necessary – or optionally adding value Plan to give states advance notice to the extent possible Work with states to minimize effects

32. User Implementation Allowed users to start slow Specify format preference by message key for each ORI XML can be sent via multiple protocols; not limited to web services Provide support to users Stylesheets, transformations were a necessity because users are and will continue to be at different stages As new services are offered, they may only be available via XML and web services

33. Typos, other minor mistakes Do you fix and risk disrupting users or leave incorrect? Namespaces Has become considerably more complicated than with initial specifications New namespaces (example AA) Multiple GJXDM versions/namespaces Inheritance – differing requirements for same element Support of multiple transports Difficult to utilize SOAP header, WS-Attachments, etc Hindsight… Multiple ways to do everything – each with own pros, cons Problems

34. Benefits Allows NLETS to share more information with users Allows users to customize data presentation via stylesheets XML is non-proprietary Presents enormous cost savings to states

35. Benefits Implementing new services quicker and easier If a data provider already has info in GJXDM XML, we can quickly position ourselves to receive it and provide to users Sharing data within a state is simpler – eliminates the need for XML-to-XML transformations (example: Amber Alert)

36. Lessons Learned Be sensitive to early adopters Recognize that you will need careful change management (“ripple effect”) Consider large issues carefully before beginning message structure, namespaces, schemas Remember that the GJXDM will not provide everything

37. Future Plans Expand membership and services offered Web Services opens new doors to new providers Standardize all NLETS transactions in GJXDM-compliant XML

39. Thank you Kate Silhol NLETS Senior Software Engineer ksilhol@nlets.org (602) 224-0744 www.nlets.org