Data transport standard
Download
1 / 58

Data Transport Standard - PowerPoint PPT Presentation


  • 104 Views
  • Uploaded on
  • Presentation posted in: General

Session 51. Data Transport Standard. Nathan Chitty Gary Allen. Data Transport Issues in Higher Ed. E-mail is not reliable or flexible enough for confidential and time-sensitive data No guarantee of delivery No guarantee of order of delivery for sequence dependent data

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha

Download Presentation

Data Transport Standard

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Session 51

Data Transport Standard

Nathan Chitty

Gary Allen


Data Transport Issues in Higher Ed

  • E-mail is not reliable or flexible enough for confidential and time-sensitive data

    • No guarantee of delivery

    • No guarantee of order of delivery for sequence dependent data

    • No automatic confirmation of receipt or facility for retransmit

    • No synchronous response available

    • Email size limitations


Data Transport Issues in Higher Ed

  • FTP data exchange has own challenges

    • Possible to overwrite earlier files

    • No confirmation of receipt

    • No synchronous response


Data Transport Issues in Higher Ed

  • Encryption today is always separate and subject to its own

    • Issues

    • Maintenance

    • Failures


DTS Addresses These Transport Issues

  • DTS addresses

    • The confirmation issue with a send-receive protocol – confirmation is built in

    • The order of delivery problem by actively delivering and receiving the data – no unconfirmed hand-offs


DTS Addresses These Transport Issues

  • DTS addresses

    • The size problem through data compression

    • The FTP overwrite problem by not using filenames


DTS Addresses These Transport Issues

  • DTS addresses

    • The lack of a synchronous response by building in a required synchronous response, even if only for handling status

    • The encryption issue by using standard HTTPS for encryption – the same technology as for online banking


What is DTS?

  • DTS – Data Transport Standard – is a specification for an adjunct to or a replacement for existing data transport mechanisms

    • PGP / GnuPG encryption

    • SecretAgent w/ Email

    • FTP and SecureFTP


What is DTS?

  • DTS is a specification not a product

  • DTS was developed by the Postsecondary Education Standards Council (PESC) for exchanging data for:

    • Inquiries

    • Reports

    • Transactions


What is DTS?

  • DTS uses Internet technologies to facilitate real time data exchange and transaction processing

  • DTS builds on stable technologies, not specific products

  • DTS, once implemented, reduces programming and per-transaction costs through standardization


DTS Benefits

  • A Web Services implementation

    • Delivery confirmation included – no guessing

    • All requests get a response

    • All submissions get an answer of some kind

  • Facilitates real time data exchange


DTS Benefits

  • Includes automatic data encryption

  • Uses digital signature standards

  • Platform independent

  • Strong authentication with non-repudiation


Benefits To “System Providers”

  • Adds value to schools’ systems

    • Schools want transport added to systems and are generally not concerned with the technologies

  • Easier to build one transport protocol for all recipients

    • Just as CommonRecord created the drive to build one XML format


Benefits to “Service Providers”

  • As everyone implements DTS, the need to support other transports will drop

  • If any school implements DTS, service providers will have to support it

  • Also provides a single communication infrastructure option for internal systems


DTS Specification

  • Specification covers

    • Technical interchange rules and processes

    • Recommended best practices

  • Technical Specification is the pure Simple Object Access Protocol (SOAP) interface

  • Implementation Guide is for both .Net and Java reference implementations


DTS Specification

  • Reference implementation examples are available

  • Specification does not cover

    • Business rules for transaction processing

    • Operational oversight, monitoring or escalation


DTS Technical Workgroup

  • Task: Create a written specification for real-time exchange of data between organizations

    • Meets business requirements

    • Standards based

    • Standard technologies (Java, .Net)

    • Payload Insensitive

    • Secure and reliable


DTS Technologies

  • Global XML Web Service Architecture (GXA), generally accepted as the foundation for building Web Services

    • WSDL (Web Service Definition Language)

    • SOAP (Simple Object Access Protocol)

    • WS-I (Web Service Interoperability)

    • WS-S (Web Service Security)


DTS Technologies

  • WS-Security (Digital Signatures)

    • Strong authentication with non-repudiation

    • X.509 encryption keys and certificate authorities

  • SSL encryption of HTTP streams


Anticipated Architectures

  • Immediate processing

    • Request and processed Result Response

  • “Push/Push” deferred processing

    • Request and Acknowledge Response

    • Request with Result and Acknowledge Response

  • “Push/Pull” deferred processing

    • Request and Acknowledge Response (just send)

    • Request for Result and Result Response


Immediate


“Push/Push” (part 1)


“Push/Push” (part 2)


“Push/Pull” (part 1)


“Push/Pull” (part 2)


DTS Analogy

  • DTS is the definition of the “Pipe” and the structure of its contents

    • The “Pipe” is the internet

    • The content is SOAP

    • The end points/junctions are Web Services

    • The sources are Web Service enabled clients


DTS Analogy

  • DTS defines how others can connect to the “Pipe” already installed

    • Any connections must have certain “threads”

    • Any connections must handle two way traffic independent of how the traffic will be used


DTS Analogy

  • By knowing about the pipe and the type of connections, any “plumber” can use his/her own tools to make connections; just so long as the threads match


How Did We Do It?

  • Created basic HelloWorld service and client

    • Worked interoperable

  • Added simple Headers to HelloWorld

    • Was not interoperable

  • Added complex Header to HelloWorld

    • Was not interoperable


Why SOAP Headers

  • To answer routing and processing expectations without opening the payload

  • Remain payload insensitive

  • Allow extensibility for new processes


DTS SOAP Headers

  • DTSResponseRouting

  • DTSResponseAcknowledge

  • DTSResponsePayloadType

  • DTSResponsePayloadBytes

  • DTSResponseSignature

  • DTSRequestRouting

  • DTSRequestServiceExpectation

  • DTSRequestPayloadType

  • DTSRequestPayloadBytes

  • DTSRequestSignature


Convoluted Filename vs Header Elements

  • A [B] <X.Y.Z:M>

    • A = File Type, B = Encrytption, X.Y.Z = key identifier, M = Unique message ID

  • Encryption unnecessary because using HTTPS

  • DTSRequestPayloadType = A

  • DTSRequestRouting

    • SourceIDSubCode = X, SourceID =Y(.Z)

    • UUID = M


Interop Problem with SOAP Headers

  • xsi:type attribute in Header elements

    • Java includes and requires this attribute

    • .Net does not


<soap:Header>

<DTSRequestPayloadType xsi:type="DTSRequestPayloadType" xmlns="http://www.datatransportstandard.com">

<value>CRC01Request</value>

</DTSRequestPayloadCode>

All about SOAP


SOAP is the Key

  • The SOAP transmitted across the wire is of primary importance

    • Element names

    • Type attribute

    • Not Namespace moniker (Java uses one by default, .Net does not)

  • How you get the correct SOAP is not important


SOAP Differences That Do Not Matter

Java:

<ns1:DTSRequestSignature

soapenv:mustUnderstand="0"

xsi:type="ns1:DTSRequestSignature" xmlns:ns1="http://www.datatransportstandard.com">

<ns1:value>SignatureValue</ns1:value>

</ns1:DTSRequestSignature>

.Net:

<DTSRequestSignature

xsi:type="DTSRequestSignature"xmlns="http://www.datatransportstandard.com">

<value>SignatureValue</value>

</DTSRequestSignature>


Reference Implementation Architecture

  • Client Application

  • Client Core

  • Service Core

  • Service Application


Client Application

  • Knows nothing of SOAP or Web Services

  • Implements Client Core Interface

    • “Setters” and “Getters” of DTS specific elements

  • Houses specific business logic


Client Core

  • Knows nothing of business logic

  • Uses properties set to construct the SOAP

  • Interface for “setting send” and “getting returned” elements

  • Handles the communication to Service Core- DTS Specification


Service Core

  • Accepts transmissions from Client Core

  • Implements Service Application Interface

    • “Setters” and “Getters” of DTS specific elements

  • Creates return SOAP

    • Format return acknowledgement or data from Service Application

    • Construct SOAP faults


Service Core (con’t)

  • Isolated business logic

    • Examples

      • Invoke Service Application based on payload

      • Place payload in “queue”


Service Application

  • Interface for “setting sent” and “getting to be returned” elements

  • Houses specific business logic

  • Knows nothing of SOAP or Web Services


Connecting the layers


Connecting the layers


Connecting the layers


Connecting the layers


Connecting the layers


Connecting the Layers


DTS Version 2

  • New tools

  • No changes to interfaces

  • Reorganization of SOAP Headers

  • Full WS-Security integration


New Tools

  • Axis 1.4 removed requirement for xsi:type attribute

    • Reduces complexity from .Net side

  • Implemented WSS4J and WSE

    • WS-Security integration


Reorganization of Headers

Version 1

Version 2

<DTSRequestRouting

xsi:type="DTSRequestRouting"

xmlns="http://www.datatransportstandard.com">

<UUID>

<transmissionDateTime>

<sourceID>

<sourceIDCode>

<recipientID>

</DTSRequestRouting>

<DTSRequestPayloadType

xsi:type="DTSRequestPayloadType"

xmlns="http://www.datatransportstandard.com">

<value>CRC01Request</value>

</DTSRequestPayloadType>

<DTSRequestPayloadBytes

xsi:type="DTSRequestPayloadBytes"

xmlns="http://www.datatransportstandard.com">

<value>53</value>

</DTSRequestPayloadBytes>

<DTSRequestServiceExpectation

xsi:type="DTSRequestServiceExpectation"

xmlns="http://www.datatransportstandard.com">

<value>Immediate</value>

</DTSRequestServiceExpectation>

<DTSRequestSignature

xsi:type="DTSRequestSignature"

xmlns="http://www.datatransportstandard.com">

<value>someting</value>

</DTSRequestSignature>

<DTSRequestHeader

xmlns:dts="urn:org:pesc:datatransport"

soap:mustUnderstand="1"

soap:actor="urn:org:pesc:datatransport/dts"

xmlns="urn:org:pesc:datatransport">

<dts:DTSRequestRouting>

<dts:DTSUUID>

<dts:DTSSourceID>

<dts:DTSSourceIDSubCode>

<dts:DTSRecipientID>

<dts:DTSRecipientIDSubCode>

</dts:DTSRequestRouting>

<dts:DTSRequestPayloadType>

<dts:DTSRequestPayloadBytes>

<dts:DTSRequestServiceExpectation>

</DTSRequestHeader>

<wsse:Security soap:mustUnderstand="1">

<wsu:Timestamp>

<wsu:Created>

<wsu:Expires>

</wsu:Timestamp>

...

<SignatureValue>

</wsse:Security>


WS-Security

  • Version 1 utilized portions

    • Authenticity of data via Digital Signatures

    • However, proprietary Signature header element

  • Version 2 implements WS-S Header construct

    • Removal of proprietary signature element

    • Inclusion of Security block

    • Inclusion of Binary Security Token element


WS-Security (con’t)

  • Binary Security Token Element

    • Specification allows for the public X.509 certificate to be included in the transmission

    • Since, Certificate Authority signed X.509 certificate

      • No need to exchange public key

      • Trust the certificate chain (CA)


WS-Security

  • .Net

    • Implementing WS-S via WSE in .Net adds WS-Addressing block

    • Creates interoperability problem between .Net and Java

    • SOAP Output handler created to remove it


Technical Wrap Up

  • Version 1 approved by PESC (5/2006)

  • Electronic Exchange Advisory Team (EEAT) within NCHELP- Electronic Standards Committee is currently writing FFEL Commonline, CAM, CR:C implementation rules

  • DTS Technical group is currently writing change proposal for PESC

    • No changes to interfaces, Just Better!


Connecting the Layers


Additional DTS Information

  • Visit PESC at www.pesc.org

  • Materials available include

    • Executive summaries

    • Specifications

    • Reference (proof of concept) implementations


Contact Information

We appreciate your feedback and comments. We can be reached at:

Name:Nathan Chitty, Nelnet, Inc.Phone:904-281-7235Email:nathan.chitty@nelnet.net

Name:Gary Allen, ELM Resources

Phone:510-903-0674

Email:gallen@elmresources.com


ad
  • Login