Data transport standard
Download
1 / 62

Data Transport Standard - PowerPoint PPT Presentation


  • 308 Views
  • Updated On :

Data Transport Standard Nathan Chitty - PESC Gary Sandler - PESC Data Transport Standard - DTS DTS uses Internet technologies to facilitate real time data exchange and transaction processing DTS builds on stable technologies, not specific products

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Data Transport Standard' - jaden


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Data transport standard l.jpg

Data Transport Standard

Nathan Chitty - PESC

Gary Sandler - PESC


Data transport standard dts l.jpg
Data Transport Standard - DTS

  • DTS uses Internet technologies to facilitate real time data exchange and transaction processing

  • DTS builds on stable technologies, not specific products

  • DTS, once implemented, reduces programming and per-transaction costs through standardization


Dts defined l.jpg
DTS Defined

  • Data Transport Standard is a specification not a product

  • Established by Postsecondary Education Standards Council (PESC) for exchanging data for:

    • Inquiries

    • Reports

    • Transactions


Dts defined4 l.jpg
DTS Defined

  • A specification for an adjunct to or a replacement for existing data transport mechanisms

    • PGP / GnuPG encryption

    • SecretAgent w/ Email

    • FTP and SecureFTP


Dts benefits l.jpg
DTS Benefits

  • A Web Services implementation

    • Delivery confirmation included – no guessing

    • All requests get a response

    • All submissions get an answer of some kind

  • Facilitates real time data exchange


Dts benefits6 l.jpg
DTS Benefits

  • Includes automatic data encryption

  • Uses digital signature standards

  • Platform independent

  • Strong authentication with non-repudiation


Benefits to system providers l.jpg
Benefits To “System Providers”

  • Add value to schools’ systems

    • Schools want transport added to systems and are generally not concerned with the technologies

  • Easier to build one transport protocol for all recipients

    • Just as CommonRecord created the drive to build one XML format


Benefits to service providers l.jpg
Benefits to “Service Providers”

  • As everyone implements DTS, the need to support other transports will drop

  • If any school implements DTS, service providers will have to support it

  • Also provides a single communication infrastructure option for internal systems


Dts specification l.jpg
DTS Specification

  • Specification covers

    • Technical interchange rules and processes

    • Recommended best practices

  • Technical Specification is the pure Simple Object Access Protocol (SOAP) interface

  • Implementation Guide is for both .Net and Java reference implementations


Dts specification10 l.jpg
DTS Specification

  • Reference implementation examples are available

  • Specification does not cover

    • Business rules for transaction processing

    • Operational oversight, monitoring or escalation


Data transport issues in higher ed l.jpg
Data Transport Issues in Higher Ed

  • E-mail is not reliable or flexible enough

    • No guarantee of delivery

    • No guarantee of order of delivery for sequence dependent data

    • No automatic confirmation of receipt or facility for retransmit


Data transport issues in higher ed12 l.jpg
Data Transport Issues in Higher Ed

  • E-mail is not reliable or flexible enough (continued)

    • No synchronous response available

    • Email size limitations


Data transport issues in higher ed13 l.jpg
Data Transport Issues in Higher Ed

  • FTP data exchange has own challenges

    • Possible to overwrite earlier files

    • No confirmation of receipt

    • No synchronous response


Data transport issues in higher ed14 l.jpg
Data Transport Issues in Higher Ed

  • Encryption today is always separate and subject to its own

    • Issues

    • Maintenance

    • Failures


Dts addresses these transport issues l.jpg
DTS Addresses These Transport Issues

  • DTS addresses

    • The confirmation issue with a send-receive protocol – confirmation is built in

    • The order of delivery problem by actively delivering and receiving the data – no unconfirmed hand-offs


Dts addresses these transport issues16 l.jpg
DTS Addresses These Transport Issues

  • DTS addresses

    • The size problem through data compression

    • The FTP overwrite problem by not using filenames


Dts addresses these transport issues17 l.jpg
DTS Addresses These Transport Issues

  • DTS addresses

    • The lack of a synchronous response by building in a required synchronous response, even if only for handling status

    • The encryption issue by using standard HTTPS for encryption – the same technology as for online banking


Dts technical workgroup l.jpg
DTS Technical Workgroup

  • Task: Create a written specification for real-time exchange of data between organizations

    • Meets business requirements

    • Standards based

    • Standard technologies (Java, .Net)

    • Payload Insensitive

    • Secure and reliable


Dts technologies l.jpg
DTS Technologies

  • Global XML Web Service Architecture (GXA), generally accepted as the foundation for building Web Services

    • WSDL (Web Service Definition Language)

    • SOAP (Simple Object Access Protocol)

    • WS-I (Web Service Interoperability)

    • WS-S (Web Service Security)


Dts technologies20 l.jpg
DTS Technologies

  • WS-Security (Digital Signatures)

    • Strong authentication with non-repudiation

    • X.509 encryption keys and certificate authorities

  • SSL encryption of HTTP streams


Anticipated architectures l.jpg
Anticipated Architectures

  • Immediate processing

    • Request and processed Result Response

  • “Push/Push” deferred processing

    • Request and Acknowledge Response

    • Request with Result and Acknowledge Response

  • “Push/Pull” deferred processing

    • Request and Acknowledge Response (just send)

    • Request for Result and Result Response





Dts analogy l.jpg
DTS Analogy

  • DTS is the definition of the “Pipe” and the structure of its contents

    • The “Pipe” is the internet

    • The content is SOAP

    • The end points/junctions are Web Services

    • The sources are Web Service enabled clients


Dts analogy26 l.jpg
DTS Analogy

  • DTS defines how others can connect to the “Pipe” already installed

    • Any connections must have certain “threads”

    • Any connections must handle two way traffic independent of how the traffic will be used


Dts analogy27 l.jpg
DTS Analogy

  • By knowing about the pipe and the type of connections, any “plumber” can use his/her own tools to make connections; just so long as the threads match


Extending the analogy l.jpg
Extending the Analogy

  • We all have plumbing and fixtures

  • Very unlikely we all have the same type of fixtures

  • Yet our water companies still deliver to us all

  • All our fixtures use (“process”) it

  • All our drains return it


How did we do it l.jpg
How Did We Do It?

  • Created basic HelloWorld service and client

    • Worked interoperable

  • Added simple Headers to HelloWorld

    • Was not interoperable

  • Added complex Header to HelloWorld

    • Was not interoperable


Why soap headers l.jpg
Why SOAP Headers

  • To answer routing and processing expectations without opening the payload

  • Remain payload insensitive

  • Allow extensibility for new processes


Dts soap headers l.jpg
DTS SOAP Headers

  • DTSRequestRouting

  • DTSRequestServiceExpectation

  • DTSRequestPayloadType

  • DTSRequestSignature

  • DTSResponseRouting

  • DTSResponseAcknowledge

  • DTSResponsePayloadType

  • DTSResponseSignature


Convoluted filename vs header elements l.jpg
Convoluted Filename vs Header Elements

  • A [B] <X.Y.Z:M>

    A = File Type, B = Encrytption, X.Y.Z = key identifier, M = Unique message ID

  • Encryption unnecessary because using HTTPS

  • DTSRequestPayloadType = A

  • DTSRequestRouting

    • SourceIDSubCode = X, SourceID =Y(.Z)

    • UUID = M


Interop problem with soap headers l.jpg
Interop Problem with SOAP Headers

  • xsi:type attribute in Header elements

    • Java includes and requires this attribute

    • .Net does not


All about soap l.jpg
All about SOAP

<soap:Header>

<DTSRequestPayloadType xsi:type="DTSRequestPayloadType" xmlns="http://www.datatransportstandard.com">

<value>CRC01Request</value>

</DTSRequestPayloadCode>


Soap is the key l.jpg
SOAP is the Key

  • The SOAP transmitted across the wire is of primary importance

    • Element names

    • Type attribute

    • Not Namespace moniker (Java uses one by default, .Net does not)

  • How you get the correct SOAP is not important


Soap differences that do not matter l.jpg
SOAP Differences That Do Not Matter

Java:

<ns1:DTSRequestSignature

soapenv:mustUnderstand="0"

xsi:type="ns1:DTSRequestSignature" xmlns:ns1="http://www.datatransportstandard.com">

<ns1:value>SignatureValue</ns1:value>

</ns1:DTSRequestSignature>

.Net:

<DTSRequestSignature

xsi:type="DTSRequestSignature"xmlns="http://www.datatransportstandard.com">

<value>SignatureValue</value>

</DTSRequestSignature>


Reference implementation architecture l.jpg
Reference Implementation Architecture

  • Client Application

  • Client Core

  • Service Core

  • Service Application


Client application l.jpg
Client Application

  • Knows nothing of SOAP or Web Services

  • Implements Client Core Interface

    • “Setters” and “Getters” of DTS specific elements

  • Houses specific business logic


Client core l.jpg
Client Core

  • Knows nothing of business logic

  • Uses properties set to construct the SOAP

  • Interface for “setting send” and “getting returned” elements

  • Handles the communication to Service Core- DTS Specification


Service core l.jpg
Service Core

  • Accepts transmissions from Client Core

  • Implements Service Application Interface

    • “Setters” and “Getters” of DTS specific elements

  • Creates return SOAP

    • Format return acknowledgement or data from Service Application

    • Construct SOAP faults


Service core continued l.jpg
Service Core (continued)

  • Isolated business logic

    • Examples

      • Invoke Service Application based on payload

      • Place payload in “queue”


Service application l.jpg
Service Application

  • Interface for “setting sent” and “getting to be returned” elements

  • Houses specific business logic

  • Knows nothing of SOAP or Web Services







Additional dts information l.jpg
Additional DTS Information

  • Visit PESC at www.pesc.org

  • Materials available include

    • Executive summaries

    • Specifications

    • Reference (proof of concept) implementations


Adding soap headers l.jpg
Adding SOAP Headers

  • Change WSDL – Still has problems

  • Create Container Classes

  • Container Classes require serialization/de-serialization directives


Adding soap headers50 l.jpg
Adding SOAP Headers

  • Augment Service Code

  • Augment Client Code


Slide51 l.jpg

package com.datatransportstandard.www.serializable;

import java.io.Serializable;

public class DTSRequestRouting implements Serializable

{

private String sourceID = null;

private String sourceIDCode = null;

private String recipientID = null;

private String recipientIDCode = null;

private String uuid = null;

private String transmissionDateTime = null;

public String getSourceID() { return sourceID; }

public void setSourceID(String newSourceID)

{

sourceID = newSourceID;

}

public String getSourceIDCode() { return sourceIDCode; }

public void setSourceIDCode(String newSourceIDCode)

{

sourceIDCode = newSourceIDCode;

}

public String getRecipientID() { return recipientID; }

public void setRecipientID(String newRecipientID)

{

recipientID = newRecipientID;

}

public String getRecipientIDCode() { return recipientIDCode; }

public void setRecipientIDCode(String newRecipientIDCode)

{

recipientIDCode = newRecipientIDCode;

}

public String getUUID() { return uuid; }

public void setUUID(String newUUID)

{

uuid = newUUID;

}

public String getTransmissionDateTime()

{

return transmissionDateTime;

}

public void setTransmissionDateTime(String newTransDateTime)

{

transmissionDateTime = newTransDateTime;

}

}

Java Creating the Container Classes for the Service (IN)


Java creating the container classes for the service out l.jpg

package com.datatransportstandard.www.serializable;

import java.io.Serializable;

public class DTSResponseRouting implements Serializable

{

private String sourceID = null;

private String sourceIDCode = null;

private String recipientID = null;

private String recipientIDCode = null;

private String uuid = null;

private String transmissionDateTime = null;

public String getSourceID() { return sourceID; }

public void setSourceID(String newSourceID)

{

sourceID = newSourceID;

}

public String getSourceIDCode() { return sourceIDCode; }

public void setSourceIDCode(String newSourceIDCode)

{

sourceIDCode = newSourceIDCode;

}

public String getRecipientID() { return recipientID; }

public void setRecipientID(String newRecipientID)

{

recipientID = newRecipientID;

}

public String getRecipientIDCode() { return recipientIDCode; }

public void setRecipientIDCode(String newRecipientIDCode)

{

recipientIDCode = newRecipientIDCode;

}

public String getUUID() { return uuid; }

public void setUUID(String newUUID)

{

uuid = newUUID;

}

public String getTransmissionDateTime()

{

return transmissionDateTime;

}

public void setTransmissionDateTime(String newTransDateTime)

{

transmissionDateTime = newTransDateTime;

}

}

Java Creating the Container Classes for the Service (OUT)


Java examples service l.jpg
Java Examples (Service)

  • Modify the WSDD

    <beanMapping languageSpecificType="java:com.datatransportstandard.www.serializable.DTSRequestRouting" qname="ns2:DTSRequestRouting" xmlns:ns2="http://www.datatransport.com"/>

    <beanMapping languageSpecificType="java:com.datatransportstandard.www.serializable.DTSResponseRouting" qname="ns3:DTSResponseRouting" xmlns:ns3="http://www.datatranportstandard.com"/>

  • Augment the Code (inbound)

    MessageContext ctx = MessageContext.getCurrentContext();

    SOAPEnvelope requestEnv = ctx.getRequestMessage().getSOAPEnvelope();

    SOAPHeaderElement requestHeader = requestEnv.getHeaderByName("http://www.datatransportstandard.com",

    “DTSRequestRouting");

    DTSRequestRouting incontainer = (DTSRequestRouting) requestHeader.getObjectValue();

    System.out.println(“SourceId = “ + incontainer.getSourceId());


Java examples service54 l.jpg
Java Examples (Service)

  • Augment the Code (outbound)

    DTSResponseRouting outcontainer = new DTSResponseRouting();

    outcontainer.setSourceId(“TEST SOURCE ID”);

    outcontainer.setSourceIdType(“TEST SOURCE ID TYPE”);

    SOAPHeaderElement responseHeader = new SOAPHeaderElement(“http://www.datatransportstandard.com”

    ,“DTSResponseRouting”)

    responseHeader.setObjectValue(outcontainer);

    SOAPEnvelope responseEnv = ctx.getResponseMessage().getSOAPEnvelope();

    responseEnv.addHeader(responseHeader);


Java examples client l.jpg
Java Examples (Client)

  • Augment the Code (General)

    QName qn = new QName(“http://www.datatransportstandard.com”,“DTSRequestRouting”);

    call.registerTypeMapping(DTSRequestRouting.class, qn,

    new BeanSerializerFactory(DTSRequestRouting.class, qn),

    new BeanDeserializerFactory(DTSRequestRouting.class, qn));

    QName qn1 = new QName(“http://www.datatranpsortstandard.com”,“DTSResponseRouting”);

    call.registerTypeMapping(DTSResponseRouting.class, qn1,

    new

    BeanSerializerFactory(DTSResponseRouting.class, qn1),

    new BeanDeserializerFactory(DTSResponseRouting.class, qn1));


Java examples client56 l.jpg
Java Examples (Client)

  • Augment the Code (outbound)

    DTSRequestRouting outcontainer = new DTSRequestRouting();

    outcontainer.setSourceId(“TEST SOURCE ID”);

    outcontainer.setSourceIdType(“TEST SOURCE ID TYPE”);

    SOAPHeaderElement requestHeader = new SOAPHeaderElement(“http://www.datatransportstandard.com”, “DTSRequestRouting”);

    requestHeader.setObjectValue(outcontainer);

    call.addHeader(requestHeader);


Java examples client57 l.jpg
Java Examples (Client)

  • Augment the Code (inbound)

    SOAPEnvelope responseEnv = call.getMessageContext().getResponseMessage().getSOAPEnvelope();

    SOAPHeaderElement responseHeader = responseEnv.getHeaderByName(“http://www.datatransportstandard.com”,“DTSResposneHeader”);

    DTSResponseHeader incontainer = (DTSResponseHeader) responseHeader.getObjectValue();

    System.out.println(“Source Id=“ + intcontainer.getSourceId());


Net creating the container classes for service l.jpg
.Net: Creating the Container classes for Service

[XmlTypeAttribute(Namespace=“http://www.datatransportstandard.com”)]

[XmlRootAttribute(ElementName=“DTSRequestRouting”, Namespace=“http://www.datatransportstandard.com”, IsNullable=false)]

[XmlInclude(typeof(DTSRequestRouting))]

public class DTSRequestRouting : System.Web.Services.Protocols.SoapHeader

{

public string UUID;

public string transmissionDateTime;

public string sourceID;

public string sourceIDCode;

public string recipientID;

}

[XmlTypeAttribute(Namespace=“http://www.datatransportstandard.com”)]

[XmlRootAttribute(ElementName=“DTSResponseRouting”, Namespace=“http://www.datatransportstandard.com”, IsNullable=false)]

[XmlInclude(typeof(DTSResponseRouting))]

public class DTSResponseRoutingElements : System.Web.Services.Protocols.SoapHeader

{

public string UUID;

public string transmissionDateTime;

public string sourceID;

public string sourceIDCode;

public string recipientID;

}

[XmlTypeAttribute(Namespace=“http://www.datatransportstandard.com")]

public class DTSResponseRouting : DTSResponseRoutingElements{};


Net augment the service l.jpg
.Net: Augment the Service

  • Add declarations to service Class

    public DTSRequestRouting DTSRequestRoutingVal;

    public DTSResponseRoutingElements DTSResponseRoutingVal;

    public SoapUnknownHeader[] unknownHeaders;

  • Add serialization directives to WebMethod()

    [SoapHeaderAttribute("DTSRequestRoutingVal")]

    [SoapHeaderAttribute("DTSResponseRoutingVal", Direction=SoapHeaderDirection.Out)]

    [SoapHeader("unknownHeaders")]


Net creating the container classes for client l.jpg
.Net: Creating the Container classes for Client

[XmlTypeAttribute(Namespace="http://www.datatransportstandard.com")]

[XmlRootAttribute(ElementName="DTSRequestRouting",Namespace="http://www.datatransportstandard.com", IsNullable=false)]

[XmlInclude(typeof(DTSRequestRouting))]

public class DTSRequestRoutingElements : System.Web.Services.Protocols.SoapHeader

{

public string UUID;

public string transmissionDateTime;

public string sourceID;

public string sourceIDCode;

public string recipientID;

}

[XmlTypeAttribute(Namespace="http://www.datatransportstandard.com")]

public class DTSRequestRouting : DTSRequestRoutingElements{}

[XmlTypeAttribute(Namespace="http://www.datatransportstandard.com")]

[XmlIncludeAttribute(typeof(DTSResponseRouting))]

[XmlRootAttribute("DTSResponseRouting", Namespace="http://www.datatransportstandard.com", IsNullable=false)]

public class DTSResponseRouting : System.Web.Services.Protocols.SoapHeader

{

public string UUID;

public string transmissionDateTime;

public string sourceID;

public string sourceIDCode;

public string recipientID;

}


Net augment the client l.jpg
.Net: Augment the Client

  • Add declarations to Client Web Reference/Proxy Class

    public DTSRequestRoutingElementsDTSRequestRoutingVal;

    public DTSResponseRouting DTSResponseRoutingVal;

  • Add serialization directives to WebMethod()

    [SoapHeaderAttribute("DTSRequestRoutingVal")]

    [SoapHeaderAttribute("DTSResponseRoutingVal", Direction=SoapHeaderDirection.Out)]


Contact information l.jpg
Contact Information

We appreciate your feedback and comments. We can be reached at:

Nathan Chitty, Nelnet, Inc.(904) [email protected]

Gary Sandler, ELM Resources

(510) 903-7960

[email protected]


ad