Data Transport Standard - PowerPoint PPT Presentation

Data transport standard l.jpg
Download
1 / 62

Data Transport Standard Nathan Chitty - PESC Gary Sandler - PESC Data Transport Standard - DTS DTS uses Internet technologies to facilitate real time data exchange and transaction processing DTS builds on stable technologies, not specific products

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Data Transport Standard

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Data transport standard l.jpg

Data Transport Standard

Nathan Chitty - PESC

Gary Sandler - PESC


Data transport standard dts l.jpg

Data Transport Standard - DTS

  • DTS uses Internet technologies to facilitate real time data exchange and transaction processing

  • DTS builds on stable technologies, not specific products

  • DTS, once implemented, reduces programming and per-transaction costs through standardization


Dts defined l.jpg

DTS Defined

  • Data Transport Standard is a specification not a product

  • Established by Postsecondary Education Standards Council (PESC) for exchanging data for:

    • Inquiries

    • Reports

    • Transactions


Dts defined4 l.jpg

DTS Defined

  • A specification for an adjunct to or a replacement for existing data transport mechanisms

    • PGP / GnuPG encryption

    • SecretAgent w/ Email

    • FTP and SecureFTP


Dts benefits l.jpg

DTS Benefits

  • A Web Services implementation

    • Delivery confirmation included – no guessing

    • All requests get a response

    • All submissions get an answer of some kind

  • Facilitates real time data exchange


Dts benefits6 l.jpg

DTS Benefits

  • Includes automatic data encryption

  • Uses digital signature standards

  • Platform independent

  • Strong authentication with non-repudiation


Benefits to system providers l.jpg

Benefits To “System Providers”

  • Add value to schools’ systems

    • Schools want transport added to systems and are generally not concerned with the technologies

  • Easier to build one transport protocol for all recipients

    • Just as CommonRecord created the drive to build one XML format


Benefits to service providers l.jpg

Benefits to “Service Providers”

  • As everyone implements DTS, the need to support other transports will drop

  • If any school implements DTS, service providers will have to support it

  • Also provides a single communication infrastructure option for internal systems


Dts specification l.jpg

DTS Specification

  • Specification covers

    • Technical interchange rules and processes

    • Recommended best practices

  • Technical Specification is the pure Simple Object Access Protocol (SOAP) interface

  • Implementation Guide is for both .Net and Java reference implementations


Dts specification10 l.jpg

DTS Specification

  • Reference implementation examples are available

  • Specification does not cover

    • Business rules for transaction processing

    • Operational oversight, monitoring or escalation


Data transport issues in higher ed l.jpg

Data Transport Issues in Higher Ed

  • E-mail is not reliable or flexible enough

    • No guarantee of delivery

    • No guarantee of order of delivery for sequence dependent data

    • No automatic confirmation of receipt or facility for retransmit


Data transport issues in higher ed12 l.jpg

Data Transport Issues in Higher Ed

  • E-mail is not reliable or flexible enough (continued)

    • No synchronous response available

    • Email size limitations


Data transport issues in higher ed13 l.jpg

Data Transport Issues in Higher Ed

  • FTP data exchange has own challenges

    • Possible to overwrite earlier files

    • No confirmation of receipt

    • No synchronous response


Data transport issues in higher ed14 l.jpg

Data Transport Issues in Higher Ed

  • Encryption today is always separate and subject to its own

    • Issues

    • Maintenance

    • Failures


Dts addresses these transport issues l.jpg

DTS Addresses These Transport Issues

  • DTS addresses

    • The confirmation issue with a send-receive protocol – confirmation is built in

    • The order of delivery problem by actively delivering and receiving the data – no unconfirmed hand-offs


Dts addresses these transport issues16 l.jpg

DTS Addresses These Transport Issues

  • DTS addresses

    • The size problem through data compression

    • The FTP overwrite problem by not using filenames


Dts addresses these transport issues17 l.jpg

DTS Addresses These Transport Issues

  • DTS addresses

    • The lack of a synchronous response by building in a required synchronous response, even if only for handling status

    • The encryption issue by using standard HTTPS for encryption – the same technology as for online banking


Dts technical workgroup l.jpg

DTS Technical Workgroup

  • Task: Create a written specification for real-time exchange of data between organizations

    • Meets business requirements

    • Standards based

    • Standard technologies (Java, .Net)

    • Payload Insensitive

    • Secure and reliable


Dts technologies l.jpg

DTS Technologies

  • Global XML Web Service Architecture (GXA), generally accepted as the foundation for building Web Services

    • WSDL (Web Service Definition Language)

    • SOAP (Simple Object Access Protocol)

    • WS-I (Web Service Interoperability)

    • WS-S (Web Service Security)


Dts technologies20 l.jpg

DTS Technologies

  • WS-Security (Digital Signatures)

    • Strong authentication with non-repudiation

    • X.509 encryption keys and certificate authorities

  • SSL encryption of HTTP streams


Anticipated architectures l.jpg

Anticipated Architectures

  • Immediate processing

    • Request and processed Result Response

  • “Push/Push” deferred processing

    • Request and Acknowledge Response

    • Request with Result and Acknowledge Response

  • “Push/Pull” deferred processing

    • Request and Acknowledge Response (just send)

    • Request for Result and Result Response


Immediate l.jpg

Immediate


Push push l.jpg

“Push/Push”


Push pull l.jpg

Push/Pull”


Dts analogy l.jpg

DTS Analogy

  • DTS is the definition of the “Pipe” and the structure of its contents

    • The “Pipe” is the internet

    • The content is SOAP

    • The end points/junctions are Web Services

    • The sources are Web Service enabled clients


Dts analogy26 l.jpg

DTS Analogy

  • DTS defines how others can connect to the “Pipe” already installed

    • Any connections must have certain “threads”

    • Any connections must handle two way traffic independent of how the traffic will be used


Dts analogy27 l.jpg

DTS Analogy

  • By knowing about the pipe and the type of connections, any “plumber” can use his/her own tools to make connections; just so long as the threads match


Extending the analogy l.jpg

Extending the Analogy

  • We all have plumbing and fixtures

  • Very unlikely we all have the same type of fixtures

  • Yet our water companies still deliver to us all

  • All our fixtures use (“process”) it

  • All our drains return it


How did we do it l.jpg

How Did We Do It?

  • Created basic HelloWorld service and client

    • Worked interoperable

  • Added simple Headers to HelloWorld

    • Was not interoperable

  • Added complex Header to HelloWorld

    • Was not interoperable


Why soap headers l.jpg

Why SOAP Headers

  • To answer routing and processing expectations without opening the payload

  • Remain payload insensitive

  • Allow extensibility for new processes


Dts soap headers l.jpg

DTS SOAP Headers

  • DTSRequestRouting

  • DTSRequestServiceExpectation

  • DTSRequestPayloadType

  • DTSRequestSignature

  • DTSResponseRouting

  • DTSResponseAcknowledge

  • DTSResponsePayloadType

  • DTSResponseSignature


Convoluted filename vs header elements l.jpg

Convoluted Filename vs Header Elements

  • A [B] <X.Y.Z:M>

    A = File Type, B = Encrytption, X.Y.Z = key identifier, M = Unique message ID

  • Encryption unnecessary because using HTTPS

  • DTSRequestPayloadType = A

  • DTSRequestRouting

    • SourceIDSubCode = X, SourceID =Y(.Z)

    • UUID = M


Interop problem with soap headers l.jpg

Interop Problem with SOAP Headers

  • xsi:type attribute in Header elements

    • Java includes and requires this attribute

    • .Net does not


All about soap l.jpg

All about SOAP

<soap:Header>

<DTSRequestPayloadType xsi:type="DTSRequestPayloadType" xmlns="http://www.datatransportstandard.com">

<value>CRC01Request</value>

</DTSRequestPayloadCode>


Soap is the key l.jpg

SOAP is the Key

  • The SOAP transmitted across the wire is of primary importance

    • Element names

    • Type attribute

    • Not Namespace moniker (Java uses one by default, .Net does not)

  • How you get the correct SOAP is not important


Soap differences that do not matter l.jpg

SOAP Differences That Do Not Matter

Java:

<ns1:DTSRequestSignature

soapenv:mustUnderstand="0"

xsi:type="ns1:DTSRequestSignature" xmlns:ns1="http://www.datatransportstandard.com">

<ns1:value>SignatureValue</ns1:value>

</ns1:DTSRequestSignature>

.Net:

<DTSRequestSignature

xsi:type="DTSRequestSignature"xmlns="http://www.datatransportstandard.com">

<value>SignatureValue</value>

</DTSRequestSignature>


Reference implementation architecture l.jpg

Reference Implementation Architecture

  • Client Application

  • Client Core

  • Service Core

  • Service Application


Client application l.jpg

Client Application

  • Knows nothing of SOAP or Web Services

  • Implements Client Core Interface

    • “Setters” and “Getters” of DTS specific elements

  • Houses specific business logic


Client core l.jpg

Client Core

  • Knows nothing of business logic

  • Uses properties set to construct the SOAP

  • Interface for “setting send” and “getting returned” elements

  • Handles the communication to Service Core- DTS Specification


Service core l.jpg

Service Core

  • Accepts transmissions from Client Core

  • Implements Service Application Interface

    • “Setters” and “Getters” of DTS specific elements

  • Creates return SOAP

    • Format return acknowledgement or data from Service Application

    • Construct SOAP faults


Service core continued l.jpg

Service Core (continued)

  • Isolated business logic

    • Examples

      • Invoke Service Application based on payload

      • Place payload in “queue”


Service application l.jpg

Service Application

  • Interface for “setting sent” and “getting to be returned” elements

  • Houses specific business logic

  • Knows nothing of SOAP or Web Services


Connecting the layers l.jpg

Connecting the layers


Connecting the layers44 l.jpg

Connecting the layers


Connecting the layers45 l.jpg

Connecting the layers


Connecting the layers46 l.jpg

Connecting the layers


Connecting the layers47 l.jpg

Connecting the layers


Additional dts information l.jpg

Additional DTS Information

  • Visit PESC at www.pesc.org

  • Materials available include

    • Executive summaries

    • Specifications

    • Reference (proof of concept) implementations


Adding soap headers l.jpg

Adding SOAP Headers

  • Change WSDL – Still has problems

  • Create Container Classes

  • Container Classes require serialization/de-serialization directives


Adding soap headers50 l.jpg

Adding SOAP Headers

  • Augment Service Code

  • Augment Client Code


Slide51 l.jpg

package com.datatransportstandard.www.serializable;

import java.io.Serializable;

public class DTSRequestRouting implements Serializable

{

private String sourceID = null;

private String sourceIDCode= null;

private String recipientID= null;

private String recipientIDCode= null;

private String uuid= null;

private String transmissionDateTime= null;

public String getSourceID() { return sourceID;}

public void setSourceID(String newSourceID)

{

sourceID = newSourceID;

}

public String getSourceIDCode() { return sourceIDCode; }

public void setSourceIDCode(String newSourceIDCode)

{

sourceIDCode = newSourceIDCode;

}

public String getRecipientID() { return recipientID; }

public void setRecipientID(String newRecipientID)

{

recipientID = newRecipientID;

}

public String getRecipientIDCode() { return recipientIDCode; }

public void setRecipientIDCode(String newRecipientIDCode)

{

recipientIDCode = newRecipientIDCode;

}

public String getUUID() { return uuid; }

public void setUUID(String newUUID)

{

uuid = newUUID;

}

public String getTransmissionDateTime()

{

return transmissionDateTime;

}

public void setTransmissionDateTime(String newTransDateTime)

{

transmissionDateTime = newTransDateTime;

}

}

Java Creating the Container Classes for the Service (IN)


Java creating the container classes for the service out l.jpg

package com.datatransportstandard.www.serializable;

import java.io.Serializable;

public class DTSResponseRouting implements Serializable

{

private String sourceID = null;

private String sourceIDCode= null;

private String recipientID= null;

private String recipientIDCode= null;

private String uuid= null;

private String transmissionDateTime= null;

public String getSourceID() { return sourceID;}

public void setSourceID(String newSourceID)

{

sourceID = newSourceID;

}

public String getSourceIDCode() { return sourceIDCode; }

public void setSourceIDCode(String newSourceIDCode)

{

sourceIDCode = newSourceIDCode;

}

public String getRecipientID() { return recipientID; }

public void setRecipientID(String newRecipientID)

{

recipientID = newRecipientID;

}

public String getRecipientIDCode() { return recipientIDCode; }

public void setRecipientIDCode(String newRecipientIDCode)

{

recipientIDCode = newRecipientIDCode;

}

public String getUUID() { return uuid; }

public void setUUID(String newUUID)

{

uuid = newUUID;

}

public String getTransmissionDateTime()

{

return transmissionDateTime;

}

public void setTransmissionDateTime(String newTransDateTime)

{

transmissionDateTime = newTransDateTime;

}

}

Java Creating the Container Classes for the Service (OUT)


Java examples service l.jpg

Java Examples (Service)

  • Modify the WSDD

    <beanMapping languageSpecificType="java:com.datatransportstandard.www.serializable.DTSRequestRouting" qname="ns2:DTSRequestRouting" xmlns:ns2="http://www.datatransport.com"/>

    <beanMapping languageSpecificType="java:com.datatransportstandard.www.serializable.DTSResponseRouting" qname="ns3:DTSResponseRouting" xmlns:ns3="http://www.datatranportstandard.com"/>

  • Augment the Code (inbound)

    MessageContext ctx = MessageContext.getCurrentContext();

    SOAPEnvelope requestEnv = ctx.getRequestMessage().getSOAPEnvelope();

    SOAPHeaderElement requestHeader = requestEnv.getHeaderByName("http://www.datatransportstandard.com",

    “DTSRequestRouting");

    DTSRequestRouting incontainer = (DTSRequestRouting) requestHeader.getObjectValue();

    System.out.println(“SourceId = “ + incontainer.getSourceId());


Java examples service54 l.jpg

Java Examples (Service)

  • Augment the Code (outbound)

    DTSResponseRouting outcontainer = new DTSResponseRouting();

    outcontainer.setSourceId(“TEST SOURCE ID”);

    outcontainer.setSourceIdType(“TEST SOURCE ID TYPE”);

    SOAPHeaderElement responseHeader = new SOAPHeaderElement(“http://www.datatransportstandard.com”

    ,“DTSResponseRouting”)

    responseHeader.setObjectValue(outcontainer);

    SOAPEnvelope responseEnv = ctx.getResponseMessage().getSOAPEnvelope();

    responseEnv.addHeader(responseHeader);


Java examples client l.jpg

Java Examples (Client)

  • Augment the Code (General)

    QName qn = new QName(“http://www.datatransportstandard.com”,“DTSRequestRouting”);

    call.registerTypeMapping(DTSRequestRouting.class, qn,

    new BeanSerializerFactory(DTSRequestRouting.class, qn),

    new BeanDeserializerFactory(DTSRequestRouting.class, qn));

    QName qn1 = new QName(“http://www.datatranpsortstandard.com”,“DTSResponseRouting”);

    call.registerTypeMapping(DTSResponseRouting.class, qn1,

    new

    BeanSerializerFactory(DTSResponseRouting.class, qn1),

    new BeanDeserializerFactory(DTSResponseRouting.class, qn1));


Java examples client56 l.jpg

Java Examples (Client)

  • Augment the Code (outbound)

    DTSRequestRouting outcontainer = new DTSRequestRouting();

    outcontainer.setSourceId(“TEST SOURCE ID”);

    outcontainer.setSourceIdType(“TEST SOURCE ID TYPE”);

    SOAPHeaderElement requestHeader = new SOAPHeaderElement(“http://www.datatransportstandard.com”, “DTSRequestRouting”);

    requestHeader.setObjectValue(outcontainer);

    call.addHeader(requestHeader);


Java examples client57 l.jpg

Java Examples (Client)

  • Augment the Code (inbound)

    SOAPEnvelope responseEnv = call.getMessageContext().getResponseMessage().getSOAPEnvelope();

    SOAPHeaderElement responseHeader = responseEnv.getHeaderByName(“http://www.datatransportstandard.com”,“DTSResposneHeader”);

    DTSResponseHeader incontainer = (DTSResponseHeader) responseHeader.getObjectValue();

    System.out.println(“Source Id=“ + intcontainer.getSourceId());


Net creating the container classes for service l.jpg

.Net: Creating the Container classes for Service

[XmlTypeAttribute(Namespace=“http://www.datatransportstandard.com”)]

[XmlRootAttribute(ElementName=“DTSRequestRouting”, Namespace=“http://www.datatransportstandard.com”, IsNullable=false)]

[XmlInclude(typeof(DTSRequestRouting))]

public class DTSRequestRouting : System.Web.Services.Protocols.SoapHeader

{

public string UUID;

public string transmissionDateTime;

public string sourceID;

public string sourceIDCode;

public string recipientID;

}

[XmlTypeAttribute(Namespace=“http://www.datatransportstandard.com”)]

[XmlRootAttribute(ElementName=“DTSResponseRouting”, Namespace=“http://www.datatransportstandard.com”, IsNullable=false)]

[XmlInclude(typeof(DTSResponseRouting))]

public class DTSResponseRoutingElements : System.Web.Services.Protocols.SoapHeader

{

public string UUID;

public string transmissionDateTime;

public string sourceID;

public string sourceIDCode;

public string recipientID;

}

[XmlTypeAttribute(Namespace=“http://www.datatransportstandard.com")]

public class DTSResponseRouting : DTSResponseRoutingElements{};


Net augment the service l.jpg

.Net: Augment the Service

  • Add declarations to service Class

    public DTSRequestRoutingDTSRequestRoutingVal;

    public DTSResponseRoutingElementsDTSResponseRoutingVal;

    public SoapUnknownHeader[] unknownHeaders;

  • Add serialization directives to WebMethod()

    [SoapHeaderAttribute("DTSRequestRoutingVal")]

    [SoapHeaderAttribute("DTSResponseRoutingVal", Direction=SoapHeaderDirection.Out)]

    [SoapHeader("unknownHeaders")]


Net creating the container classes for client l.jpg

.Net: Creating the Container classes for Client

[XmlTypeAttribute(Namespace="http://www.datatransportstandard.com")]

[XmlRootAttribute(ElementName="DTSRequestRouting",Namespace="http://www.datatransportstandard.com", IsNullable=false)]

[XmlInclude(typeof(DTSRequestRouting))]

public class DTSRequestRoutingElements : System.Web.Services.Protocols.SoapHeader

{

public string UUID;

public string transmissionDateTime;

public string sourceID;

public string sourceIDCode;

public string recipientID;

}

[XmlTypeAttribute(Namespace="http://www.datatransportstandard.com")]

public class DTSRequestRouting : DTSRequestRoutingElements{}

[XmlTypeAttribute(Namespace="http://www.datatransportstandard.com")]

[XmlIncludeAttribute(typeof(DTSResponseRouting))]

[XmlRootAttribute("DTSResponseRouting", Namespace="http://www.datatransportstandard.com", IsNullable=false)]

public class DTSResponseRouting : System.Web.Services.Protocols.SoapHeader

{

public string UUID;

public string transmissionDateTime;

public string sourceID;

public string sourceIDCode;

public string recipientID;

}


Net augment the client l.jpg

.Net: Augment the Client

  • Add declarations to Client Web Reference/Proxy Class

    public DTSRequestRoutingElementsDTSRequestRoutingVal;

    public DTSResponseRoutingDTSResponseRoutingVal;

  • Add serialization directives to WebMethod()

    [SoapHeaderAttribute("DTSRequestRoutingVal")]

    [SoapHeaderAttribute("DTSResponseRoutingVal", Direction=SoapHeaderDirection.Out)]


Contact information l.jpg

Contact Information

We appreciate your feedback and comments. We can be reached at:

Nathan Chitty, Nelnet, Inc.(904) 281-7235nathan.chitty@nelnet.net

Gary Sandler, ELM Resources

(510) 903-7960

gsandler@elmresources.com


  • Login