data transport standard
Download
Skip this Video
Download Presentation
Data Transport Standard

Loading in 2 Seconds...

play fullscreen
1 / 62

Data Transport Standard - PowerPoint PPT Presentation


  • 308 Views
  • Uploaded on

Data Transport Standard Nathan Chitty - PESC Gary Sandler - PESC Data Transport Standard - DTS DTS uses Internet technologies to facilitate real time data exchange and transaction processing DTS builds on stable technologies, not specific products

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Data Transport Standard' - jaden


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
data transport standard

Data Transport Standard

Nathan Chitty - PESC

Gary Sandler - PESC

data transport standard dts
Data Transport Standard - DTS
  • DTS uses Internet technologies to facilitate real time data exchange and transaction processing
  • DTS builds on stable technologies, not specific products
  • DTS, once implemented, reduces programming and per-transaction costs through standardization
dts defined
DTS Defined
  • Data Transport Standard is a specification not a product
  • Established by Postsecondary Education Standards Council (PESC) for exchanging data for:
    • Inquiries
    • Reports
    • Transactions
dts defined4
DTS Defined
  • A specification for an adjunct to or a replacement for existing data transport mechanisms
    • PGP / GnuPG encryption
    • SecretAgent w/ Email
    • FTP and SecureFTP
dts benefits
DTS Benefits
  • A Web Services implementation
    • Delivery confirmation included – no guessing
    • All requests get a response
    • All submissions get an answer of some kind
  • Facilitates real time data exchange
dts benefits6
DTS Benefits
  • Includes automatic data encryption
  • Uses digital signature standards
  • Platform independent
  • Strong authentication with non-repudiation
benefits to system providers
Benefits To “System Providers”
  • Add value to schools’ systems
    • Schools want transport added to systems and are generally not concerned with the technologies
  • Easier to build one transport protocol for all recipients
    • Just as CommonRecord created the drive to build one XML format
benefits to service providers
Benefits to “Service Providers”
  • As everyone implements DTS, the need to support other transports will drop
  • If any school implements DTS, service providers will have to support it
  • Also provides a single communication infrastructure option for internal systems
dts specification
DTS Specification
  • Specification covers
    • Technical interchange rules and processes
    • Recommended best practices
  • Technical Specification is the pure Simple Object Access Protocol (SOAP) interface
  • Implementation Guide is for both .Net and Java reference implementations
dts specification10
DTS Specification
  • Reference implementation examples are available
  • Specification does not cover
    • Business rules for transaction processing
    • Operational oversight, monitoring or escalation
data transport issues in higher ed
Data Transport Issues in Higher Ed
  • E-mail is not reliable or flexible enough
    • No guarantee of delivery
    • No guarantee of order of delivery for sequence dependent data
    • No automatic confirmation of receipt or facility for retransmit
data transport issues in higher ed12
Data Transport Issues in Higher Ed
  • E-mail is not reliable or flexible enough (continued)
    • No synchronous response available
    • Email size limitations
data transport issues in higher ed13
Data Transport Issues in Higher Ed
  • FTP data exchange has own challenges
    • Possible to overwrite earlier files
    • No confirmation of receipt
    • No synchronous response
data transport issues in higher ed14
Data Transport Issues in Higher Ed
  • Encryption today is always separate and subject to its own
    • Issues
    • Maintenance
    • Failures
dts addresses these transport issues
DTS Addresses These Transport Issues
  • DTS addresses
    • The confirmation issue with a send-receive protocol – confirmation is built in
    • The order of delivery problem by actively delivering and receiving the data – no unconfirmed hand-offs
dts addresses these transport issues16
DTS Addresses These Transport Issues
  • DTS addresses
    • The size problem through data compression
    • The FTP overwrite problem by not using filenames
dts addresses these transport issues17
DTS Addresses These Transport Issues
  • DTS addresses
    • The lack of a synchronous response by building in a required synchronous response, even if only for handling status
    • The encryption issue by using standard HTTPS for encryption – the same technology as for online banking
dts technical workgroup
DTS Technical Workgroup
  • Task: Create a written specification for real-time exchange of data between organizations
    • Meets business requirements
    • Standards based
    • Standard technologies (Java, .Net)
    • Payload Insensitive
    • Secure and reliable
dts technologies
DTS Technologies
  • Global XML Web Service Architecture (GXA), generally accepted as the foundation for building Web Services
    • WSDL (Web Service Definition Language)
    • SOAP (Simple Object Access Protocol)
    • WS-I (Web Service Interoperability)
    • WS-S (Web Service Security)
dts technologies20
DTS Technologies
  • WS-Security (Digital Signatures)
    • Strong authentication with non-repudiation
    • X.509 encryption keys and certificate authorities
  • SSL encryption of HTTP streams
anticipated architectures
Anticipated Architectures
  • Immediate processing
    • Request and processed Result Response
  • “Push/Push” deferred processing
    • Request and Acknowledge Response
    • Request with Result and Acknowledge Response
  • “Push/Pull” deferred processing
    • Request and Acknowledge Response (just send)
    • Request for Result and Result Response
dts analogy
DTS Analogy
  • DTS is the definition of the “Pipe” and the structure of its contents
    • The “Pipe” is the internet
    • The content is SOAP
    • The end points/junctions are Web Services
    • The sources are Web Service enabled clients
dts analogy26
DTS Analogy
  • DTS defines how others can connect to the “Pipe” already installed
    • Any connections must have certain “threads”
    • Any connections must handle two way traffic independent of how the traffic will be used
dts analogy27
DTS Analogy
  • By knowing about the pipe and the type of connections, any “plumber” can use his/her own tools to make connections; just so long as the threads match
extending the analogy
Extending the Analogy
  • We all have plumbing and fixtures
  • Very unlikely we all have the same type of fixtures
  • Yet our water companies still deliver to us all
  • All our fixtures use (“process”) it
  • All our drains return it
how did we do it
How Did We Do It?
  • Created basic HelloWorld service and client
    • Worked interoperable
  • Added simple Headers to HelloWorld
    • Was not interoperable
  • Added complex Header to HelloWorld
    • Was not interoperable
why soap headers
Why SOAP Headers
  • To answer routing and processing expectations without opening the payload
  • Remain payload insensitive
  • Allow extensibility for new processes
dts soap headers
DTS SOAP Headers
  • DTSRequestRouting
  • DTSRequestServiceExpectation
  • DTSRequestPayloadType
  • DTSRequestSignature
  • DTSResponseRouting
  • DTSResponseAcknowledge
  • DTSResponsePayloadType
  • DTSResponseSignature
convoluted filename vs header elements
Convoluted Filename vs Header Elements
  • A [B] <X.Y.Z:M>

A = File Type, B = Encrytption, X.Y.Z = key identifier, M = Unique message ID

  • Encryption unnecessary because using HTTPS
  • DTSRequestPayloadType = A
  • DTSRequestRouting
    • SourceIDSubCode = X, SourceID =Y(.Z)
    • UUID = M
interop problem with soap headers
Interop Problem with SOAP Headers
  • xsi:type attribute in Header elements
    • Java includes and requires this attribute
    • .Net does not
all about soap
All about SOAP

<soap:Header>

<DTSRequestPayloadType xsi:type="DTSRequestPayloadType" xmlns="http://www.datatransportstandard.com">

<value>CRC01Request</value>

</DTSRequestPayloadCode>

soap is the key
SOAP is the Key
  • The SOAP transmitted across the wire is of primary importance
    • Element names
    • Type attribute
    • Not Namespace moniker (Java uses one by default, .Net does not)
  • How you get the correct SOAP is not important
soap differences that do not matter
SOAP Differences That Do Not Matter

Java:

<ns1:DTSRequestSignature

soapenv:mustUnderstand="0"

xsi:type="ns1:DTSRequestSignature" xmlns:ns1="http://www.datatransportstandard.com">

<ns1:value>SignatureValue</ns1:value>

</ns1:DTSRequestSignature>

.Net:

<DTSRequestSignature

xsi:type="DTSRequestSignature"xmlns="http://www.datatransportstandard.com">

<value>SignatureValue</value>

</DTSRequestSignature>

reference implementation architecture
Reference Implementation Architecture
  • Client Application
  • Client Core
  • Service Core
  • Service Application
client application
Client Application
  • Knows nothing of SOAP or Web Services
  • Implements Client Core Interface
    • “Setters” and “Getters” of DTS specific elements
  • Houses specific business logic
client core
Client Core
  • Knows nothing of business logic
  • Uses properties set to construct the SOAP
  • Interface for “setting send” and “getting returned” elements
  • Handles the communication to Service Core- DTS Specification
service core
Service Core
  • Accepts transmissions from Client Core
  • Implements Service Application Interface
    • “Setters” and “Getters” of DTS specific elements
  • Creates return SOAP
    • Format return acknowledgement or data from Service Application
    • Construct SOAP faults
service core continued
Service Core (continued)
  • Isolated business logic
    • Examples
      • Invoke Service Application based on payload
      • Place payload in “queue”
service application
Service Application
  • Interface for “setting sent” and “getting to be returned” elements
  • Houses specific business logic
  • Knows nothing of SOAP or Web Services
additional dts information
Additional DTS Information
  • Visit PESC at www.pesc.org
  • Materials available include
    • Executive summaries
    • Specifications
    • Reference (proof of concept) implementations
adding soap headers
Adding SOAP Headers
  • Change WSDL – Still has problems
  • Create Container Classes
  • Container Classes require serialization/de-serialization directives
adding soap headers50
Adding SOAP Headers
  • Augment Service Code
  • Augment Client Code
slide51
package com.datatransportstandard.www.serializable;

import java.io.Serializable;

public class DTSRequestRouting implements Serializable

{

private String sourceID = null;

private String sourceIDCode = null;

private String recipientID = null;

private String recipientIDCode = null;

private String uuid = null;

private String transmissionDateTime = null;

public String getSourceID() { return sourceID; }

public void setSourceID(String newSourceID)

{

sourceID = newSourceID;

}

public String getSourceIDCode() { return sourceIDCode; }

public void setSourceIDCode(String newSourceIDCode)

{

sourceIDCode = newSourceIDCode;

}

public String getRecipientID() { return recipientID; }

public void setRecipientID(String newRecipientID)

{

recipientID = newRecipientID;

}

public String getRecipientIDCode() { return recipientIDCode; }

public void setRecipientIDCode(String newRecipientIDCode)

{

recipientIDCode = newRecipientIDCode;

}

public String getUUID() { return uuid; }

public void setUUID(String newUUID)

{

uuid = newUUID;

}

public String getTransmissionDateTime()

{

return transmissionDateTime;

}

public void setTransmissionDateTime(String newTransDateTime)

{

transmissionDateTime = newTransDateTime;

}

}

Java Creating the Container Classes for the Service (IN)

java creating the container classes for the service out
package com.datatransportstandard.www.serializable;

import java.io.Serializable;

public class DTSResponseRouting implements Serializable

{

private String sourceID = null;

private String sourceIDCode = null;

private String recipientID = null;

private String recipientIDCode = null;

private String uuid = null;

private String transmissionDateTime = null;

public String getSourceID() { return sourceID; }

public void setSourceID(String newSourceID)

{

sourceID = newSourceID;

}

public String getSourceIDCode() { return sourceIDCode; }

public void setSourceIDCode(String newSourceIDCode)

{

sourceIDCode = newSourceIDCode;

}

public String getRecipientID() { return recipientID; }

public void setRecipientID(String newRecipientID)

{

recipientID = newRecipientID;

}

public String getRecipientIDCode() { return recipientIDCode; }

public void setRecipientIDCode(String newRecipientIDCode)

{

recipientIDCode = newRecipientIDCode;

}

public String getUUID() { return uuid; }

public void setUUID(String newUUID)

{

uuid = newUUID;

}

public String getTransmissionDateTime()

{

return transmissionDateTime;

}

public void setTransmissionDateTime(String newTransDateTime)

{

transmissionDateTime = newTransDateTime;

}

}

Java Creating the Container Classes for the Service (OUT)
java examples service
Java Examples (Service)
  • Modify the WSDD

<beanMapping languageSpecificType="java:com.datatransportstandard.www.serializable.DTSRequestRouting" qname="ns2:DTSRequestRouting" xmlns:ns2="http://www.datatransport.com"/>

<beanMapping languageSpecificType="java:com.datatransportstandard.www.serializable.DTSResponseRouting" qname="ns3:DTSResponseRouting" xmlns:ns3="http://www.datatranportstandard.com"/>

  • Augment the Code (inbound)

MessageContext ctx = MessageContext.getCurrentContext();

SOAPEnvelope requestEnv = ctx.getRequestMessage().getSOAPEnvelope();

SOAPHeaderElement requestHeader = requestEnv.getHeaderByName("http://www.datatransportstandard.com",

“DTSRequestRouting");

DTSRequestRouting incontainer = (DTSRequestRouting) requestHeader.getObjectValue();

System.out.println(“SourceId = “ + incontainer.getSourceId());

java examples service54
Java Examples (Service)
  • Augment the Code (outbound)

DTSResponseRouting outcontainer = new DTSResponseRouting();

outcontainer.setSourceId(“TEST SOURCE ID”);

outcontainer.setSourceIdType(“TEST SOURCE ID TYPE”);

SOAPHeaderElement responseHeader = new SOAPHeaderElement(“http://www.datatransportstandard.com”

,“DTSResponseRouting”)

responseHeader.setObjectValue(outcontainer);

SOAPEnvelope responseEnv = ctx.getResponseMessage().getSOAPEnvelope();

responseEnv.addHeader(responseHeader);

java examples client
Java Examples (Client)
  • Augment the Code (General)

QName qn = new QName(“http://www.datatransportstandard.com”,“DTSRequestRouting”);

call.registerTypeMapping(DTSRequestRouting.class, qn,

new BeanSerializerFactory(DTSRequestRouting.class, qn),

new BeanDeserializerFactory(DTSRequestRouting.class, qn));

QName qn1 = new QName(“http://www.datatranpsortstandard.com”,“DTSResponseRouting”);

call.registerTypeMapping(DTSResponseRouting.class, qn1,

new

BeanSerializerFactory(DTSResponseRouting.class, qn1),

new BeanDeserializerFactory(DTSResponseRouting.class, qn1));

java examples client56
Java Examples (Client)
  • Augment the Code (outbound)

DTSRequestRouting outcontainer = new DTSRequestRouting();

outcontainer.setSourceId(“TEST SOURCE ID”);

outcontainer.setSourceIdType(“TEST SOURCE ID TYPE”);

SOAPHeaderElement requestHeader = new SOAPHeaderElement(“http://www.datatransportstandard.com”, “DTSRequestRouting”);

requestHeader.setObjectValue(outcontainer);

call.addHeader(requestHeader);

java examples client57
Java Examples (Client)
  • Augment the Code (inbound)

SOAPEnvelope responseEnv = call.getMessageContext().getResponseMessage().getSOAPEnvelope();

SOAPHeaderElement responseHeader = responseEnv.getHeaderByName(“http://www.datatransportstandard.com”,“DTSResposneHeader”);

DTSResponseHeader incontainer = (DTSResponseHeader) responseHeader.getObjectValue();

System.out.println(“Source Id=“ + intcontainer.getSourceId());

net creating the container classes for service
.Net: Creating the Container classes for Service

[XmlTypeAttribute(Namespace=“http://www.datatransportstandard.com”)]

[XmlRootAttribute(ElementName=“DTSRequestRouting”, Namespace=“http://www.datatransportstandard.com”, IsNullable=false)]

[XmlInclude(typeof(DTSRequestRouting))]

public class DTSRequestRouting : System.Web.Services.Protocols.SoapHeader

{

public string UUID;

public string transmissionDateTime;

public string sourceID;

public string sourceIDCode;

public string recipientID;

}

[XmlTypeAttribute(Namespace=“http://www.datatransportstandard.com”)]

[XmlRootAttribute(ElementName=“DTSResponseRouting”, Namespace=“http://www.datatransportstandard.com”, IsNullable=false)]

[XmlInclude(typeof(DTSResponseRouting))]

public class DTSResponseRoutingElements : System.Web.Services.Protocols.SoapHeader

{

public string UUID;

public string transmissionDateTime;

public string sourceID;

public string sourceIDCode;

public string recipientID;

}

[XmlTypeAttribute(Namespace=“http://www.datatransportstandard.com")]

public class DTSResponseRouting : DTSResponseRoutingElements{};

net augment the service
.Net: Augment the Service
  • Add declarations to service Class

public DTSRequestRouting DTSRequestRoutingVal;

public DTSResponseRoutingElements DTSResponseRoutingVal;

public SoapUnknownHeader[] unknownHeaders;

  • Add serialization directives to WebMethod()

[SoapHeaderAttribute("DTSRequestRoutingVal")]

[SoapHeaderAttribute("DTSResponseRoutingVal", Direction=SoapHeaderDirection.Out)]

[SoapHeader("unknownHeaders")]

net creating the container classes for client
.Net: Creating the Container classes for Client

[XmlTypeAttribute(Namespace="http://www.datatransportstandard.com")]

[XmlRootAttribute(ElementName="DTSRequestRouting",Namespace="http://www.datatransportstandard.com", IsNullable=false)]

[XmlInclude(typeof(DTSRequestRouting))]

public class DTSRequestRoutingElements : System.Web.Services.Protocols.SoapHeader

{

public string UUID;

public string transmissionDateTime;

public string sourceID;

public string sourceIDCode;

public string recipientID;

}

[XmlTypeAttribute(Namespace="http://www.datatransportstandard.com")]

public class DTSRequestRouting : DTSRequestRoutingElements{}

[XmlTypeAttribute(Namespace="http://www.datatransportstandard.com")]

[XmlIncludeAttribute(typeof(DTSResponseRouting))]

[XmlRootAttribute("DTSResponseRouting", Namespace="http://www.datatransportstandard.com", IsNullable=false)]

public class DTSResponseRouting : System.Web.Services.Protocols.SoapHeader

{

public string UUID;

public string transmissionDateTime;

public string sourceID;

public string sourceIDCode;

public string recipientID;

}

net augment the client
.Net: Augment the Client
  • Add declarations to Client Web Reference/Proxy Class

public DTSRequestRoutingElementsDTSRequestRoutingVal;

public DTSResponseRouting DTSResponseRoutingVal;

  • Add serialization directives to WebMethod()

[SoapHeaderAttribute("DTSRequestRoutingVal")]

[SoapHeaderAttribute("DTSResponseRoutingVal", Direction=SoapHeaderDirection.Out)]

contact information
Contact Information

We appreciate your feedback and comments. We can be reached at:

Nathan Chitty, Nelnet, Inc.(904) [email protected]

Gary Sandler, ELM Resources

(510) 903-7960

[email protected]

ad