580 likes | 724 Views
Session 51. Data Transport Standard. Nathan Chitty Gary Allen. Data Transport Issues in Higher Ed. E-mail is not reliable or flexible enough for confidential and time-sensitive data No guarantee of delivery No guarantee of order of delivery for sequence dependent data
E N D
Session 51 Data Transport Standard Nathan Chitty Gary Allen
Data Transport Issues in Higher Ed • E-mail is not reliable or flexible enough for confidential and time-sensitive data • No guarantee of delivery • No guarantee of order of delivery for sequence dependent data • No automatic confirmation of receipt or facility for retransmit • No synchronous response available • Email size limitations
Data Transport Issues in Higher Ed • FTP data exchange has own challenges • Possible to overwrite earlier files • No confirmation of receipt • No synchronous response
Data Transport Issues in Higher Ed • Encryption today is always separate and subject to its own • Issues • Maintenance • Failures
DTS Addresses These Transport Issues • DTS addresses • The confirmation issue with a send-receive protocol – confirmation is built in • The order of delivery problem by actively delivering and receiving the data – no unconfirmed hand-offs
DTS Addresses These Transport Issues • DTS addresses • The size problem through data compression • The FTP overwrite problem by not using filenames
DTS Addresses These Transport Issues • DTS addresses • The lack of a synchronous response by building in a required synchronous response, even if only for handling status • The encryption issue by using standard HTTPS for encryption – the same technology as for online banking
What is DTS? • DTS – Data Transport Standard – is a specification for an adjunct to or a replacement for existing data transport mechanisms • PGP / GnuPG encryption • SecretAgent w/ Email • FTP and SecureFTP
What is DTS? • DTS is a specification not a product • DTS was developed by the Postsecondary Education Standards Council (PESC) for exchanging data for: • Inquiries • Reports • Transactions
What is DTS? • DTS uses Internet technologies to facilitate real time data exchange and transaction processing • DTS builds on stable technologies, not specific products • DTS, once implemented, reduces programming and per-transaction costs through standardization
DTS Benefits • A Web Services implementation • Delivery confirmation included – no guessing • All requests get a response • All submissions get an answer of some kind • Facilitates real time data exchange
DTS Benefits • Includes automatic data encryption • Uses digital signature standards • Platform independent • Strong authentication with non-repudiation
Benefits To “System Providers” • Adds value to schools’ systems • Schools want transport added to systems and are generally not concerned with the technologies • Easier to build one transport protocol for all recipients • Just as CommonRecord created the drive to build one XML format
Benefits to “Service Providers” • As everyone implements DTS, the need to support other transports will drop • If any school implements DTS, service providers will have to support it • Also provides a single communication infrastructure option for internal systems
DTS Specification • Specification covers • Technical interchange rules and processes • Recommended best practices • Technical Specification is the pure Simple Object Access Protocol (SOAP) interface • Implementation Guide is for both .Net and Java reference implementations
DTS Specification • Reference implementation examples are available • Specification does not cover • Business rules for transaction processing • Operational oversight, monitoring or escalation
DTS Technical Workgroup • Task: Create a written specification for real-time exchange of data between organizations • Meets business requirements • Standards based • Standard technologies (Java, .Net) • Payload Insensitive • Secure and reliable
DTS Technologies • Global XML Web Service Architecture (GXA), generally accepted as the foundation for building Web Services • WSDL (Web Service Definition Language) • SOAP (Simple Object Access Protocol) • WS-I (Web Service Interoperability) • WS-S (Web Service Security)
DTS Technologies • WS-Security (Digital Signatures) • Strong authentication with non-repudiation • X.509 encryption keys and certificate authorities • SSL encryption of HTTP streams
Anticipated Architectures • Immediate processing • Request and processed Result Response • “Push/Push” deferred processing • Request and Acknowledge Response • Request with Result and Acknowledge Response • “Push/Pull” deferred processing • Request and Acknowledge Response (just send) • Request for Result and Result Response
DTS Analogy • DTS is the definition of the “Pipe” and the structure of its contents • The “Pipe” is the internet • The content is SOAP • The end points/junctions are Web Services • The sources are Web Service enabled clients
DTS Analogy • DTS defines how others can connect to the “Pipe” already installed • Any connections must have certain “threads” • Any connections must handle two way traffic independent of how the traffic will be used
DTS Analogy • By knowing about the pipe and the type of connections, any “plumber” can use his/her own tools to make connections; just so long as the threads match
How Did We Do It? • Created basic HelloWorld service and client • Worked interoperable • Added simple Headers to HelloWorld • Was not interoperable • Added complex Header to HelloWorld • Was not interoperable
Why SOAP Headers • To answer routing and processing expectations without opening the payload • Remain payload insensitive • Allow extensibility for new processes
DTS SOAP Headers • DTSResponseRouting • DTSResponseAcknowledge • DTSResponsePayloadType • DTSResponsePayloadBytes • DTSResponseSignature • DTSRequestRouting • DTSRequestServiceExpectation • DTSRequestPayloadType • DTSRequestPayloadBytes • DTSRequestSignature
Convoluted Filename vs Header Elements • A [B] <X.Y.Z:M> • A = File Type, B = Encrytption, X.Y.Z = key identifier, M = Unique message ID • Encryption unnecessary because using HTTPS • DTSRequestPayloadType = A • DTSRequestRouting • SourceIDSubCode = X, SourceID =Y(.Z) • UUID = M
Interop Problem with SOAP Headers • xsi:type attribute in Header elements • Java includes and requires this attribute • .Net does not
<soap:Header> <DTSRequestPayloadType xsi:type="DTSRequestPayloadType" xmlns="http://www.datatransportstandard.com"> <value>CRC01Request</value> </DTSRequestPayloadCode> All about SOAP
SOAP is the Key • The SOAP transmitted across the wire is of primary importance • Element names • Type attribute • Not Namespace moniker (Java uses one by default, .Net does not) • How you get the correct SOAP is not important
SOAP Differences That Do Not Matter Java: <ns1:DTSRequestSignature soapenv:mustUnderstand="0" xsi:type="ns1:DTSRequestSignature" xmlns:ns1="http://www.datatransportstandard.com"> <ns1:value>SignatureValue</ns1:value> </ns1:DTSRequestSignature> .Net: <DTSRequestSignature xsi:type="DTSRequestSignature"xmlns="http://www.datatransportstandard.com"> <value>SignatureValue</value> </DTSRequestSignature>
Reference Implementation Architecture • Client Application • Client Core • Service Core • Service Application
Client Application • Knows nothing of SOAP or Web Services • Implements Client Core Interface • “Setters” and “Getters” of DTS specific elements • Houses specific business logic
Client Core • Knows nothing of business logic • Uses properties set to construct the SOAP • Interface for “setting send” and “getting returned” elements • Handles the communication to Service Core- DTS Specification
Service Core • Accepts transmissions from Client Core • Implements Service Application Interface • “Setters” and “Getters” of DTS specific elements • Creates return SOAP • Format return acknowledgement or data from Service Application • Construct SOAP faults
Service Core (con’t) • Isolated business logic • Examples • Invoke Service Application based on payload • Place payload in “queue”
Service Application • Interface for “setting sent” and “getting to be returned” elements • Houses specific business logic • Knows nothing of SOAP or Web Services
DTS Version 2 • New tools • No changes to interfaces • Reorganization of SOAP Headers • Full WS-Security integration
New Tools • Axis 1.4 removed requirement for xsi:type attribute • Reduces complexity from .Net side • Implemented WSS4J and WSE • WS-Security integration