1 / 20

Data challenges

Data challenges.

jonah
Download Presentation

Data challenges

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data challenges 3030303030393534303035343130303030303130324241534532342D45505343414645582020202020303030303030303030302031343031323931343338323031342D30312D32392031332E33382E30372E3637352E303231343031323931343338303320202020202020202020202020202020202020202044425452414E3234322E3420204644495F30303030303131382020202032303134303132393133333830373637353030303030302020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020203134303132393134333830332020202020202020303030303239313333383037303030353537333531303630383638333134344120202020202020202020202020202020202020202020202020202020202020202020202020323031353031303120202020202020202020202020202020202020202020202020202020202020202020202020202020202020203230313430313239313333383037303030303030303031302E303038323630303030303030303030303030414D3534393947312032484C202020474252492056202020202020202020202030202020202020202020203120202020202020202020202020202020202020202020202020202020202020202020592020202020202020202020202020202020202020202020202020202020202020202020202053686F7070696E672063656E74726520202020202020202020202020202020202020202020202020476C6173676F77202020202020202020202020202020202020202020202047425220203131202020202020202020202020202020202020202020203030303035353733353130363038363833313434202020202020202020202020202020202020202020202020202020202020202020202020202020204F202020202020202020202020202020202020202020202020202020204E4E2059594E4E203030303030303030303030303030303030303030303030303030303030303030303030303030302020202030303230303020202020202020323736393031202020202020383236323030303030303220202020202020202020303120202020204120203830303030343030303006010A03E0810730303420202020202030363838382020202020202020202020202020202020202020202020202020202030303931312034353839363537383532343638323120202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020203535373335313036303836383331343420202030313135303120202020202031313030303020202020202020202020202020202020202020202020202020202020202020202020202020202020202020202020

  2. What is AtomBox Out of the box solution for quick and easy data access

  3. Key components • Atom/Box Data Model • AtomBox Script

  4. AtomBox Data Model Understands your data

  5. Atom/Box Data Model • Atom – the data unit • Holds a value – the smallest logical data unit • Packs (consumes)binary data into the internal storage • Unpacks the internal storage into binary data • Does validation against defined constraints • Has internal states – OK, WARNING, ERROR, DEFAULT • Emits SIGNALS – when packed, unpacked, value/state changed

  6. Atom/Box Data Model • Box – The container • Container of other Boxes and Atoms (does not hold a value) • Packs (consumes)binary data into the contained atoms and boxes • Unpacks the model into a binary data • Has internal states – OK, WARNING, ERROR, DEFAULT (all depending on its Atoms/Boxes it contains) • Emits SIGNALS – when packed, unpacked, value/state changed

  7. Atom/Box DDL • Textual Domain Specific Language aiming at describing any kind of data. • Compatible with HP NonStop DDL • Custom DDL language extensions for dynamic data: • VLENGTH DEPENDING ON • VLENGTH DELIMITED BY • ISOBITMAP • CHOICE – speculative parsing • EBCDIC – character set • Others. • Direct mapping to an Atom/Box Data Model

  8. Other Atom/Box mappings • ACI’s metadata storage CSV – file • Generated by MetaMan utility • Direct mapping to an Atom/Box Data Model • Further possible improvements includes: DFDL for IBM – Message Broker, or a Graphical DSL

  9. { workflow: { }<16><OK>, recordType_mb: {DBTRAN21}<8><Warning>, dataSpecificationVersion: {2.4 }<5><OK>, clientIdFromHeader: {FDI_00000118 }<16><OK>, recordCreationDate: { yyyy: {2014}<4><OK>, mm: {1}<2><OK>, dd: {29}<2><OK> }, recordCreationTime: { hh: {13}<2><OK>, mm: {38}<2><OK>, ss: {7}<2><OK> }, recordCreationMilliseconds: {675}<3><OK>, gmtOffset: {0}<6><OK>, customerIdFromHeader: { }<20><OK>, customerAcctNumber: { }<40><OK>, externalTransactionId: {140129143803 000029133807}<32><OK>, pan: {0005573510608683144}<19><OK>, authPostFlag: {A}<1><OK>, cardPostalCode: { }<9><OK>, cardCountryCode: { }<3><OK>, openDate: { }<8><OK>, plasticIssueDate: { }<8><OK>, cardExpireDate: { yyyy: {2015}<4><OK>, mm: {1}<2><OK>, dd: {1}<2><OK> }, transactionDate: { yyyy: {2014}<4><OK>, mm: {1}<2><OK>, dd: {29}<2><OK> }, transactionTime: { hh: {13}<2><OK>, mm: {38}<2><OK>, ss: {7}<2><OK> }, transactionAmount: {0}<13><Warning>, transactionCurrencyCode: {826}<3><OK>, transactionCurrencyConversionRate: {0}<13><OK>} userData01: { ref_anti_ref_days: { }<3><OK>, vac_cntry_cd: { }<3><OK>, user_ind_4: { }<2><OK>, card_act_stat: { }<1><OK>, ed_card_tech: {1}<1><OK> }, userData02: { acct_block_code_1: { }<1><OK>, acct_block_code_2: { }<1><OK>, card_block_code: { }<1><Warning>, bs_user_field_50: { }<1><OK>, amed_fraud_monitor: { }<2><OK>, some_filler: { }<4><OK> ……… Atom/Box model

  10. No data model is complete without an easy way of accessing it.

  11. A different approach: Interpreting vs Generating

  12. AtomBox Script Do more with less code

  13. Functional vs Imperative programming • Imperative programming • You tell the computer WHAT to do • Functional programming • You DESCRIBE what you want rather HOW you want something done

  14. JavaScript – The (only) language of the Web now coming to the Server • One of the world most popular language • Part of the WEB 2.0 revolution • Functional programming paradigm • Easy to use • Unlimited talent pool available • Built with the latest technologies (QT and C++)

  15. AtomBox Script • Based on the latest ECMA standards • Direct mapping from Atom/Box data models to JavaScript classes (Atom/Box instances) • Runs on multiple platforms: • UNIX, HP NonStop – Guardian and Windows • Asynchronous and event based • Remote debugging capabilities

  16. AtomBox Script (cont) • Full set of cross-platform API’s for: • Binary classes and streams • Accessing file systems and processes, terminals • Cryptography (including EFT modules) • TCP/IP Networking – including a TCP/IP server • Built-in WEB server for creating WEB Services • SQL module • XML • Extensive documentation

  17. AtomBox Script examples: !file helloddl! DEFINITIONHelloClass.     02 message.         04 part1 PIC X(5).         04 part2 PIC X(5) DEFAULT "world" MUST BE "world". END. var console = require("console"); //load the ddl that describe the message var Hello = require("helloddl","ddl").HelloClass; var o = new Hello(); //create an object of class Hello o.message.part1.value = "Hello"; //change the value of the part1 //unpack the message, decode the resulting ByteArray to a String //and invoke the `write` function on the 'console' object console.write(o.unpack().decodeToString());

  18. AtomBox Script examples: // //  TCP/Server Server example // var net     = require("net"); var server =  newnet.TcpServer(); server.listen(8150); // Connect the newConnection signal server.newConnection.connect(function() { // Accept the client connection     var client = server.nextPendingConnection();     client.write(“Hello World!”.toByteArray()); // Wait for the server to send the message and then disconnect     client.bytesWritten.connect(function() {          client.disconnectFromHost(); //close the connection     }); });

  19. AtomBox in the real world

  20. http://www.atombox.org

More Related