1 / 19

Using Quadstone’s Data Build Manager

Using Quadstone’s Data Build Manager. Thursday, September 15, 2005 9am Pacific, 12pm Eastern, 5pm UK/Ireland Friday, September 16, 2005 2pm UK/Ireland, 3pm Central European, 9am Eastern US. Please join the teleconference call now; if you have any difficulty, contact support@quadstone.com.

misu
Download Presentation

Using Quadstone’s Data Build Manager

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using Quadstone’s Data Build Manager Thursday, September 15, 2005 9am Pacific, 12pm Eastern, 5pm UK/Ireland Friday, September 16, 2005 2pm UK/Ireland, 3pm Central European, 9am Eastern US Please join the teleconference call now; if you have any difficulty, contact support@quadstone.com. Starting in 15 minutes Starting in 10 minutes Starting in 5 minutes Starting in 2 minutes Starting now

  2. How to ask questions Use Q&A (not Chat please): • Click on the Q&A Panel icon at the bottom-right of your screen: • Type in your question:

  3. Using Data Build Manager • Presenter: Patrick Surry, VP Customer Services • Overview: The Data Build Manager (also known as qsbuild) is a powerful tool to manage all of the interdependent steps in real-world data-preparation, including parameterization for automated scheduling and the ability to run tasks in parallel. • Audience: Existing Quadstone data architects, looking to improve the processing speed and their productivity in creating customer analysis datasets. • Format: • A live demo with slides for sign-posting • Downloadable exercises in the form of a workbook and dataset • Duration: 1 hour, including Q&A

  4. Customer IDs To be filled Measurement table A simple data preparation process Customer data Transaction data SORT SORT DERIVE MEASURE JOIN DERIVE

  5. Quadstone data preparation tools XML Build Plan • Efficient modular utilities operating primarily on foci • Run via Quadstone System Explorer, the command line, or an XML build plan RDBMS1 Measure RDBMS1 Sort RDBMS2 RDBMS2 FOCUS FOCUS Join Flat files Flat files Enhance Third-party Third-party etc.

  6. FOCUS FOCUS Data-build commands qsbuild EXPORTING qsdbcreatetable qsdbinsert qsdbupdate qsexportflat qsexportstat TRANSFORMING qssort qsrenamefields qsselect qsderive qsmeasure qstrack ENHANCING qsimportmetadata qsupdate [qsinterp] [qsexportmetadata] IMPORTING qsdbaccess qsimportdb qsgenfdd qsimportflat qsimportstat qsimportfocus MANAGING qscopy qslink qsmove qsremove [qsremoveflat] qstml REPORTING qsdescribe qsdescribestat qsaudit qsdtsnapshot qsscsnapshot qsxt qsxt2spec qsmapgen [qsinfo] COMBINING qsjoin qsappendfields qsmerge SeeQuadstone System data-build command and TML reference

  7. What does Data Build Manager do? • Flexible environment for implementing data-builds • keep simple builds simple but support advanced requirements • XML build plan; qsbuild DBC, point & click build execution • Key features: • Simple & robust – simple structure with many different tasks • Complete – everything in one place, including inline TML/FDL/SQL (if desired), and/or non-Quadstone tasks • Modular & portable – structure, reuse and move builds easily • Parameters – no code changes for similar builds • Incremental builds – failure recovery, only do what’s needed • Concurrency – run multiple jobs at the same time • Logging – various ways to track build status and performance

  8. How do I launch it? • Double-click a .qsb file (Build Plan) in the Quadstone Explorer

  9. What’s a build plan look like? • Right-click a .qsb file in the QSE and choose View or Edit

  10. What is XML? • Borrowed from: http://www.w3.org/XML/1999/XML-in-10-points 1. XML is for structuring data Structured data includes things like spreadsheets, address books, configuration parameters, financial transactions, and technical drawings. … XML makes it easy for a computer to generate data, read data, and ensure that the data structure is unambiguous. … 2. XML looks a bit like HTML Like HTML, XML makes use of tags (words bracketed by '<' and '>') and attributes (of the form name="value"). While HTML specifies what each tag and attribute means, and often how the text between them will look in a browser, XML uses the tags only to delimit pieces of data, and leaves the interpretation of the data completely to the application that reads it. … 3. XML is text, but isn't meant to be read … One advantage of a text format is that it allows people, if necessary, to look at the data without the program that produced it; in a pinch, you can read a text format with your favorite text editor. Text formats also allow developers to more easily debug applications. Like HTML, XML files are text files that people shouldn't have to read, but may when the need arises. …

  11. How can I change it? • Extend a target with new steps (tasks) • Cut & paste examples from documentation • Cut & paste from command-line or focus history • Create new targets for logical separation • Note build’s default target (and initial, final); dependencies • Nest targets if desired • Increase efficiency • Conditional execution with ‘unless’ to avoid rework • Temporary outputs to avoid clutter • Inline or external scripts (TML, FDL, SQL, …)

  12. Making it reusable • Use properties to avoid repetitive changes • Like variables but can’t change once set • Tasks to set and manipulate in many ways • Parameters are user-visible properties • E.g. User selects build snapshot date • E.g. User selects full or sample datasets • Example: • Parameterize build with target month

  13. More flexible ways of editing • Some editors know XML (& schema!): very helpful • See documentation for how to set it up, e.g. jEdit

  14. Going further • RTFM – good overview of capabilities • Concurrency • Default values for common attributes • Date manipulation via qsdateproperty, e.g. today • Debugging techniques • Running from the command-line • Good practices: tips & traps • Other resources on XML, Ant, etc

  15. Where to get help • Start>All Programs>Quadstone • Using Data Build Manager • Also see: Data-build command and TML reference; Data-build command Tutorial • Latest at support.quadstone.com/documentation • Quadstone System Support: • Web Site: support.quadstone.com/ • Email: support@quadstone.com • Tel: US 1-800-335-3860; UK 0131 240 3140; All +44 131 240 3140

  16. After the webinar • These slides, a workbook and data are available via www.quadstone.com/training/webinars/ • Audio and video recordings of this webinar are available via the same site • Any problems or questions, please contact support@quadstone.com • For more in-depth training (our ½-day Automating Data Preparation course), contact support@quadstone.com

  17. Questions and answers

  18. Upcoming webinars Ideas: • Based around a real-life scenario (possibly Uplift Analysis)? • Decision trees, scorecards and quality measures: the gory math internals? • What's new in 5.2 • Cluster Builder? • More on TML? • Re-run previous webinars See www.quadstone.com/training/webinars/. If there’s a webinar topic you’d like to see, please let us know via support@quadstone.com.

  19. Your feedback Please email support@quadstone.com

More Related