Comprehensive Course on z/TPF and Operating Systems Evolution

1. Course Objective • This material is for anyone that is new to z/TPF or is looking for a refresher about z/TPF. • This is a technical course that primarily focuses on topics from a system point of view. • This course will generally cover most aspects of z/TPF and it’s environment. Some topics will be covered in more detail. • General background in Computer Science is needed to understand the general computing concepts discussed in the material.

2. TPF Agenda

3. z/TPF Detailed Summary - Basic Overview History  Hardware Software Supporting Systems System Characteristics System Configuration System Behavior Database Communications Security Console and Monitoring

4. History • Airlines Control Program(ACP) • 60’s, 70s: ACP consolidated from several individual efforts • IBM also consolidated the application package which became known as PARS and International PARS (IPARS) • Transaction Processing Facility(TPF) • 70s, 80s: TPF 1 and subsequent versions came after IBM had to unbundle (sell separately) operating systems and HW • 90s: TPF 4.1, virtual addressing, C/C++ support and more – Multiple CPU support • z/Transaction Processing Facility (z/TPF) • 2005: z/TPF 1.1 Introduced • 64 bit, open development

5. History – Compare and Contrast

6. Mainframe Operating System Heritage • OS/360 -> OS/VS -> MVS/SP -> MVS/XA -> MVS/ESA -> OS/390 -> z/OS • z/OS is IBM’s premier zSeries operating system, is a highly secure, scalable, high-performance enterprise operating system on which to build and deploy traditional and Java-enabled applications, providing a comprehensive and diverse application execution environment. • DOS/360 -> DOS/VS -> VSE/SP -> VSE/ESA -> z/VSE • VSE enables proven, robust, and cost-effective solutions. VSE provides sturdy batch and industrial strength on-line transaction processing (CICS) capabilities. VSE can fit comfortably into a legacy of thrifty, dependable VSE solutions. • ACP -> TPF-> z/TPF • TPF is the platform driving the business-critical systems for many of IBM's largest and most sophisticated users of online transaction processing - airlines, railroads, hotels, financial services, government, credit card and banking industries. • CP/67 -> VM/370 -> VM/SP -> VM/XA -> VM/ESA -> z/VM • z/VM provides a highly flexible test and production environment for enterprises deploying the latest e-business solutions. z/VM helps enterprises meet their growing demands for multi-user server solutions with support for a broad range of operating systems. • Linux on System z • IBM decided to support Linux in 2000.

7. z/TPF Detailed Summary – Basic Overview History Hardware  Software Supporting Systems System Characteristics System Configuration System Behavior Database Communications Security Console and Monitoring

8. Hardware - Needed to run z/TPF • System z Server • Console to access the system • Storage Devices • Disk (DASD) • Tape Drives and Tapes Communication Card • Open Systems Adapter (OSA)

9. Hardware – System Z • OS’s supported • z/OS • z/VM • z/Linux • z/VSE • z/TPF • Blade Center Extension z/BX • Power and X • AIX • Windows • Linux Source: If applicable, describe source origin

10. Hardware – System Z (continued) IBM Presentation Template Full Version Capacity • up to 101 Engines • z/TPF up to 99 Engines • 3TB memory per server Connectivity - Channels FICON/ESCON – Disk, Tapes, Console Cards – OSA, Crypto RAS • Reliability • Availability • Serviceability Source: If applicable, describe source origin

11. Linux Linux Linux Linux z/TPF zOS zOS zOS zOS Virtual Machine Linux Linux Linux AIX AIX AIX Virtual Machine Virtual Machine Virtual Machine Virtual Machine Linux Linux Linux Linux Linux Linux AIX AIX AIX AIX AIX AIX ISS Linux Linux Linux AIX AIX AIX Linux Linux Linux Linux Linux Linux AIX AIX AIX AIX AIX AIX ISS DP Linux Linux Linux Linux Linux Linux AIX AIX AIX AIX AIX AIX DP Cell Linux Linux Linux AIX AIX AIX AIX Windows DWA Cell DWA xHyp pHyp xHyp xHyp pHyp pHyp xHyp pHyp xHyp xHyp pHyp pHyp xHyp xHyp pHyp pHyp AMM AMM SE z Blade Extension SE AMM z Blade Extension Hardware – Diagram of System Z usage Workload Resource Groups: SLA Mgmt and Reporting x86 Power HMC (with Unified Resource Management) Data Power zHypervisors PR/SM xHyp pHyp Z CPU, Memory and IO Z CPU, Memory and IO z Blade Extension

12. Hardware – Console TPF Operation Server Provides: • Console Access • Automation • Monitoring • WinTel Hardware • Windows OS

13. z/TPF Detailed Summary - Basic Overview History Hardware Software  Supporting Systems System Characteristics System Configuration System Behavior Database Communications Security Console and Monitoring

14. Software • z/TPF is an Operating System • Runs exclusively on IBM System z hardware • High Performance, eXtreme Transaction Processing (XTP) • Dynamic, Large Centralized System • Remote Users accessing Shared Information • Attributes • Speed • Efficient processing of transactions • Reliability • Hardware and Software provide minimal downtime • Availability • 24x7x365 • Scalability • 99 I-Streams in each server • 32 way loosely coupled servers • Choose your combination

15. Software – What z/TPF Provides • z/TPF Operating System • Directly controls the management of the hardware resources assigned to the central processing complex (CPC) • Provides system services • Macros and Functions • Interfaces for Applications via API’s • Interfaces with Communication Networks, Disk, Tape, Console, and other attached devices • Data Collections provide overall utilization and performance Image – http://en.wikipedia.org/wiki/File:Operating_system_placement.svg

16. Software – z/TPF Highlights • Efficient use of resources, such as main storage and file storage • Short path lengths for critical system services, such as direct access storage device (DASD) input/output (I/O) • Short Existence Time for messages/transactions • < .3 seconds • Fast Restart and Recovery – 30 seconds to 2 minutes • Entry Protection – each entry only knows about it’s resources • Open-ended capacity growth, such as clustering as many as 32 multiprocessor Enterprise Systems Architecture (ESA) configurations with only a minimal increase in system overhead • Expandable database capacity by the addition of direct access storage devices (DASD) • Database integrity and online database maintenance capability. • Reentrant code … i.e. sharable

17. Software – Control Program Memory • Also Known As: Control Program, CP, Kernel, CPS0 • About 3.5 Meg in size • Memory Resident • Written in Assembler • Dispatch Control (CPU LOOP) aka dispatcher • Channel Programs/Drivers • Efficient • Hardware Specific • Memory/Storage Management • Linkage • CSECTS linked together make up CPS0 • Built as part of System Initialization (SIP) • Most of the Source Code is Available for Modifications • User Exits • More details in System Characteristics part of course Control Program

18. Software – Shared Objects / Programs Note: Core resident = Memory resident Memory • Any software that is not part of the CP is a Shared Object (SO) that can be pre-loaded in memory or loaded when initially called for use. • SO’s are written in: • Assembler • C/C++ • SO’s can be z/TPF System Software or Application Software Disk/DASD Core Resident Program Area Control Program

19. Software – Middleware Layer • Provides z/TPF services for applications. • Includes communications, database managers, transaction services, API’s, etc • For example – z/TPF supports • MQ (Turbo) and Client • Web Services:XML, SOAP,WS-Security • Mail Server • HTTP Server and Client • TPFDF • Tivoli Endpoint Manager • WebSphere Operational Decision Management (WDOM) • Business Events • Transaction Services • Ported services: Apache, OpenLDAP, MySQL, OpenSSL, zlib

20. Software – Application Layer • z/TPF applications are customized and maintained by thecustomer • Pros • Unique applications to the industry • Customer controls features and functions • Customization provides specific functionality compared to of-the-shelf Application • Cons • Maintenance • Adding Feature/Functions • Available Subject Matter Experts • z/TPF provides standardized system services, middleware, and application frameworks to modernize established applications as well as create new applications. • Applications can be written in languages: • Assembler, C, C++

21. Software - z/TPF Directory Structure of Software

22. Software – Comparing z/TPF to other Transaction OS’s IBM Presentation Template Full Version • Most other Transaction Processing models run on top of an operating system ===> are not as efficient as a native operating system like z/TPF Memory Based solutions • Transactional speed achieved by in-memory tables • Capturing changes to memory database is problematic Distributed Based solutions • Transactional speed achieved by multiple images/servers • Syncing the distributed database is inefficient • Heavy-weight vs Light-weight process model • z/TPF processes run on a CP or Kernel level compared to processes run on an application level. • This minimizes instruction length of processes

23. z/TPF Detailed Summary - Basic Overview History Hardware Software Supporting Systems  System Characteristics System Configuration System Behavior Database Communications Security Console and Monitoring

24. Software - Supporting Systems z/TPF depends on other systems to: -build the z/TPF configuration, -build z/TPF software, -run off-line batch jobs, -connect primary console, and -provide tools for testing, performance and debugging. • z/OS • Format DASD Environment • Create System Initialization Environment • Run batch oriented tasks e.g analysis • Access to General Data Sets • z/Linux • Used to build z/TPF object code • Create Loader components • Other Utilities

25. Software - Supporting Systems (cont) z/TPF depends on other systems to: -build the z/TPF configuration, -build z/TPF software, -run off-line batch jobs, -connect primary console, and -provide tools for testing, performance and debugging. • VM • Can be used as main operating system with z/TPF as a guest • TPF Toolkit (Eclipse based) • Recommended Integrated Development Environment • TPF Operation System (PC based) • Recommended Operational Console Management environment • Tivoli EndPoint Manager • Recommended continuous data collection environment

26. Software - z/TPF still uses z/OS for… • Assemble, compile, link of all z/OS offline programs • DASD Formatter • Data Reduction • Dump Post Processor • And others… • Loading Software • Loader General File (LGF) • Via GDS, TAPE/VTAPE, VRDR • Requirements • IBM z/OS Version 1 Release 3, or later release • High Level Assembler (HLASM) Release 5 or later • IBM Enterprise PL/I for z/OS Version 3 Release 3, or later release • GNU Make (3.79.1) * For current versions refer to “Required z/OS Software” in latest z/TPF documentation

27. Software - z/TPF uses Linux for… • Building Software • Linux utility programs • SIP deck assembly (System configuration macros) • Face table generation (File format) • Assemble, compile, link of all online programs • Assemble, compile, link of all Linux offline programs • Build Software Modules for Installation • BuildTPF, LoadTPF • Load Software • OLDR loads (via FTP) to z/TPF • Access to source code • Source Contol Manager (SCM) integration • Hosts source code for z/OS and Windows • Editing of source code • Running Utilities, Editing, Troubleshooting, Etc For current versions refer to “Required Linux Software” in latest z/TPF documentation

28. Software - z/TPF still uses VM for… • Ease of Use • Configuration changes • Virtual Tape Drives • Multiple Images of z/TPF for testing • Can create smaller test systems • CMS • PER Trace (Event recording) • VTAPE (Virtual tapes) • VPARS (Share a TPF image on mult VMs for testing)

29. Software - z/TPF still uses TPF Toolkit for.. • Integrated Development Environment(IDE) • Debugger • Performance Analyzer • Interface to Source Control Manager (SCM)

30. Software - z/TPF still uses Tivoli Endpoint Management • Continuous Data Collection • Integration with Enterprise Monitoring

31. Software - z/TPF still uses TPF Operations Server for… • Console Access • Automation • Logging of output • Interface to other Enterprise systems

32. Software – Diagram of Supporting Systems

33. z/TPF Detailed Summary - Basic Overview IBM Presentation Template Full Version History Hardware Software Supporting Systems System Characteristics System Configuration System Behavior Database Communications Security Console and Monitoring Source: If applicable, describe source origin

34. System Characteristics - Entry Control Block (ECB) z/TPF Security Features “Start a Process” Input Message Control Program Entry Control Block (ECB) “End a Process” Output Message “Interacts with..” Shared Object(s) Shared Object(s) CPU(s) “Interacts with..” Memory/Disk

35. System Characteristics – Application Transaction IBM Presentation Template Full Version • A message is a component of a transaction in z/TPF. - The number of messages to complete a transaction depends upon the application design • The time required is dependent upon the speed of the end user and the complexity of the transaction. • z/TPF is designed on the assumption that each end user message require only small amounts of CPU processing. • A transaction consists of one or more messages. • Example of a transaction • A13JULSLCJFK7P • Availability entry requesting flights from Salt Lake City to New York beginning at 1900 on July 13th • N1Y2 • Need one seat in Coach Class from the display which is on line 2 • -lastname/firstname Source: If applicable, describe source origin

36. System Characteristics – Application Transaction Examples IBM Presentation Template Full Version • An example of a transaction • A13JULSLCJFK7P • Availability entry requesting flights from Salt Lake City to New York beginning at 1900(7PM) on July 13th • N1Y2 • Need one seat in Coach Class from the display which is on line 2 • -lastname/firstname • An example of Web Service Transaction • XML Request Source: If applicable, describe source origin

37. System Characteristics - Main Supervisor (Control Program) • The principal functions performed by the main supervisor are: • Work scheduling – dispatching • Priority List scheme: Ready, Input, Defer, suspend lists (queues) • Dispatch all tasks on the list before move to the next list • I/O interface • No ‘access method’, channel program is integrated into system support • Virtual address space memory management • EVA (Primary address space) or (ECB Virtual Address) vs. SVA (Home address space) or (System Virtual Address) • Storage management • Interrupt processing • I/O, SVC, Program, External, • Keypointing (aka checkpointing) • Save system status in case of restart because of a hardware or software malfunction. In other operating systems, this usually is called

38. System Characteristics - Memory • Fencing of Data • An ECB’s memory is not addressable by another ECB • This includes memory requested by the application • Some memory can be shared via: • Global memory • Common • Virtual Memory • Memory get backed with real memory when needed • System Virtual Memory (SVM) • ECB Virtual Memory (EVM) • Protection keys • Applications cannot write to system memory without special access - restricted macros and API’s

39. Memory Layout EVM vs SVM Common Virtual Addresses 0 Prefix Area Control Program 16MB IPLB/CCIO/Keypoints/CIMR 31-bit System Tables EVA Differences SVA Differences 31-bit User Areas* -----Copy-on-Write 31-bit CRPA* Common Blocks IOBs ECB Private Area SWBs Thread Stack Area* ECBs Application/ECB Stack Area* 4K Frames Pre-allocated ECB Stack 31-bit System Heap* Pre-allocated 31-bit ECB Heap 2GB 64 bit CRPA (PJ37995) 31-bit ECB Heap* 4GB Region/Segment/Page Tables 64-bit System Tables TCP/IP Areas* 64-bit User Areas* -----Copy-on-Write 64-bit CRPA* 1MB Frames 64-bit ECB Heap* System Heap Control* 64-bit System Heap* VFA Highest SVA

40. z/TPF Characteristics - Disk • Data defined as modules • Data is spread out across modules • A database is spread out over multiple modules to minimize highly accessed data (avoid “hot” spots" • Modules duplicated logically in modules and physically on hardware • Prime • Dupes Data reads from quickest path available • Data writes duplicated • Virtual File Access (VFA) • Data reads in memory for faster access • Data writes in memory and physical file (write thru) • Highly accessed data stays in memory • Low utilized data bubbles out of memory

41. z/TPF Characteristics - Tape • Tapes defined as individual devices that are accessible via Tape API’s • Primary and Alternates defined • If a tape becomes full will switch to Alternate • If z/TPF cannot write to a tape it tries on the Alternate • Application and System Errors are written to tape • Data can be post processed to produce a dump report • Dump Buffer Area • A defined area in memory to quickly write out the error so that the snap shot time is minimized

42. z/TPF Characteristics – Loading Software • New software is loaded to z/TPF using: • General File Loader • E-type Loader • Image Loader • Data Loader • z/TPF has the ability to introduce new real-time software while running transactions • ZOLDR Functional Message • Activation number • Software bundled in loadsets • Activated/Deactivated • Accepted/Deleted

43. z/TPF Detailed Summary - Basic Overview History Hardware Software Supporting Systems System Characteristics System Configuration  System Behavior Database Communications Security Console and Monitoring

44. System Configurations • Uni Configuration • Single CPU • Tightly Coupled Configuration • Multiple CPU’s (Instruction Streams) Sharing Memory • Single Copy TPF Executing Concurrently • Loosely Coupled Configuration • Two or more CPC sharing a common set of DASD (Database) and using an external lock facility (XLF) to synchronize multiple access to the DASD records. • Each CPC can be Tightly Coupled or in uni-processor mode • Each CPC runs a copy of TPF concurrently

45. System Configuration - Uni Environment Network System Z LPAR Only one I-stream used in Uni system Up to 99 I-Streams DASD Online Programs Control Program z/TPF

46. System Configuration - Tightly Coupled Network System Z LPAR Up to 99 I-Streams DASD Online Programs Control Program z/TPF

47. System Configuration - Loosely Coupled Environment Network System Z LPAR System Z LPAR System Z LPAR Up to 99 I-Streams Up to 99 I-Streams Up to 99 I-Streams DASD Online Programs Online Programs Online Programs Control Program Control Program Control Program z/TPF z/TPF z/TPF

48. z/TPF Detailed Summary - Basic Overview History Hardware Software Supporting Systems System Characteristics System Configuration System Behavior Database Communications Security Console and Monitoring

49. System Behavior - Initial Program Load (IPL) • Initial IPL - IPL From Loader General File (LGF) • The Loader General File is a disk module >contains programs and keypoints >used to initialize a system. >sequential collection of records >created offline under z/OS control.

50. System Behavior - Initial Program Load (IPL) • The TPF system is loaded to the online system in 2 steps: • The online keypoints, the file resident and main storage resident program segments: are loaded to the loader general file by the system loader offline (ALDR). • They are then loaded from the loader general file to the online files by IPLing the loader general file. (ACPL) • The TPF system is started by IPLing the prime (or backup) online system pack.