1 / 56

PeopleTools Application Server & BEA Tuxedo Internals

PeopleTools Application Server & BEA Tuxedo Internals. david.kurtz@go-faster.co.uk Go-Faster Consultancy Ltd. http://www.go-faster.co.uk. Who am I?. Independent Consultant Performance Tuning PeopleSoft / Oracle. Rules of Engagement. Interrupt me: if you think I have got something wrong

Download Presentation

PeopleTools Application Server & BEA Tuxedo Internals

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. PeopleTools Application Server & BEA Tuxedo Internals david.kurtz@go-faster.co.uk Go-Faster Consultancy Ltd. http://www.go-faster.co.uk

  2. Who am I? • Independent Consultant • Performance Tuning • PeopleSoft / Oracle

  3. Rules of Engagement • Interrupt me: • if you think I have got something wrong • if you have a question • if you can’t hear me

  4. BEA Tuxedo • Historical Overview • Tuxedo Internal Architecture • PeopleSoft 3-tier client implementation • How does PeopleSoft Internet Architecture (PIA) fit in • Configuration issues • Monitoring

  5. Data Access (DBMS) Business Logic Presentation Brief Historical Background • In the beginning • monolithic

  6. SQL Data Access (DBMS) Business Logic Presentation Raw Data LAN Client Server Client Server • Two-tier client-server

  7. Data Access (DBMS) Request GUI + Application Logic Response Business Logic LAN Client Server 3-Tier Client Server • “1st Generation”

  8. Presentation Business Logic Data Management Service 1 Service 2 ... Service n TUXEDO Messaging SQL Presentation Logic DBMS 1 ... DBMS n TUXEDO Clients TUXEDO Servers Database Servers 3-Tier Client Server • “2nd Generation” - Tuxedo

  9. Where did Tuxedo come from? • Bell Labs - 1993 • Transactions for • UniX • Extended for • Distributed • Operations

  10. What is Tuxedo? • It is the knots on the ends of the string!

  11. Presentation Logic Panel Load Panel Save Field Change etc... PeopleSoft 2-Tier Client

  12. Tuxedo Client Tuxedo Server Panel Load Panel Save Field Change etc... PeopleSoft 3-Tier Client Presentation Logic Server Side Client Side

  13. PeopleSoft 2-Tier Client • Client Connects Directly to the Database • 1 database connection per window • inactivity timeout DATABASE

  14. PeopleSoft 2-Tier Client • Client Connects Directly to the Database • 1 database connection per window • inactivity timeout • What happens in 3-Tier? DATABASE

  15. PS/Tuxedo Internal Structure • 1st Application Server Process Started • Bulletin Board Liaison (BBL) • Reads PSTUXCFG configuration file • Shared Memory Segment • Bulletin Board (BB) BB DATABASE PSTUXCFG BBL

  16. PS/Tuxedo Internal Structure • Workstation Listener (WSL) • Listens for connection request from clients on a specified port • Client initially contacts the WSL WSL BB DATABASE BBL

  17. PS/Tuxedo Internal Structure • Workstation Listener (WSL) • Client needs to know where to find the WSL • Machine and port specified in Configuration Manager WSL BB BBL

  18. PS/Tuxedo Internal Structure • Workstation Handler (WSH) • handles traffic from client after connection • Client is assigned to a port on the WSH • WSH port range can be constrained to match a firewall • Additional WSH can be spawned if necessary • Compress Messages WSL WSH BB DATABASE BBL

  19. PSAUTH AUTHQ APPQ PSAPPSRV QCKQ PSQCKSRV QRYQ PSQRYSRV APIQ PSAPISRV SAMQ PSSAMSRV PS/Tuxedo Internal Structure WSL WSH BB DATABASE BBL

  20. PSAUTH AUTHQ APPQ PSAPPSRV QCKQ PSQCKSRV QRYQ PSQRYSRV APIQ PSAPISRV SAMQ PSSAMSRV How does it work? WSHQ WSL WSH BB DATABASE BBL

  21. How does PIA work? • Java Client & PIA use Jolt • Java Class Library • Jolt Listener maps Java functions to Tuxedo Messages

  22. PSAUTH AUTHQ APPQ PSAPPSRV QCKQ PSQCKSRV QRYQ PSQRYSRV APIQ PSAPISRV JSH SAMQ PSSAMSRV Jolt Internal Structure JSHQ JSL JREPSRV WSL WSH BB JREPOSITORY DATABASE BBL

  23. Disk (static objects) New structure for PIA (Apache Webserver) Sessionid = “xy1k…vm8010 http 1.1 address+cookie (stateless) Cookie Java JVM Servlet Engine Servlet thread IS stateful Servlet Container Servlet Webserver Servlet Thread Servlet Thread HTTPD Client Servlet Thread Client mod_jserv Port 8010 Tuxedo Application Server JSL JSH JSH JSH

  24. What is the difference between the web servers? • Apache • Separate Servlet Engine • Separate encryption module (Raven) • Weblogic • Contains an integral servlet container

  25. Progressive Connection Concentration 35000 users • Typical picture • 35,000 users • 1000 concurrent connections • 1000 users active in the last n minutes • where n is the Java + JSH timeout • therefore need 1000 threads • but maybe only 4 JVMs • 100 JSH • assumes 10 clients per JSH • 10-20 PSAPPSRV processes • each PSAPPSRV connects to database • 1 database 1000 connections 4 JVMs 1000 Java Threads 100 JSHs 10-20 PSAPPSRVs 1 database

  26. Configuration psappsrv.val psadmin (configuration dialogue) psappsrv.cfg ubbgen (PS Utility) psappsrv.ubx psappsrv.ubb tmloadcf (tuxedo process) PSTUXCFG BBL (domain startup)

  27. psappsrv.cfg contains configurable parameters psappsrv.val validation file for configurable values psappsrv.ubx configuration template for Tuxedo domain psappsrv.ubb configuration file for Tuxedo domain PSTUXCFG compiled configuration file read by BBL at domain startup Configuration Files

  28. psappsrv.ubx & psappsrv.cfg PeopleSoft file make changes to these files difficult to read psappsrv.ubb Tuxedo file never make changes to this file easier to read this file is compiled by tmloadcf to generate domain configuration file PSTUXCFG variables fully resolved psappsrv.ubx -v- psappsrv.ubb

  29. psappsrv.ubb • Five (or Six) Sections • Resources • resource limits -> BB sizing • Machine • Resource limits. Environment • (Network • Connection between Machines) • Groups • Logical Split (BASE, APPSRV, JREPGRP, JSLGRP) • Servers • Number of servers, Command line • Services • Timeout, Load, Priority

  30. Application Server Configuration Issue • Move quick PSAPPSRV services into a second server (PSQCKSRV) (y/n)? • <=PT7.58 this question is misleading • services are additionally advertised on PSQCKSRV as well as PSAPPSRV • so they are advertised on more than one queue • >=PT7.59 and PT8.1 • services are de-advertised from PSAPPSRV • quick services not used in PIA, only IC* services • so not a problem

  31. APPQ PSAPPSRV QCKQ PSQCKSRV PeopleTools <=7.58 • Quick Service WSL WSH BB DATABASE BBL

  32. Inside the Bulletin Board • Without Load Balancing (default) • Table of servers process where each service is advertised • Read and populated as a stack • WSL looks for first free server where request can be enqueued • This can cause problems PSAPPSRV PSQCKSRV PSQCKSRV PSQCKSRV PSAPPSRV PSAPPSRV PSAPPSRV

  33. 50 50 50 APPQ PSAPPSRV 50 50 50 50 50 50 50 QCKQ PSQCKSRV Load Balancing • Load Balancing is about choosing the queue upon which to place the message • by default all requests have the same load

  34. Load Balancing Trace • Add -r to application server command line • CLOPT="-r -e APPQ.stderr -A -- -C psappsrv.cfg -D GP81O81 -S PSAPPSRV” • -r enables trace to file ‘stderr’ (not channel 2) • -e overrides the output filename • process trace file with Tuxedo utility ‘txrpt’ • obtain average service time • make load proportional to average service time

  35. Load Balancing Trace • Output from TXRPT SVCNAME 18p-19p TOTALS Num/Avg Num/Avg --------------- -------- ------- SqlRequest 16/0.10 16/0.10 MgrClear 5/0.04 5/0.04 RamList 1/0.43 1/0.43 SamGetParmsSvc 1/0.50 1/0.50 --------------- ------- ------- TOTALS 23/0.12 23/0.12

  36. 78 386 135 APPQ PSAPPSRV 50 6 30 3 63 3 7 QCKQ PSQCKSRV Load Balancing • load = execution time (seconds) x 100 599 116

  37. Multiple Queues on Busy System • You can have many server processes on a single queue • BEA recommend no more that 10 server processes per queue • If you have queue contention it is likely to impede the WSH/JSH processes enqueuing requests. You are unlikely to observe queuing with PQ. • So, if you want more than 10 PSAPPSRV processes you should use two APPQ queues • In, which case you should to enable load balancing and you must calculate the load. • Less significant with PIA because services different.

  38. APPQ1 PSAPPSRV APPQ2 PSAPPSRV Multiple Queues on Busy System WSL WSH BB DATABASE BBL

  39. Multiple Queues on Busy System • psadmin will not correctly calculate • MAXACCESSERS • MAXSERVICES • tmadmin->bbs • will report maximum number of services • count of all services advertised on each server process • and if you have multiple work station listeners • MAXWSCLIENTS

  40. 80 50 50 50 50 50 PSQCKSRV Service Priority • Priority is about queue jumping • servers de-queue the request • 9 out of 10 de-queued on the basis of priority • 1 out 10 is taken from the front of the queue QCKQ

  41. Service Priority • All PeopleSoft Services have same priority • specified in SERVICES section in psappsrv.ubb • therefore there is no queue jumping • no reason to believe that it would be advantageous

  42. Tuning Tip • Financials • On-line journal edit & post - remote call • Service initiates a Cobol process • Process can take minutes • RemoteCall services can block APPQ • Move RemoteCall service to a separate queue

  43. BB APPQ PSAPPSRV BBL BRIDGE BBL WSL WSH WSL WSH BB APPQ PSAPPSRV Multi-Node Domain • 1 domain on many nodes DBBL BRIDGE TLISTEN DATABASE TLISTEN

  44. Multi-Node Domain • Distinct Bulletin Board Liaison process • Master Node • TLISTEN process • Listens for incoming requests • BRIDGE • Sends & Receives messages between nodes • in a load balancing environment • configurable bridge load add to remote queues • limiting number of messages sent across the bridge

  45. Multi-Node Domain • DO ATTEMPT THIS WITHOUT BEA CONSULTANCY • At UBS • PT 7.5x • susceptible to network errors • integrated into HP service guard • ultimately this was removed from production because of transient network errors between the nodes causing the domain to ‘partition’

  46. So what is going on? • tmadmin • Tuxedo command line interface • also available within psadmin

  47. tmadmin commands • pq - printqueues • amount of work queued and waiting • not always accurate

  48. tmadmin commands • psr - printservers • reports activity of individual server processes

  49. tmadmin commands • psc - printservices • each service on each server

  50. tmadmin commands • Lots of other tmadmin commands • documented in the Tuxedo documentation • Use them to • dynamically reconfigure the domain • dynamically resize the domain • start/shutdown extra servers • monitoring scripts • tmadmin -r for read only mode

More Related