1 / 59

E139 Why Troubleshooting ? Avoid Problems Upfront !

E139 Why Troubleshooting ? Avoid Problems Upfront !. Markus Ohly Sybase European CS&S Markus.Ohly@sybase.com. Agenda (1/2). Introduction Have the Right Software in Place Have an eye on your Jaguar Write Components that ease your life Benchmarking Memory Usage Connection Caching.

rudolf
Download Presentation

E139 Why Troubleshooting ? Avoid Problems Upfront !

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. E139Why Troubleshooting ? Avoid Problems Upfront ! • Markus Ohly • Sybase European CS&S • Markus.Ohly@sybase.com

  2. Agenda (1/2) • Introduction • Have the Right Software in Place • Have an eye on your Jaguar • Write Components that ease your life • Benchmarking • Memory Usage • Connection Caching

  3. Agenda (2/2) • Character Sets, Globalization • C++ components and 3rd party libraries • Fault Tolerance and Service Continuity • Limitative Configuration Options • Threading • (BOOT)CLASSPATH

  4. Agenda ... • Introduction • Have the Right Software in Place • Have an eye on your Jaguar • Write Components that ease your life • Benchmarking • Memory Usage • Connection Caching

  5. Introduction • This presentation is not a classical troubleshooting session, it rather explains the known cliffs and how to avoid them. • This will help you to avoid a lot of trouble before you get into it and will in turn avoid • projects to stall • production services to interrupt • technicians and management to be upset and • customers to shy away

  6. Agenda ... • Introduction • Have the Right Software in Place • Have an eye on your Jaguar • Write Components that ease your life • Benchmarking • Memory Usage • Connection Caching

  7. Have the Right Software in Place • Choose the right Edition of EAServer • Developer Edition • Small Business Edition • Advanced Edition • Enterprise Edition

  8. DE SB AE EE Connections 20/10 * 10/nol‘t nolimit nolimit Clustering Y N Y Y 2PC Y N Y Y In Memory Failover N N Y Y Failover Stateless Y N N Y Failover Stateful Y N N Y Have the Right Software in Place • Choose the right Edition of EAServer • * Earlier Versions had different limitations

  9. Having the Right Software in Place • Sybase continuously improves EAServer and releases updated versions. The latest are: • EAServer 3.5: 3.5 C5 • EAServer 3.6.1: 3.6.1 ESD 4 • EAServer 4: 4.1.1 • This applies as well to third party software, eg. • Java Virtual Machine • Operating System and Patches • DBMS and Drivers

  10. Having YOUR Right Software in Place Repository Versioning • Repository Versioning has been introduced with EAServer 4.0. • It allows you to do changes to components and server configuration in a clean, explicit, traceable and undoable way.

  11. Agenda ... • Introduction • Have the Right Software in Place • Have an eye on your Jaguar • Write Components that ease your life • Benchmarking • Memory Usage • Connection Caching

  12. Have an eye on your Jaguar • Having an eye on your Jaguar will allow you to notice the very firsts signs of unnormal functioning • This gives you time to react, intervene and keep trains on time • Suggestions: • Reading log files with Jaguar::FileViewer • Statistics about Components and Connections • Third Party Tools • Operating System Level Tools

  13. Have an eye on your Jaguar • Jaguar::FileViewer allows remote access to log files LOG_BEGIN = 100 1. fileviewer.initialize(„Jaguar“); 2. int ids[] = fileviewer.getFileIds(); 3. String txt = fileviewer.text(id, LOG_BEGIN, length); 4. fileviewer.destroy();

  14. Have an eye on your Jaguar Jaguar Monitoring • Number of components • Number of connections

  15. Have an eye on your Jaguar Jaguar Monitoring • MONITOR_COMPONENT_ACTIVE • MONITOR_COMPONENT_POOLED • MONITOR_SESSION_IIOP • MONITOR_SESSION_HTTP • MONITOR_CONNCACHE_OPENED • MONITOR_CONNCACHE_CLOSED • MONITOR_CONNCACHE_FORCED

  16. Have an eye on your Jaguar Jaguar Monitoring • MONITOR_PEAK_MAXIMUM • MONITOR_LAST_MAXIMUM

  17. Have an eye on your Jaguar • Your Operating System provides many useful tools, eg: • vmstat, mpstat • NT Performance Monitor • Third Party Tools do exist • SNMP and JMDK interfaces exist since 4.0

  18. Agenda ... • Introduction • Have the Right Software in Place • Have an eye on your Jaguar • Write Components that ease your life • Benchmarking • Memory Usage • Connection Caching

  19. Write Components that ease your Life • Write Components that ease your life when things do not work as expected • Connections to Databases can fail • Running Commands on Databases may abort • Avoid the neccessity to restart jaguar, allow reinitialization

  20. Components that ease your Life (ctd) Error and Warning Messages • Should be precise and useful for users, administrators, and engineers • Choose to write warnings and errors into a separate log file • Mind the cost of I/O • Mind the cost of String concatenation • Think about a configurable verbosity level

  21. Components that ease your Life (ctd) • The J2EE Reference Implementation includes an EJB Verifier that checks the beans if they respect the conditions set in the specification • Components should have built-in methods to perform simple testing, eg. Resource References, EJB Links, Environment settings • Nested try-catch will make it very difficult to find the root place where the error was first raised.

  22. Components that ease your Life (ctd) • Environment and Configuration Reports • Admin/Debug Interface that allows to inspect the running system • CORBA components only: If the methods test() / debug() / control() are defined in a separate interface, you can get a generic stub which facilitates coding a lot.

  23. Agenda ... • Introduction • Having the Right Software in Place • Having an eye on your Jaguar • Write Components that ease your life • Benchmarking • Memory Usage, Pool Management • Connection Caching

  24. Benchmarking Functionality • More commonly referred to as „Testing“ • Programmatic verification of the functionality with given data and known results • Either manual or automatic • Performed under laboratory conditions

  25. Benchmarking Functionality • Embedded Test Functions inside the released product to check the correct settings for J2EE Applications after deployment, eg. • Resource Links • EJB Links • Environment settings • Connection Caches

  26. Benchmarking Performance • Record the average expected execution time of known functionality with known data • idle • under load • Repeat this in production, you will then be able to see where your system is behind • Com.sybase.jaguar.component.trace = true

  27. Agenda ... • Introduction • Have the Right Software in Place • Have an eye on your Jaguar • Write Components that ease your life • Benchmarking • Memory Usage • Connection Caching

  28. Memory Usage Caching of Objects • Component Instance Pool Management • Variable Scope: method variables preferred over instance variables • Home Interfaces • Stateful vs. Stateless

  29. Memory Usage Stateful vs. Stateless • A stateful component has a 1:1 relation with its client • A stateless component can be used by many clients in a sequence, thus less instances are needed. • In case that the client fails to call remove() for a stateful instance, the instance will be kept until a timeout fires • Instance Timeouts: Server Default and per-Component Setting

  30. Memory Usage Caching of Data • Data Caches • Keep results of queries in memory that are frequently needed but change rarely • Optimally Changes are done only via Beans • Files to be read from disk • ResultSet • Tools can help a lot: JVM Profiling, OptimizeIt, Jprobe, Hpjmeter, jmeter

  31. Memory Usage Runtime System Managed Memory • Sometimes referred to as Garbage Collection • Set variables holding references objects to NULL, especially in pooled Components such as Beans and Connections • This requirement needs to be added to a Project Coding Style Guide

  32. Memory Usage Operating System Configuration • Windows NT processes can address up to 3 GB (NT/2000 Server Enterprise Edition) • Thread Stack Size • 1M by default • The number of threads „times“ the thread stack size must fit into the maximal process space size

  33. Agenda ... • Introduction • Have the Right Software in Place • Have an eye on your Jaguar • Write Components that ease your life • Benchmarking • Memory Usage • Connection Caching

  34. Connection Caching • The number of connections in the connection cache is a balance: either the concurrency is managed in EAServer (few connections) or in the database (many connections, force connections) • The Jaguar Connection Cache Manager allows CORBA-Components do specify explicitly how to behave if the configured number of connections is taken • With JCMForce, only as many connections as configured on the database server can be obtained – at this point the trouble is at two points

  35. Connection Caching • The options are implicit for EJB that have Resource References • You could as well set the Sanity Checking query to a more elaborate query than „select 1“ in order to cover planned service outage.

  36. Connection Caching • You should foresee DB admin means to overcome db stall situations, such as Full segments or full transaction logs • In ASE, sp_thresholdaction can be used to configure automatic action for full segments, especially the log segment.

  37. Connection Caching ORB Properties for Connection Caching • The ORB Connection Caching depends upon the setting of two options • socketReuseLimit -- number of method invocations after which the socket is closed and a new one is created for the next invocation • IdleConnectionTimeout -- time after which an unused (idle) connection will be discarded. • Tradeoff between load balancing and performance

  38. Agenda ... • Character Sets, Globalization • C++ components and 3rd party libraries • Fault Tolerance and Service Continuity • Limitative Configuration Options • Threading • (BOOT)CLASSPATH

  39. Character Sets • Traditional character sets do only cover part of the characters used in todays publications • Because e-commerce applications are open to a global customer base, the application must be capable to deal with different encodings at the same time. • Java holds characters and Strings as Unicode Characters -- conversion are done during reading and writing ! • A decompiler can help you to find which uni-codes have been embedded into a class file by the compiler • LC_CTYPE, LC_ALL

  40. Globalization • Customers all over the world do live in different Timezones • Even worse, this changes regularly: Daylight Savings Time • Names for the very same location can vary in different languages • Roma – Rome – Rom, • Den Haag – La Haye – The Hague

  41. Agenda ... • Character Sets, Globalization • C++ components and 3rd party libraries • Fault Tolerance and Service Continuity • Limitative Configuration Options • Threading • (BOOT)CLASSPATH

  42. C++ Components • A serious failure in C or C++ code (SIGxxx, GPF) will cause the whole EAServer process to terminate or to be blocked, and all clients will be affected. • In EAS 4.0, a stateless C++ component may be run in a separate process automatically • If the component is stateful, you will need to write your own stub implementation (in Java or C, again)

  43. Agenda ... • Character Sets, Globalization • C++ components and 3rd party libraries • Fault Tolerance and Service Continuity • Limitative Configuration Options • Threading • (BOOT)CLASSPATH

  44. Fault Tolerance and Service Continuity • In the case of failure • Data must still be available • Transactions should be restarted • Provide continuous access to resources to clients • Jaguar provides High Availability and Service Continuity per Failover and Clusterting

  45. Fault Tolerance and Service Continuity • For the client the failover should be as transparent as possible • However, hiding the failover fully is not possible in as situations (and sometimes too costly) • The client application must therefore be capable to repeat the last couple of steps with our without notification to the user • Make sure that the user’s input is not discarded !

  46. Fault Tolerance and Service Continuity • Databases provide sophisticated failover options • Clustering on machine and database-level • Replication to a Hot Standby (plus OpenSwitch) • other Sessions with more details. • E148 Achieving 24x7 Availability • E150 Building Highly Available Sybase Servers

  47. Fault Tolerance and Service Continuity • Eliminate all single points of failure by adding redundancy: • Multiple Machines • Multiple Application Servers • Multiple Name Servers

  48. Fault Tolerance and Service Continuity • Clients have a list of available name servers: • iiop://host1:9000;iiop://host2:9000 • Specify RetryCount and RetryDelay: • RetryCount – number of times to check server • RetryDelay – how long to wait between retries Properties props = new Properties(); props.put(“com.sybase.CORBA.RetryCount”, “10” ); props.put(“com.sybase.CORBA.RetryDelay”, “5”);

  49. Fault Tolerance and Service Continuity • Clients accessing Multiple Name Servers Client Name Server Application Server Client Name Server Application Server

  50. Fault Tolerance and Service Continuity When one application server fails, another will take over Routing Agent Routing Agent Web Server Application Server Browsers Web Server Application Server

More Related