Lecture XI: Logging and Monitoring

Lecture XI: Logging and Monitoring CS 4593Cloud-Oriented Big Data and Software Engineering

Logging and Monitoring • Basic and Logging frameworks • Logging and monitoring for web services 2

Basic

First Logging: System.out.println Easy Quick No dependencies Redirect to file or null in Production Log everything or nothing System.out.println(“Start method1”);

Example Start method1 Before retrieve information from DB After retrieve information from DB Updating information Before save information in the DB After save information in the DB Finish method1 Start method2 Before retrieve information from DB Error retrieving information from DB: NullPointerException

Next evolution:Logging Libraries Log4j, Java Logging… log.info(“Start method1”); Provide Time of the message The class, the method, the line … Print easily the stacktrace Log to files, databases, system log mechanisms

Example 06:05:56 07.11.2011 INFO (Class1.java:211) Start method2 06:05:56 07.11.2011 INFO (Class1.java:220) Before retrieve information from DB 06:05:56 07.11.2011 INFO (Class1.java:243) Error retrieving information from DB: NullPointerException java.lang.NullPointerException at pkg.Class1.method2(Class1.java:225) at pkg.Class1.methodN(Class1.java:458) at pkg2.ClassZ.methodUpdate(ClassZ.java:458) at … at java.lang.Thread.run(Thread.java:595)

Example 06:05:56 07.11.2011INFO (Class1.java:211) Start method2 06:05:56 07.11.2011 INFO (Class1.java:220) Before retrieve information from DB 06:05:56 07.11.2011 INFO (Class1.java:243) Error retrieving information from DB: NullPointerException java.lang.NullPointerException at pkg.Class1.method2(Class1.java:225) at pkg.Class1.methodN(Class1.java:458) at pkg2.ClassZ.methodUpdate(ClassZ.java:458) at … at java.lang.Thread.run(Thread.java:595)

Example 06:05:56 07.11.2011 INFO (Class1.java:211) Start method2 06:05:56 07.11.2011 INFO (Class1.java:220) Before retrieve information from DB 06:05:56 07.11.2011 INFO (Class1.java:243) Error retrieving information from DB: NullPointerException java.lang.NullPointerException at pkg.Class1.method2(Class1.java:225) at pkg.Class1.methodN(Class1.java:458) at pkg2.ClassZ.methodUpdate(ClassZ.java:458) at … at java.lang.Thread.run(Thread.java:595)

Logging levels (I) • Log4j • Trace • Debug • Info • Warning • Error • Fatal • Java Logging • Finest • Finer • Fine • Config • Info • Warning • Severe

Example 06:05:56 07.11.2011 INFO (Class1.java:211) Start method2 06:05:56 07.11.2011 INFO (Class1.java:211) Parameter ‘age’ is not an Integer 06:05:56 07.11.2011 INFO (Class1.java:211) Finish method2 06:05:56 07.11.2011 INFO (Class1.java:211) Start method2 06:05:56 07.11.2011 DEBUG (Class1.java:220) Before retrieve information from DB 06:05:56 07.11.2011 ERROR (Class1.java:243) Error retrieving information from DB: NullPointerException java.lang.NullPointerException at pkg.Class1.method2(Class1.java:225) at pkg.Class1.methodN(Class1.java:458) at pkg2.ClassZ.methodUpdate(ClassZ.java:458) at … at java.lang.Thread.run(Thread.java:595)

Where to log? Text files (Plain text, XML,…) Binary files Database Syslog (Windows & Unix/Linux) Network (TCP & UDP)

Where to log? Text files (Plain text, XML,…) Binary files Database Syslog (Windows & Unix/Linux) Network (TCP & UDP) Always to a Text file (+ something else)?

Performance "hidden" cost of parameter construction logger.debug("Entry number: " + i + " is " + String.valueOf(entry[i]));

Performance "hidden" cost of parameter construction logger.debug("Entry number: " + i + " is " + String.valueOf(entry[i])); Disable logging with options if (logger.isDebugEnabled()) { logger.debug("Entry number: " + i + " is " + String.valueOf(entry[i])); }

Multiple users (I) Different users working simultaneously Log messages mixed for different users Difficult to trace user’s actions

Multiple users (I) • Different users working simultaneously • Log messages mixed for different users • Difficult to trace user’s actions • Log User ID

Multiple users - Example 06:05:55 07.11.2011 INFO (Class1.java:41) User=bfernan; message=‘Start method1’; 06:05:55 07.11.2011 DEBUG (Class1.java:44) message=‘Before retrieve information from DB’; 06:05:55 07.11.2011 DEBUG (Class1.java:46) message=‘After retrieve information from DB’; 06:05:55 07.11.2011 INFO (Class1.java:50) message=‘Updating information’; 06:05:55 07.11.2011 DEBUG (Class1.java:94) message=‘Before save information in the DB’; 06:05:56 07.11.2011 DEBUG (Class1.java:96) message=‘After save information in the DB’; 06:05:56 07.11.2011 INFO (Class1.java:100) User=bfernan; message=‘Finish method1’; 06:05:56 07.11.2011 INFO (Class1.java:211) User=xxxxxxx; message=‘Start method2’; 06:05:56 07.11.2011 DEBUG (Class1.java:220) message=‘Before retrieve information from DB’; 06:05:56 07.11.2011 ERROR (Class1.java:243) message=‘Error retrieving information from DB: NullPointerException’;

Multiple users (II) Mapped Diagnostic Contexts (MDC) public void serviceMethod1(String userName,…) { String userName = getLoggedUserName(); MDC.put( “userName”, userName); } Configuration …PatternLayout=%T User=%X{userName};message=‘%m’;%n

Multiple users (III) public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException { HttpServletRequest req = (HttpServletRequest) request; Principal principal = req.getUserPrincipal();// Please note that we could have also used a cookie to // retrieve the user nameif (principal != null) { String username = principal.getName(); MDC.put(USER_KEY, username); } try { chain.doFilter(request, response); } finally { MDC.remove(USER_KEY); } }

Multiple users (III) public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException { HttpServletRequest req = (HttpServletRequest) request; Principal principal = req.getUserPrincipal(); // Please note that we could have also used a cookie to // retrieve the user name if (principal != null) { String username = principal.getName();MDC.put(USER_KEY, username);} try { chain.doFilter(request, response); } finally {MDC.remove(USER_KEY);} }

Multiple users - Example 06:05:55 07.11.2011 INFO (Class1.java:41) User=bfernan; message=‘Start method1’; 06:05:55 07.11.2011 DEBUG (Class1.java:44) User=bfernan; message=‘Before retrieve information from DB’; 06:05:55 07.11.2011 DEBUG (Class1.java:46) User=bfernan; message=‘After retrieve information from DB’; 06:05:55 07.11.2011 INFO (Class1.java:50) User=bfernan; message=‘Updating information’; 06:05:55 07.11.2011 DEBUG (Class1.java:94) User=bfernan; message=‘Before save information in the DB’; 06:05:56 07.11.2011 DEBUG (Class1.java:96) User=bfernan; message=‘After retrieve information in the DB’; 06:05:56 07.11.2011 INFO (Class1.java:100) User=bfernan; message=‘Finish method1’; 06:05:56 07.11.2011 INFO (Class1.java:211) User=xxxxxxx; message=‘Start method2’; 06:05:56 07.11.2011 DEBUG (Class1.java:220) User=xxxxxxx; message=‘Before retrieve information from DB’; 06:05:56 07.11.2011 ERROR (Class1.java:243) User=xxxxxxx; message=‘Error retrieving information from DB: NullPointerException’;

Logging for Web Services • What to log? • Where the logs go to?

What to log? • Platform Level • Time and Memory • Stamp and profiling tools • Database access • Queries and size of result • Network incoming and outgoing traffic • Requests and size of packets • System software usages • Parameters

What to log? • Third Party Services • Time stamp • Parameters • Size of responses

Where the logs go to? • Log files • Using virtual machine • Databases • Using data store services by the platform • Web services • Using a web service to show the logs

Summary • Basic concepts • Auditing, logging, and tracing • Logging levels • Multi-user logging • Logging of web services

Lecture XI: Logging and Monitoring