Application Security Logging and Monitoring The Next Frontier

1. Peter Freiberg Principal Consultant Shelde pfreiberg@shelde.com +61 3 9650 7852 Application Security Logging and MonitoringThe Next Frontier April 2012

2. Application Security Logging and Monitoring This talk will discuss: The four key challenges for application security logging and monitoring Common issues in current logging practices Current resources (or lack of) available to developers for security logging Tools for correlating and alerting from log sources Logging in multi-tiered architectures and disparate systems Which logging capabilities can be driven by application security and what types of logging might be required by audit and the business The Next Frontier

3. Key challenges for Application Security Logging Lack of Security Logging Frameworks Lack of guidance on what and how to log Lack of requirements for security logging Lack of correlation and alerting capabilities

4. Where is the real threat? According to the 2010 Fraud and Misconduct Survey by KPMG Australia 65% of Fraud was from ‘Inside Jobs’ This represented 98% of the total fraud loss. Do you think an insider used SQL injection or XSS to commit fraud? (Probably not) Source: https://www.kpmg.com/AU/en/IssuesAndInsights/ArticlesPublications/Fraud-Survey/Documents/Fraud-and-Misconduct-Survey-2010.pdf

5. Key challenges for Application Security Logging Many applications have poor security logs (and sometimes have none at all) Without good security event information it is difficult to: detect attacks detect compromised user account detect fraud detect abuse of privileges and Respond to events

6. Challenge #1 Lack of Security Logging Frameworks

7. Java Common Logging Severity Levels Where is the security log level? Source: http://en.wikipedia.org/wiki/Java_logging_framework

8. Current Logging frameworks are focused on identifying where a technical fault has occurred.

9. Challenges for detail in logging functions Example given to developers public class Foo { private Log log = LogFactory.getLog(Foo.class);public void foo() { ... try { } … } catch (IllegalStateException e) log.error("Something bad happened to " + name, e); } Source: http://commons.apache.org/logging/apidocs/org/apache/commons/logging/package-summary.html Log Event Data should include the following: 1. Time stamp from a trusted system component 2. Severity rating for each event 3. Tagging of security relevant events, if they are mixed with other log entries 4. Identity of the account/user that caused the event 5. Source IP address associated with the request 6. Event outcome (success or failure) 7. Description of the event Source: https://www.owasp.org/index.php/OWASP_Secure_Coding_Practices_-_Quick_Reference_Guide

10. Security Logging needs to be more tightly coupled with the user performing the action (instead of the object class where the error occurred)

11. How to add a security log level in Framework X OWASP guide on how to add a security log level in the Java log4j framework. https://www.owasp.org/index.php/How_to_add_a_security_log_level_in_log4j However, there are lots of logging frameworks.

12. There are many logging frameworks Java (commonly used) Commons Logging Log4j Logback SLF4J Other frameworks Craftsman Spy, Houston, jLo, Jmyra, JTraceDump, Just4log, Limpid Log, Logging Toolkit, Monolog, ObjectGuy Framework, Protomatter, RP Logging, Simple Log, SmartInspect,TraceTool Source: http://www.java-logging.com/ • .Net(commonly used) • Microsoft Enterprise Library • Log4net • Logger.NET • NLog • Other frameworks • C# Logger, CommonData, CSharp Dot Net Logger, DebugWriter, LogThis, NetTrace, Nspring, ObjectGuy Framework, SmartInspect, TcpTrace, Traceract, TraceRT.NET, Traffic Monitor • Source: http://www.dotnetlogging.com/

13. Other considerations for Logging Frameworks Data Validation (Who does the escaping – the developer or the framework?) XSS in web based logging interfaces Log Forging(Can an attacker guess the log format and alter it using carriage return injection?) Signal to noise ratio Separating Application Logs and Security logs Maybe the noise is the signal?(Is 40,000 exceptions and stack traces an attack?) Access control of security logs Who has access to view or modify the security logs?(Are they just text files? Or database rows? Or centrally stored?)

14. Other considerations for Logging Frameworks Storage growth requirements (e.g. What volume of log data is generated per day or month? 1 MB, 100MB, 1GB?) Data retention periods (e.g. How long do you need logs for? 3 months? 3 years?) Regulatory restrictions on centralised logging solutions (e.g. It might not be allowed to centralise application logs outside of a country’s borders)

15. Challenge #2 Lack of guidance on what and how to log

16. What detail needs to be logged? What developers have log.error("Something bad happened to " + name, e); • What security people want • Time stamp • Severity rating for each event • Identity of the account/user that caused the event • Source IP address associated with the request • User context across application tiers (internal webservice, MQ, Database) • Event outcome (success or failure) It would be nice if we had something like: Log.security( UserContext, // containing username, ip address, browser etc ApplicationEvents.DataValidationFailure, e); Free form text for log messages makes it harder to identify and correlate events

17. The level of details for logging • Log Event Data should include the following: • 1. Time stamp from a trusted system component • 2. Severity rating for each event • 3. Tagging of security relevant events, if they are mixed with other log entries • 4. Identity of the account/user that caused the event • 5. Source IP address associated with the request • 6. Event outcome (success or failure) • 7. Description of the event • Source: https://www.owasp.org/index.php/OWASP_Secure_Coding_Practices_-_Quick_Reference_Guide Challenges: • What event types are there (technical and logical) and are they defined for the application and the organisation? • What rating do you give events? • Do you have the details of the requesting user? (username, IP address, etc) • Are the logs in a format that can be easily consumed by logging and correlation tools

18. High level security logging requirements Ensure log entries that include un-trusted data will not execute as code in the intended log viewing interface or software Restrict access to logs to only authorized individuals Utilize a master routine for all logging operations Do not store sensitive information in logs, including unnecessary system details, session identifiers or passwords Ensure that a mechanism exists to conduct log analysis Source: https://www.owasp.org/index.php/OWASP_Secure_Coding_Practices_-_Quick_Reference_Guide

19. High level security logging events Log all input validation failures Log all authentication attempts, especially failures Log all access control failures Log all apparent tampering events, including unexpected changes to state data Log attempts to connect with invalid or expired session tokens Log all system exceptions Log all administrative functions, including changes to the security configuration settings Log all backend TLS connection failures Log cryptographic module failures Challenge with this guidance: Mostly technical nature Source: https://www.owasp.org/index.php/OWASP_Secure_Coding_Practices_-_Quick_Reference_Guide

20. Starting Point - OWASP AppSensor Project Overview The AppSensor project defines a conceptual framework and methodology that offers prescriptive guidance to implement intrusion detection and automated response into an existing application. Current efforts are underway to create the AppSensor tool which can be utilized by any existing application interested in adding detection and response capabilities.  Detection AppSensor defines over 50 different detection points which can be used to identify a malicious attacker. Response AppSensor provides guidance on how to respond once a malicious attacker has been identified. Possible actions include: logging out the user, locking the account or notifying an administrator. More than a dozen response actions are described. Source: https://www.owasp.org/index.php/OWASP_AppSensor_Project

21. What to log? What level of detail is required to identify a user and reliably trace back to an unauthorised malicious action? Where are they? (is it a green screen internal application? An internet facing web application, a corporate desktop application?) What technical details can be identified and logged? (e.g. XSS, Change of IP address mid-session, data validation issues etc.) What business level detail must be logged? (e.g. viewing sensitive data, who did what for segregation of duties, etc.)

22. Challenge #3 Lack of requirements for security logging

23. Types of Business Events • Logical, Behavioural and Compliance • Privileged User Access • Process Violations • Segregation of Duties bypass • Bulk Downloads • Privacy Violations • Influencers: • Business Owners • Internal Audit • Regulatory Bodies (PCI-DSS, SOX, Government, etc.) • Operational Risk • Fraud Teams • Security Consultants

24. Challenges for Security Logging requirements • Communication and Education • Does the fraud team talk to the business, security and developers? • What incidents have occured in the past? Could these have been avoided? • How can requirements be defined that can be technically implemented and tested? • How many applications and systems need to log to gain value by reducing risk and fraud? • Technology: • What will be used to monitor and correlate events? • Business: • Who pays for the monitoring technology?(Is it a security initiative or business solution?)

25. Challenge #4 Lack of correlation and alerting capabilities

26. Correlation and alerting capabilities If an business application has made it this far, its got: Some type Security Logging Framework Some list of security event types (including technical, logical and process) Business requirements for logging So, we’ve got security event logs, what do we do with them?

27. Correlation Technology Security Information and Event Management (SIEM)technology provides: Security Information Management (SIM) – log management and compliance reporting Security Event Management (SEM) – real-time monitoring and incident management for security-related events from networks, security devices, systems and application SIEM Technology is typically deployed to support three primary use cases: Compliance – Log management and compliance reporting Threat Management – real-time monitoring of user activity, data access and application activity and incident management A deployment that provides a mix of compliance and threat management capabilities Source: Gartner Magic Quadrant for SIEM. http://www.arcsight.com/collateral/whitepapers/Gartner_Magic_Quadrant_2011.pdf

28. Event correlation requires a broader view

29. Event correlation requires a broader view • Often business processes rely on many applications and systems. To see the enterprise view or end to end process correlation needs to occur outside. Enterprise Event Correlation & Response Centralised Log Aggregation & Management Application 1Meaningful Logs Application 2Meaningful Logs Application 3Meaningful Logs Application 4Meaningful Logs … Application NMeaningful Logs

30. Where should correlation occur? Enterprises typically have more than one interface for a given application. For example: • Web application for customers • Green screen Mainframe application for back end administration • Desktop GUI for Help Desk staff Abuse case or threat scenario: • An administrator using the green screen application changes a help desk staff password who is on holidays • then logs on as a help desk operator and changes a customers password • then logs on as the customer from an internet café Can you detect this scenario in any single application?

31. Solving these challenges Technology challenges Lack of Security Logging Frameworks Lack of guidance on what and how to log Business challenges Lack of requirements for security logging Lack of correlation and alerting capabilities

32. Solving these challenges Where to start? Focus on high risks and incidents What risk or impact are you minimising: Reduced internal fraud Compromised user accounts (leading to external fraud) Process Violations Compliance with regulatory bodies External compromise

33. Solving these challenges Technology challenge Security Logging Frameworks Possible solutions: OWASP Project providing security in common Logging Frameworks Security professionals working with the producers of software(E.g. Oracle Java, Microsoft, PHP) Create a reference implementation

34. Solving these challenges Technology challenge 2. Guidance on what and how to log Possible solutions: Logging frameworks that support security Defined event types with example implementations and outputs(Supporting projects like AppSensor which are defining event types and responses will help build industry knowledge) Building and sharing experience through use of centralised logging, monitoring and correlation tools(Experience in the trenches of real applications and business will help drive what works and what doesn’t) Event Standards like Common Event Expression (CEE)(see http://cee.mitre.org)

35. Solving these challenges Business challenge 3. Requirements for security logging Possible solutions: Make it easy for the business – defined logging events allow for easier communication and implementation Better Education and Consulting – How will this help the business? Break down corporate silos (Understand what Fraud and Incidence response is occurring, what audit requirements exists and socialise with security and development for implementation) Be Realistic – What value will security logging and monitoring provide and what risks can this help mitigate

36. Solving these challenges Business challenge 4. Correlation and alerting capabilities Possible solutions: Look for Compliance drivers such as SOX, PCI-DSS(Spending on monitoring and correlation infrastructure can’t always be justified, possibly can piggy back off other initiatives) Adopt security logging standards for applications(Having key applications log security events is better than waiting for the full enterprise solution.) Be realistic – what value will security logging and monitoring provide Common event types and logging formats

37. Application Security Logging and Monitoring Summary: The four key challenges for application security logging and monitoring Common issues in current logging practices Current resources (or lack of) available to developers for security logging Tools for correlating and alerting from log sources Logging in multi-tiered architectures and disparate systems Which logging capabilities can be driven by application security and what types of logging might be required by audit and the business The Next Frontier

38. Questions? Application Security Logging and MonitoringThe Next Frontier Peter Freiberg Principal Consultant Shelde pfreiberg@shelde.com +61 3 9650 7852