1 / 26

Normalizing Data for use in Forensics Analysis

Normalizing Data for use in Forensics Analysis. Major Activities. XML Parser for the Application Log data provider Metadata map Forensics template Modified Forensics.xml file to make custom forensics template available in the Control Center Console

zudora
Download Presentation

Normalizing Data for use in Forensics Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Normalizing Data for use in Forensics Analysis

  2. Major Activities • XML Parser for the Application Log data provider • Metadata map • Forensics template • Modified Forensics.xml file to make custom forensics template available in the Control Center Console • Rule to archive data from the application log

  3. Custom XML Parser

  4. Metadata • Used to tag events with information to store the data in an IDMEF compliant manner in the database • IDMEF – Intrusion Detection Message Exchange Format http://www.rfc-editor.org/rfc/rfc4765.txt • Used to correlate different event properties (i.e. Source IP from two different events) • Maps are built in an XML file and imported into the database • Mapping is stored in the OnePoint database • MetadataFieldDefinition • MetadataLocalization • MetadataMap • MetadataMapMain

  5. Field Map • Used to uniquely identify fields to be used in mapping • Each field that can be used in a metadata map is defined here • Beginning in SM 6, the field map can no longer be modified • Any prior modifications will be removed. • Datatypes are restricted in the field map • Example mapping of standard IDMEF field: <field id="501" IDMEF="analyzer.model" datatype="string"> <Locale id="1033"> <DisplayName>Device Model</DisplayName> </Locale> </field> • Example mapping for an additional data field: <field id="915" IDMEF="Signature Name" datatype="string"> <Locale id="1033"> <DisplayName>Signature Name</DisplayName> </Locale> </field>

  6. The Hierarchy of Maps Catchall Map • Metadata Indexes must be unique at each level • As an event flows down the system, additional properties are mapped Event Source Map EventID Map • The most specific mapping definition is the one that will be applied. DLL Specific Map

  7. Metadata Hierarchy (Cont…) • Catchall – used only by NetIQ to ensure that specific fields are mapped so that they are available to the system for reporting purposes • Event Source - must be unique throughout the entire system • Event ID - Metadata Index should start at 1 and must not be repeated per Event Source • Dll Specific - last level and it follows the same rules as Event ID

  8. Constructing the Map – EventIDMap <EventIDMap name=“TippingPoint” id=“0” version=“5.50.0.158”> </EventIDMap> • Name is an arbitrary name for the map (and cannot be renamed) • Can leave the id = 0 since SM will automatically assign this value when you import the map • Version is used in the import process • When incrementing the version number do not make major leaps in the versioning. Use the lowest level version segment to increment it.

  9. Constructing the Map – Source <Source id=“TippingPoint” MetadataIndex=“1002”> <Params> Define Parameters here that are consistent throughout the event source </Params> </Source> Source id MUST match the value you put for the event source in the XML parser in order for SM to determine that this is the appropriate map for that event • MetadataIndex number must be unique (for custom maps, use ID numbers 1000-9999). Don’t use ID 1001!

  10. Constructing the Map – Event <Event id=“200” MetadataIndex=“1”> <Params> Define Parameters specifically for this event id. </Params> </Event> Event IDs should align to what has been defined in the parser. • MetadataIndex always start at “1” at the Event level and increase for each new event defined

  11. Constructing the Map – DLL Specific <DLLSpecific dllname="MSAuditE.dll" start="5.0.2149.1" end="6.0.0.0" MetadataIndex="1"> <Params> Define Parameters here that are consistent throughout the particular message dll. </Params> </DLLSpecific> DllSpecific is used for events coming from a Windows NT event log. Requires the raw name of the msg dll. MSAuditE.dll is the Security Log. SM_Msgs.dll is the Security Manager dll. The start and stop are the version ranges of the dll that will have this map applied. • MetadataIndex always start at “1” at the Event level and increase for each new event defined

  12. Constructing the Map – Defining Parameters • The Params section is the where the mapping to IDMEF actually occurs. <Params> <param field=“source.node.address.address” source=“6”/> <param field=“target.node.address.address” source=“8”/> <param field="Logon Type" source="4" additionaldata="true" /> </Params> The numbers refer to parameters - “6” = “Parameter 6” Parameters that do not have a specific IDMEF Field have to be identified as being additional data by setting additionaldata = “true”.

  13. Supported Lookups & Operators • Lookups • Lookup=“SID” - resolves the SID contained in the specified parameter <param field="target.user.userid.name" source="1" lookup="SID"/> • Lookup=“IP” - reverse lookups the host name in the specified parameter <param field="source.node.Address.address" source="6" lookup="IP"/> NOTE: Using IP Address lookups can be expensive. If the name resolution fails, it has to time out before the map can be complete • Operators • string literals: <param field=“X" source=“’foo’"/> • concatenate: <param field="source.user.userid.name" source="2 + '\' + 1"/> • conditional:<param field="classification.name" source="if(6 = ‘foo') Then 5 else 3"/>

  14. Dynamic Event Properties • Dynamic event properties are pieces of each event that are typically defined in the workflow. • $GUID $ProviderInstance • $EventNumber $EventType • $EventTime $Category • $ProviderName $Message • $UserName $UserDomainName • $Domain $Computer • $AgentComputer $AgentDomain • $SourceName $MessageDLL • $DllVersion $Description • $EventImportance $AnalyzerModel

  15. Example of a Completed Map <EventIDMap name="Security Event Log" id="0" version="5.10.0.154"> <Source id="Security" MetadataIndex="0"> <Params> <param field="message" source="$Message" additionaldata="true" /> <param field="analyzer.model" source="'Windows'" /> </Params> (...) <Event id="642" MetadataIndex="43"> <Params> <param field="CLS" source="'PENTASAFE:3.2.2.55'" /> <param field="source.node.name" source="$Computer" /> <param field="source.node.Address.address" source="$Computer" lookup="IP" /> </Params> <DLLSpecific dllname="MSAuditE.dll" start="5.0.2149.1" end="6.0.0.0" MetadataIndex="1"> <Params> <!-- Windows 2000 and later  --> <param field="target.user.userid.name" source="2" /> <param field="source.user.userid.name" source="5" /> <param field="Account Change" source="1" additionaldata="true" /> <param field="target_account_id" source="4" lookup="SID" additionaldata="true" /> <param field="target_object_domain" source="3" lookup="SID" additionaldata="true" /> </Params> </DLLSpecific> <DLLSpecific dllname="MSAuditE.dll" start="4.0.1371.1" end="5.0.0.0" MetadataIndex="2"> <Params> <!-- Windows NT 4.0  --> <param field="target.user.userid.name" source="1" /> <param field="target_account_id" source="3" lookup="SID" additionaldata="true" /> <param field="source.user.userid.name" source="4" lookup="SID" /> </Params> </DLLSpecific> </Event> (...) </Source> </EventIDMap>

  16. Parameters of Interest • Analyzer.model • This is an arbitrary name. Typically this will be the data provider name. • Analyzer.node.name • This should be set to equal the parameter value indicating the device that generated the event. • Classification.name • This should be set to equal the parameter value of the Event ID number. • Any additional IDMEF field that is a part of the Reporting Cube

  17. MetadataImporter.exe This tool is contained in C:\Program Files\NetIQ Security Manager\Onepoint. Be careful….this tool has limited error checking capabilities – just because it says an import completed successfully does not necessarily mean that it did!

  18. Forensics Template • Allows you to generate platform specific forensic reports • Think about what fields you want to include in the report and associate them with the appropriate IDMEF data. • Must be copied to all UI machines in the following directory: • <InstallLocation>\NetIQ Security Manager\Onepoint\VSOC\Config directory

  19. Forensics Template Notice the syntax for “nonstandard” IDMEF metadata fields. Just like in the metadata map, you need to have a special syntax associated: The syntax is: <Field name=“Forensic label” idmef=“ADDITIONALDATA.DATA” add_meaning=“nonstandardIDEMEFfieldname” /> Forensic label = whatever you wanted the display name to be in the template nonstandardIDMEFfieldname = whatever you called the nonstandard field in the metadata map.

  20. Forensics.xml • Stored in the C:\Program Files\NetIQ Security Manager\Onepoint\VSOC\Config directory • Makes the custom forensics template you already created available in the dropdown menu • Add a row to the file Containing the name of the report type, the PRG GUID, and the name of the template XML file. • Note for 5.6 – the network node MUST be a member of an SM Computer Group in order for report to populate properly.

  21. Forensics.xml Row Name = Arbitrary display name for the forensic template PRGID = GUID of the PRG containing archive rules for the provider. Note: SM will list the computer groups associated with this GUID when defining a forensic query. The analyzer nodes that are returned by the query MUST be contained in one of these computer groups, or the data will not display in SM 5.6 (this is less of an issue in 5.0-5.5 since there is no security filtering on archival). AnalyzerModel = Whatever was defined as the analyzer model in the metadata map FILENAME = name of the XML file used for the Forensic template

  22. Adding Custom Computer Groups to the Cross Platform Template • The Cross Platform forensic template has a PRGID of “00000000-0000-0000-0000-000000000000” allowing it to return all of the computer groups associated with a “NetIQ” module.    • The forensics wizard queries the Modules tables and uses the idProcessRule to in turn look up all of the computer groups tied to a module.  • The correct way to address this is to insert a record into the Module table that builds the association to your custom PRG.  Here is a SQL query that will handle this insertion for you: DECLARE @RuleGUIDuniqueidentifier DECLARE @ModuleNamenvarchar DECLARE @Version nvarchar SET  @RuleGUID = 'F1D5A4A5-C05D-406E-BC61-7BB4658FAFAA'  --Insert the idProcessRule of your PRG here. SET @ModuleName = <Some Name>' --Insert a name for the module here. SET @Version = '5.60.001' --Insert a version number here. ------------Do not edit anything below this line------------ Insert INTO Module (idModule, Name, Description, idProcessRule, FeatureCode, Reserve, Lastmodified, Version, idNQMModule) VALUES  (NEWID(), @ModuleName, '', @RuleGUID, '', '', GETUTCDATE(), @Version, NULL) • Make sure you have a description in the Computer Group or you will create an unhandled exception in the Forensic Wizard. 

  23. Adding endpoints to CG outside of Config Wizard • To force inclusion in the computer group outside of a Configuration Wizard, run this stored procedure against the Onepoint database via SQL Query Analyzer: exec dbo.ComputerInsertNetworkNode 'domainname', 'computername', 'computergroupname' Or (if endpoint does not have a domain membership) exec dbo.ComputerInsertNetworkNode NULL, 'computername', 'computergroupname‘ • For example, to add the device “10.1.1.14” to the Snort Intrusion Detection computer group, you would run: exec dbo.ComputerInsertNetworkNode NULL, ’10.1.1.14', 'Snort Intrusion Detection'

  24. Best Practices • Before you even start on the XML parser, try to think about what IDMEF fields you want to report on. Keep notes on what IDMEF parameters these IDMEF fields will map to when constructing your parser. *Remember the dimensions used in the SMReporting OLAP Cube. • When making your parser, try to make sure that a given metadata value will always map to the same event parameter. This makes the metadata map a lot simpler. • Note that importing metadata maps will NOT overwrite what is already in the Onepoint database unless the version is a later version

  25. Best Practices - cont • You can test any text based log with a syslog simulator • NEVER modify the field map • Datatypes are restricted in the field map • Watch out if you want to use lookups in the metadata map • Be constantly aware of the dimensions available in the cube. These are the only fields that will be summarized. • And did I mention “Be constantly aware of the dimensions available in the cube.”

  26. Report Dimensions

More Related