1 / 1

Improving Efficiency of the KURT-Linux Data Streams Performance Evaluation Framework

Improving Efficiency of the KURT-Linux Data Streams Performance Evaluation Framework. Solution Approach. Problem Description. Key design and performance issues identified: Instrumentation effect Data loss when DS daemon falls too far behind Event logging overhead variance

percy
Download Presentation

Improving Efficiency of the KURT-Linux Data Streams Performance Evaluation Framework

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Improving Efficiency of the KURT-Linux Data Streams Performance Evaluation Framework Solution Approach Problem Description • Key design and performance issues identified: • Instrumentation effect • Data loss when DS daemon falls too far behind • Event logging overhead variance • Due to occasional event buffer management • Heavy load on local file system • Lack of file system on many embedded targets • Methods addressing issues: decrease overheads, increase efficiency, and reduce data loss • Group Scheduling of the DS daemon (1&2) • Data collection aware scheduling • Creation of Buffer Management Kernel Thread (BMKT) (1&3) • Creation of Direct Logging Kernel Thread (DLKT) (3) • Direct logging of DS data to a file (4) • Direct logging of DS data to a socket (4&5) • Increasingly complex Distributed Real-Time and Embedded (DRE) systems must satisfy complex and precise behaviour constraints • Performance evaluation methods must match this precision • Interference of measurement methods with the system being measured (Instrumentation Effect) must be minimized • Data Streams (DS) has used a simple user-space daemon to read data from the OS and write the data stream to a file • System analysis data should suffer little (or no) loss • Coarse control of daemon and caused data loss in high volume and performance constrained test situations • Goal of this project was to identify key components affecting DS function and develop methods to decrease overhead and loss • Several DS configuration options have proved useful Example Experimental Results Extended Data Streams Architecture • Greedy process logging events • 15 million events in 10 minutes • 25000 events every second • 1 event every 40 microseconds • Top: Overhead of DS daemon writing event buffers under group scheduling control into a local file • Vastly decreases data loss • Bottom: Overhead of writing DS event buffers into a socket inside OS • Data loss only in network • Dedicated DS Ethernet fixes this • Data Streams enhancements have • Decreased overhead, data loss, instrumentation effect Group Scheduling of DS Daemon BMKT removes buffer allocation overhead from the application thread, reducing event logging variance DLKT reduces data loss and overhead by logging event buffers from OS • Reducing overhead and group scheduling of event buffer logging reduces instrumentation effect and data loss, improving DS quality 040405

More Related