1 / 23

Real-Time Databases and Data Services

Real-Time Databases and Data Services. Krithi Ramamritham, Sang Son, Lissa Dipippo. Satisfying QoS/QoD requirements. Transaction exec times & data access patterns are not known a priori, but vary dynamically Transaction timeliness & data freshness may pose conflicting requirements. Motivation.

norman
Download Presentation

Real-Time Databases and Data Services

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Real-Time Databases and Data Services Krithi Ramamritham, Sang Son, Lissa Dipippo

  2. Satisfying QoS/QoD requirements • Transaction exec times & data access patterns are not known a priori, but vary dynamically • Transaction timeliness & data freshness may pose conflicting requirements

  3. Motivation • Increasing amount of sensor data • Agile manufacturing, target tracking, surveillance, structual monitoring, weather forecast, traffic control, ... • Wireless sensors push the limit • Desired real-time data service • Timely query/transaction processing using fresh data • Existing databases based on closed-world assumption • No notion of sensor data service • Timing guarantee or data freshness not considered

  4. Real-time data Service Update Streams QoS Guarantees? User Requests Real-Time Transaction Processing Poor QoS . . . Overload Data Conflicts

  5. Standford real-time information processor (STRIP) • Addressed the problem of balancing between the freshness & timeliness requirements • Soft real-time: Maximize #transactions finishing within the deadlines using fresh data • Apreriodic updates • Freshness metrics • MA (Max Age): Similar to avi • Unapplied Updates: Be optimistic & assume a data is fresh unless an update has been received but not installed yet

  6. STRIP • Four scheduling algorithms • Update first • Transaction first • Split Updates • Update to high importance data will be installed upon arrival; otherwise, scheduled after transactions • On Demand • Transaction has precedence, but searches update queue upon accessing a stale data

  7. QMF (QoS-aware Miss ratio and Freshness guarantees) • Guaranteed Quality of real-time data service • Timing guarantee • Support desired deadline miss ratio or response time • Home-land security problem, traffic jam, ... • Quality of Data • Data freshness (temporal consistency) • Reflect the time-varying real-world status

  8. Challenges • Unpredictable workloads, access patterns • Sudden increase of service requests • Severe data contention • Conflicts between timing & freshness requirements • Some deadline misses/freshness violations are inevitable

  9. Key Ideas • Feedback control • Robustness against unpredictable workloads • Widely applied to software peformance management • Feedback Control of Computing Systems, Joseph L. Hellerstein et., Wiley Interscience • Cost-effective freshness management • Novel freshness metrics • Cost-benefit model • Adaptive update policy • Admission control • Tardy commits are worse than dropping

  10. QMF Architecture U. Thresh. Manager MR/Util. Controllers MR, Util. U Unew QoD Manager User Trans. Trans. Handler Dispatch Abort/Restart Admission Control … … … Update Streams Ready Queue Block Queue

  11. Real-Time Database Model • Main memory database model • TimesTen, STRIP, DataBlitz, Polyhedra • Firm deadlines • Sensor data updates • Periodic • Jitter? • User transactions • Arithmetic/logical operations considering the current real-world state

  12. QoS Metrics: Miss Ratio • Admitted transactions • Average • Transient • Overshoot (V) • Settling time (Ts) V Ts

  13. QoS Metrics: Data Freshness RTDB Database Freshness: Set of temporal data Perceived Freshness: Set of temporal data accessed by timely transactions

  14. Feedback-Based Miss Ratio/Utilization Control Transactions MR Threshold Current MR Error W MR Controller RTDB + _

  15. Feedback Control Details • PI controllers • Sampling period • Settling time, overshoot • 5sec • Overhead • QoD fluctuations • Think time • Tuning • Root Locus method in Matlab

  16. Cost-Effective Updates • Access Update Ratio (AUR) • AUR[i] = Access Freq[i] / Update Freq[i] • Access Frequency • Benefit • Update Frequency • Cost

  17. Cost-Benefit Model Hot Data AUR = 1 Cold Data

  18. Adaptive Update Policy D = Dimm Dimm Dimm AUR =1 AUR < 1 Dod Dod Underutilized State Moderately loaded State Overloaded State

  19. Future Work • More flexible QoD metrics and adaptive QoD management schemes • More effective feedback control & QoS/QoD management approaches • RTDB testbed • Apply RTDB techniques to e-commerce applications • 3-tier systems: Web server, application server & back-end database • Mobile, hand-held RTDB

  20. Real-time data service in embedded applications • Data needs of embedded applications become more complicated • Traffic control • Weather forecast • Put RT data server in the frontline • Collect info rather than raw data and disseminate in a timely manner • Key issues such as power/energy management should be reconsidered considering real-time data service semantics

  21. Wireless Sensor Networks • View WSN as a distributed DB (???) • TinyDB, Cougar, SINA, ... • I don’t agree, but you can have different opinions... • Real-time event detection • Real-time routing • Data aggregation • QoD in WSNs – anything more than freshness? data values? accuracy? SNR? • Security & Trustworthiness • Scalability of depandability • Resilience of availability

  22. Vision (Example) Real-time traffic control using sensor data and weather information

  23. Questions?

More Related