1 / 37

Oracle CEP : Real Time Application Training

Oracle CEP : Real Time Application Training. Osman Uygar KOSE osmnauygar@gmail.com uygar.kose@etiya.com www.etiya.com 2014. Agenda. Introduction Oracle CEP Installation Simple Demo Oracle CEP Adapters Oracle CEP Channels and Processors Introduction to Oracle Continuous Query Language

darryl
Download Presentation

Oracle CEP : Real Time Application Training

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Oracle CEP : Real Time Application Training Osman Uygar KOSE osmnauygar@gmail.com uygar.kose@etiya.com www.etiya.com 2014

  2. Agenda • Introduction • Oracle CEP Installation • Simple Demo • Oracle CEP Adapters • Oracle CEP Channels and Processors • Introduction to Oracle Continuous Query Language • CQL Design Patterns • Oracle Visualizer • Coherence and caching

  3. Financial Services Transportation & Logistics Public Sector & Military Manufacturing Utilities & Insurance Telecommunications & Services Event Driven Applications Distributed order orchestration Asset management • Need to support one or more of: • High volume • Continuous streaming • Sub-millisecond latency Algorithmic trading • Disparate sources • Time window processing • Complex pattern matching ‘Negative Working Capital’ inventory management • Proximity/Location Tracking • Intrusion detection systems • Military asset allocation Grid Infrastructure Management Reponses to calamities – earthquake, flooding

  4. Complex Event Processing (CEP/EDA) • Complex Event Processing (CEP) is continuous and incremental of event streams from multiple sources based on declarative query and pattern specifications with near-zero latency

  5. Key Element of event Stream Processing • Event • Event stream • Event type • Event processing network

  6. Event What is an event and how is it defined? Many people and technical societies define an event in many different ways, but in the context of CEP, an event is an object that has a change in its state immediately, or over a period of time.

  7. Event Stream Generated by hardware sensor devices, distributed anywhere from the "Internet of things", computer applications, database triggers, or generated from any of hundreds of different sources, events arrive for processing in an event stream or streams. Event streams can have events that are continuously flowing at high volumes or arrive in sporadic intervals, but the events never end and are always time ordered, just like in the real world.

  8. Event Type The event types that flow "along" the event stream defines the properties associated with the event. Event type definitions can range in their levels of complexity, but in most applications can be declaratively defined with a simple notation.

  9. Event Processing Network So now we have an event, probably thousands or millions of them that need to be effectively handled and processed. As these events continuously flow they need to be identified, have a response very quickly and are often "transported" only in memory, so using a database is not a recommended design option. For this purpose, many Complex Event Processing platforms provide the Event Processing Network (EPN) (otherwise known as a Directed Flow Graph). • Event Adapters • Event Channels • Event Cache • Event Processor

  10. Event Adapters Event Adapters provide the connectivity to event sources and sinks, and are relatively simple code implementations that normalize the incoming or outgoing data stream and convert this into event types that are processed downstream or upstream in the EPN.

  11. Event Channel Event channels are the conduits that effectively handle the routing of events, these event nodes not only play an important role in ensuring that the various events are analyzed efficiently, but they also can have properties that can powerfully effect the performance of the application, such as controlling the amount of memory used for the events and the number of processing threads.

  12. Event Cache Event cache and event POJO Bean nodes provide the in-memory persistence of long-term reference data and the solution-specific business logic written as a "Plain Old Java Object". These event nodes ensure that information needed for long periods of time can be managed, interrogated, and safely held in computing memory, and that any type of additional processing logic can be implemented.

  13. Event Processor Event processors are the meta-containers for the powerful event analysis needed for any type of solution. There can be one or many event processor nodes in an application and they store the event processing language, which can be rules or queries that statically executes continuously on the flow of arriving events.

  14. Example EPN

  15. HTTP Pub/Sub Engine Cluster Management Data Services Framework App Frameworks Spring (DM) Framework Services POJO WebLogic Application Grid WebLogic Real Time Core Event Infrastructure Extended Event Infrastructure Event Repository Data Caching Real Time Kernel Stream Management Complex Event Processing Latency Analyzer OSGi OSGi Security Logging Equinox OSGi™ Foundation Services Config/Admin JRockit DGC JRockit Oracle Complex Event Processor (11gR1) Oracle Complex Event Processing • Lightweight Java Application Server • Full environment for building and running Java applications • Services -- Security, Logging, User Mgmt • Module Framework - OSGi • Multiple-choice JVM • JRockit or WebLogic Real Time • Unique JVM Tools • High Throughput • Hundreds of thousands of events/second • Event Processing infrastructure • CEP Engine • Event Processing programming constructs and services • Time-critical streaming • Well-known event processing language • Easy-to-Use Development Environment • Service Framework – Spring DM, POJO • Enterprise Web 2.0 & Eclipse-based tooling • Server Management & Monitoring (Visualizer) • Visual Query Creation/Manipulation • RAD IDE for EDA/CEP Application Development • Visual EPN Editing/Dynamic Construction • Enhanced EPL Editing Capabilities

  16. Oracle CEP Installation Oracle CEP Installation Document

  17. Simple Demo Oracle CEP “HelloWorld” Demo

  18. Continuous Query Language (CQL) • System and functional capabilities: • High-volume stream processing • Complex relationships (e.g. non-existence, followed by) • SQL-based • Leverages skill-set • Integrates to relation-data • Extensions • Semantic differences: deals with streams, as opposed to tables • Additional clauses for Temporal Conditions

  19. Constant time interval t1 t2 t3 t4 t5 t6 t7 Time-based Window E3 E4 E1 E2 Time CQL: SELECT * FROM MyEvent [RANGE 3 SECONDS] Stream to Relation Operator • Constant time interval • Number of events in Relation will vary • Events will expire from Relation over time

  20. Constant time interval t1 t2 t3 t4 t5 t6 t7 Time-based Window E3 E4 E1 E2 Time CQL: SELECT * FROM MyEvent [RANGE 3 SECONDS] • Constant time interval • Number of events in Relation will vary • Events will expire from Relation over time

  21. Constant time interval t1 t2 t3 t4 t5 t6 t7 Time-based Window E3 E4 E1 E2 Time CQL: SELECT * FROM MyEvent [RANGE 3 SECONDS] • Constant time interval • Number of events in Relation will vary • Events will expire from Relation over time

  22. Constant time interval Time t1 t2 t3 t4 t5 t6 t7 Time-based Window E3 E1 E2 E4 CQL: SELECT * FROM MyEvent [RANGE 3 SECONDS] • Constant time interval • Number of events in Relation will vary • Events will expire from Relation over time

  23. t1 t2 t3 t4 t5 t6 t7 Row-based Window Constant # of events E3 E4 E1 E2 Time CQL: SELECT * FROM MyEvent [ROWS 2] Stream to Relation Operator • Constant number of events in Relation • Events will not expire from Relation until replaced

  24. t1 t2 t3 t4 t5 t6 t7 Row-based Window Constant # of events E3 E4 E1 E2 Time CQL: SELECT * FROM MyEvent [ROWS 2] • Constant number of events in Relation • Events will not expire from Relation until replaced

  25. t1 t2 t3 t4 t5 t6 t7 Row-based Window Constant # of events E3 E4 E1 E2 Time CQL: SELECT * FROM MyEvent [ROWS 2] • Constant number of events in Relation • Events will not expire from Relation until replaced

  26. Incremental vs Batched Window • Events continuously enter and exit windows • Events processing can be incremental or batched … • Incremental • Queries are evaluated on each change • Batched • Queries are evaluated only when a define number of changes have occurred • Incremental processing is the default for CQL

  27. Sliding vs Batched Window Incremental Window Batched Window CQL SELECT SUM(price) FROM Prices [ROWS 3] SELECT SUM(price) FROM Prices [ROWS 3 SLIDE 3]

  28. Sliding and Batched Window Incremental Window Batched Window CQL SELECT SUM(price) FROM Prices [ROWS 3] SELECT SUM(price) FROM Prices [ROWS 3 SLIDE 3]

  29. Sliding and Batched Window Incremental Window Batched Window CQL SELECT SUM(price) FROM Prices [ROWS 3] SELECT SUM(price) FROM Prices [ROWS 3 SLIDE 3]

  30. Sliding and Batched Window Incremental Window Batched Window CQL SELECT SUM(price) FROM Prices [ROWS 3] SELECT SUM(price) FROM Prices [ROWS 3 SLIDE 3]

  31. Sliding and Batched Window Incremental Window Batched Window CQL SELECT SUM(price) FROM Prices [ROWS 3] SELECT SUM(price) FROM Prices [ROWS 3 SLIDE 3]

  32. Sliding and Batched Window Incremental Window Batched Window CQL SELECT SUM(price) FROM Prices [ROWS 3] SELECT SUM(price) FROM Prices [ROWS 3 SLIDE 3]

  33. Sliding and Batched Window CQL SELECT SUM(price) FROM Prices [ROWS 3] SELECT SUM(price) FROM Prices [ROWS 3 SLIDE 3]

  34. Sample CQL Queries Simple Filtering SELECT * FROM inputChannel [NOW] WHERE eventValue > 10 Continuously calculate the last hour sales by store SELECTSUM(amount) assalesTotal, storeID FROMinputChannel [range 60 minutes] GROUPBYstoreID Calculate the average of the last 2 stock ticks by stock symbol SELECTAVG(stockPrice) asavgPrice, stockSymbol FROMinputChannel [PARTITIONBYstockSymbolROWS 2] GROUPBYstockSymbol Filter for events meeting specific threshold values Running total of up- to-the-moment sales by store Average of the last 2 stock ticks by symbol

  35. Sample CQL Queries Find passengers stuck in security when their flight reaches “final boarding”. SELECT stuck.reservationLocator, 'STUCK' as state FROM PassengerStateEventChannel MATCH_RECOGNIZE ( PARTITION BY reservationLocator MEASURES Entered.reservationLocatorAS reservationLocator PATTERN (CheckIn Entered NotExited*? Final) DEFINE CheckInAS state = 'CHECKIN', Entered AS state = 'ENTERED', NotExitedAS state != 'EXITED', Final AS state = 'FINAL' ) AS stuck Find passengers who are stuck in security when their flight is in the “FINAL BOARDING” process.

  36. Sample CQL Queries Track a bus on a map Track School Buses on a Map SELECT bus.busId as busId, bus.seq as seq, com.oracle.cep.cartridge.spatial.Geometry.createPoint(8307, bus.longitude, bus.latitude) as geom FROM BusPosStream as bus Alert when the Bus arrives at the Bus Stop SELECT systimestamp() as incidentTime, bus.busId as busId, busstop.seq as stopSeq FROM BusPosGeomStream[NOW] as bus, BusStopRelation as busstop WHERE CONTAIN@spatial(busstop.geom, bus.geom, 100.0d) = true and bus.busId = busstop.busId Determine when the bus is near the bus stop

  37. That’s all

More Related