Intelligent Event Processing (IEP)

1. Intelligent Event Processing (IEP) Aggregation Filtering Partition Correlation

2. Aggregation Time-based Aggregator Tuple-based Aggregator Window Aggregator

3. Time-based Aggregator Compute statistical summery over a fixed time period. • Statistics: sum, average, median, lower-quantile, upper-quantile, min, max, standard-deviation, sum-of-square, outlier-percentage. • Example: Given a stream of transactions of a stock, computes a new stream that holds hourly min, average, and max of the stock price.

4. Tuple-based Aggregator Compute statistical summery over a fixed number of events. • Statistics: same as time-based aggregator. • Example: Given a stream of transactions of a stock, computes a new stream that holds the min, average, and max of the stock price of every 10 transactions.

5. Window Aggregator Compute statistical summery over a sliding window • Statistics: same as time-based aggregator. • Example: Given the latest 2 hour window of transactions of a stock, computes the min, average, and max of the stock price of those transactions. As time passes by, the statistical summery will change too.

6. Filtering Standard Filtering Filtering by Point Estimation Filtering by Hypothesis Testing

7. Standard Filtering • Example: Given a stream of transactions of a stock, computes a new stream that holds only those transactions of which the stock price is > $50.

8. Filtering by Point Estimation Analyze historical data, and build a Statistical model. Create a filter that filters out only those events that are considered outliers by the statistical model. • Example: Given enough historical measurements of human body: (weight (kg), height (cm)), one build a Statistical model: weight ~ -105 + height + N(0, 10). Given a stream of measurement data, computes a new stream that holds those measurement of which weight is outside of (-105 + height - 20, -105 + height + 20)

9. Filtering by Hypothesis Testing Analyze historical data and make up a hypothesis. Creates a filter that filters out only those groups of events that failed the hypothesis. • Example: Given enough historical measurements of products produced by an assembly line, one found that the defect rate is 2%. Make up a hypothesis: defect rate <= 2% Given a stream of measurement data, computes a new stream that holds alert events. Test the hypothesis for every 100 measurements. If the defect rate is Statistically greater than 2%, generate an alert event.

10. Partition Standard Partition Partition by Classification

11. Standard Partition Example: Given a stream of stock transactions, computes a window that holds the latest 10 transactions for each stock symbol.

12. Partition by Classification Analyze historical data and build a Statistical classification model. Creates a partition window that holds the latest N events for each classification group.

13. Correlation Relational Join of Multiple Windows

14. Relational Join of Multiple Windows Example: Given the latest 2 hour window of stock transactions, and the latest 2 hour window of news, computes the latest 2 hour window of insider-trades by joining the trader’s name with the name mentioned in the news.