1 / 14

Efficient Variable Reduction for Predictive Modeling with Clustering

This seminar covers topics on data storage, predictive modeling, and variable clustering for efficient model construction. It discusses the role of dimension reduction in model complexity, goal of variable clustering, and the process of variable clustering in predictive modeling. It emphasizes the importance of reducing the number of variables to improve efficiency, interpretability, and reduce overfitting in predictive models. Real examples and strategies for selecting cluster representatives are also provided.

tracyd
Download Presentation

Efficient Variable Reduction for Predictive Modeling with Clustering

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Variable Reduction for Predictive Modeling with Clustering Casualty Actuarial Society Seminar on Ratemaking Bob Sanche

  2. Contents • Data Storage and Amount of Predictive Variables • Predictive Modeling and Model Generalization • Dimension Reduction • Goal of Variable Clustering • What Is Clustering? • Variable Clustering • When Does Variable Clustering Occur During the Predictive Modeling Process? • Example

  3. Data Storage and Predictive Variables • Data storage economics “In 1956, IBM sold its first magnetic disk system, RAMAC (Random Access Method of Accounting and Control). It used 50 24-inch metal disks, with 100 tracks per side. It could store 5 megabytes of data and cost $10,000 per megabyte. (As of 2005, disk storage costs less than $1 per gigabyte).” http://en.wikipedia.org/wiki/History_of_computing_hardware • 1 gigabyte = 130 numeric characteristics for 1 million policies for $1

  4. Data Storage and Predictive Variables • New data sources • Data warehousing • External sources (demographics, meteorological, etc.) • Policyholder, household or company information • Agency • Other • Data storage economics and availability of data • Increase the number of predictive variables • Data mining paradigm • Additional inputs add lift to the model

  5. Predictive Modeling and Model Generalization • A predictive model is created from a number of predictors that are likely to influence future results • Y = α1X1 + … + αnXn + β • n is universe of all available predictors • Goal of predictive modeling • Obtain coefficients for α’s and β • Predictive of future results • Model generalizes well over time • Model complexity → Overfitting

  6. Dimension Reduction • Need to reduce model complexity • Dimension reduction • Clustering (K-Means) • Rows • Variable clustering • Columns • Alternatives to variable clustering • PCA and factor analysis • Difficult to interpret and deploy

  7. Goal of Variable Clustering • Reduce the number of variables • More difficult to identify irrelevant variables than redundant variables • Y = α1X1 + … + αmXm + β • where m<n • Why do we want to reduce the number of variables? • Improve efficiency of predictive modeling process • Time to develop the model • Interpretation of the results • Reduce variance of the model estimates • Demographics example • Average household size, median household size, proportion of families, median vehicles per household • could be replaced by only one variable

  8. What Is Clustering? • “Cluster Analysis is a set of methods for constructing a sensible and informative classification of an initially unclassified set of data, using the variable values observed on each individual” B.S. Everitt , The Cambridge Dictionary of Statistics, 1998 • Divide set of data (variable) into groups of similar characteristics • Unsupervised learning technique • Useful only when there is redundancy in the data

  9. What Is Clustering? • Similarity measured by distance or correlation metrics • Types of clustering • Hierarchical clustering • Agglomerative • Divisive • Partitive (optimization) clustering

  10. Variable Clustering • Variable clustering divides a set of numeric* variables into clusters. A cluster representing a large set of variables can be replaced by a single member (cluster representative). * Hamming distance for categorical variables • Selection of the cluster representative • Intuitively, we want the cluster representative to be as closely correlated to its own cluster (R2own1) and as uncorrelated to the nearest cluster (R2nearest0). Therefore, the optimal representative of a cluster is a variable where 1-R2 ratio tends to zero

  11. When Does Variable Clustering Occur During the Predictive Modeling Process? • SEMMA process for data mining • Sample • Explore • Modify • Model • Assess

  12. Example

  13. Example Name of Variable or Cluster Population Density Legal Expenditures Snow Days Annual Snow Accumulation Rain Days Population Growth Car Density 1.00 0.95 0.90 0.85 0.80 0.75 0.70 Proportion of Variance Explained

  14. Conclusion • Need for dimension reduction for model generalization • Variable clustering reduces the amount of variables available for predictive modeling (GLM, etc.) • The predictive modeling process using variable clustering • Avoid overfitting • Increases interpretability • Reduces time for modeling

More Related