Chapter 6: Segmentation

Chapter 6: Segmentation

Objectives • Define pattern discovery. • Name some of the statistical and analytical techniques that are useful for pattern discovery.

Pattern Discovery The Essence of Data Mining? “…the discovery of interesting, unexpected, or valuable structures in large data sets.” – David Hand ...

Pattern Discovery The Essence of Data Mining? “…the discovery of interesting, unexpected, or valuable structures in large data sets.” – David Hand “If you’ve got terabytes of data, and you’re relying on data mining to find interesting things in there for you, you’ve lost before you’ve even begun.” – Herb Edelstein

Pattern Discovery Are there demographic characteristics to identify people who are more likely to preorder books at a premium price point? What types of people are most likely to be at the food court on a Saturday afternoon? Is that a good time to have a promotional activity for children (and their parents) or for teens? What sorts of complaints are most common for different call centers? If a customer bought product A this week, what is that customer most likely to buy next?

Pattern Discovery Caution • Poor data quality • Opportunity • Intervention • Separability • Obviousness • Nonstationarity ...

Pattern Discovery Caution • Poor data quality • Opportunity • Intervention • Separability • Obviousness • Nonstationarity

Pattern Discovery Applications Data reduction Novelty detection Profiling Market basket analysis Sequence analysis C A B

Pattern Discovery Tools Cluster Segmentation and Profiling Market Basket Analysis, Sequence Analysis In this chapter, you learn two techniques for unsupervised pattern discovery:

Chapter 6: Segmentation

Objectives • Describe several examples of segmentation. • Explain k-means clustering. • Explain the Ward method in SAS Enterprise Miner. • Perform cluster segmentation and generate profiles of the segments using SAS Enterprise Miner.

Unsupervised Classification cluster 1 cluster 2 cluster 3 cluster 1 cluster 2 inputs grouping Unsupervised classification:grouping of cases based onsimilarities in input values

Segmentation for Customer Types • You want to identify segments. While you have thousands of customers, there are really only a handful of major types into which most of your customers can be grouped. • Bargain hunter • Man/woman on a mission • Impulse shopper • Weary parent • DINK (dual income, no kids)

Segmentation for Fraud Detection Most fraudulent customer activity is difficult to identify by a single variable. Are there unusual combinations of behaviors that can help identify criminal activity or fraud? Spending $250.00 on shoes is not unusual.An online purchase by Dan Kelly is not unusual. Purchases in New York by Dan Kelly are not unusual although Dan lives in Raleigh. Dan Kelly buying $250.00 in shoes online while he is in New York; that is unusual. Fraud alert!

Segmentation for Store Location • You want to open new grocery stores in the U.S. based on demographics. Where should you locate the following types of new stores? • low-end budget grocery stores • small boutique grocery stores • large full-service supermarkets

Classifying Fashion Trends • Based on the four styles of pants that your customers can purchase, can you identify stores as serving similar fashion types? • country-club dresser • fashion trendsetter • comfort kick-back dresser

k-Means Clustering Algorithm Training Data 1. Select inputs. 2. Select k cluster centers. 3. Assign cases to closest center. 4. Update cluster centers. 5. Re-assign cases. 6. Repeat steps 4 and 5 until convergence. ...

k-Means Clustering Algorithm Training Data 1. Select inputs. 2. Select k cluster centers. 3. Assign cases to closest center. 4. Update cluster centers. 5. Reassign cases. 6. Repeat steps 4 and 5 until convergence. ...

Segmentation Analysis Training Data When no clusters exist, use the k-means algorithm to partition cases into contiguous groups.

6.01 Poll If you ask SAS Enterprise Miner to recover five clusters but there are not five distinct groups in the data, you do not get a five-cluster solution. You only get as many clusters as there are true groupings to find in the data.  Yes  No

6.01 Poll – Correct Answer If you ask SAS Enterprise Miner to recover five clusters but there are not five distinct groups in the data, you do not get a five-cluster solution. You only get as many clusters as there are true groupings to find in the data.  Yes  No

What Value of k to Use • The number of seeds, k, typically translates to the final number of clusters that are obtained. The choice of k can be made using a variety of methods. • Subject-matter knowledge (There are most likely five groups.) • Convenience (It is convenient to market to three to four groups.) • Constraints (You have six products and need six segments.) • Arbitrarily (Always pick 20.) • Based on the data (Ward’s method)

Ward’s Method in SAS Enterprise Miner Ward’s method is an algorithm for hierarchical cluster analysis. In this method, each observation is considered a cluster, and the clusters are hierarchically joined, based on minimizing the ratio of the variation between clusters to the variation within clusters. Based on a statistical analysis, the number of clusters is selected. This number of clusters is used for k-means cluster analysis.

Ward’s Method in SAS Enterprise Miner SAS Enterprise Miner uses an empirical approach to select the number for k, based on a preliminary analysis using Ward’s clustering in three steps: 1. Preliminary k-means clustering on original data to save many cluster centroids 2. Ward’s hierarchical clustering on saved cluster centroids to determine the ideal value for k 3. k-means clustering on the original data set using k from step 2

Step 1 Many seeds (by default, 50) are chosen from the original training data, and an initial k-means clustering is performed. The means (centroids) of the 50 preliminary clusters are saved to a data set and input to step 2.

Step 2 • Ward’s method performs hierarchical clustering on the preliminary clusters (the centroids saved in step 1). At each step (k clusters, k-1 clusters, k-2 clusters, and so on), the cubic clustering criterion statistic (CCC) is saved to a data set. The final number of clusters is selected based on the CCC with the following conditions: • The final number of clusters must be greater than or equal to the minimum number of clusters specified in the Selection Criteria properties. • The final number of clusters must have a CCC greater than the CCC threshold in the Selection Criteria properties.

Step 3 • The number of clusters determined in step 2 provides the value for k in a k-means clustering of the original training data set. • Ideally, the number of clusters should correspond to a peak in the CCC statistic. • When there is no peak in the CCC, the resulting number of clusters might be suspect. • When the CCC for the selected k is negative, the resulting number of clusters might be suspect.

6.02 Multiple Choice Poll • You should use a clustering solution that corresponds to the _____________ of the CCC. • maximum • minimum

6.02 Multiple Choice Poll – Correct Answer • You should use a clustering solution that corresponds to the _____________ of the CCC. • maximum • minimum

Grocery Store Case Study Analysis goal: • Where should you open new grocery store locations? • Group geographic regions into segments based on income, household size, and population density. Analysis plan: • Select and transform segmentation inputs. • Select the number of segments to create. • Create segments with the Cluster tool. • Interpret the segments.

Segmenting Census Data Grocery Store Case Study Task: Use tools and techniques in SAS Enterprise Miner for cluster and segmentation analysis.

Idea Exchange Do any of the segments seem to map onto the types of stores that the grocery store company is considering (budget, small boutique, large full-service supermarket)? Explore different numbers of clusters for the solution. Do your conclusions change?

Bank Marketing Segmentation Case Study • Who is the best target for a cross-sell/up-sell campaign? • A consumer bank wants to segment its customers based on historic usage patterns to identify those who might benefit from new product offerings. Analysis plan: Perform cluster analysis. Select the number of segments to create. Interpret the segments. Deploy the segmentation rules with scoring code. Analysis goal:

Accessing and Assaying the Data Bank Marketing Segmentation Case Study Task: Use tools and techniques in SAS Enterprise Miner for cluster and segmentation analysis.

Idea Exchange In the examples from this course, you have performed cluster analysis with a small number of variables. However, in real applications, it is common that there are many variables you could use in clustering. Cluster analysis does not perform well with a large number of variables, as it becomes increasingly difficult to detect differences among groups as the number of variables increases. Consider an example in which you might use many variables, such as questionnaire items, demographics, and purchasing behavior. What are some strategies you would take to reduce from a large number of variables to something more manageable?

Chapter 6: Segmentation