Confusion Matrix In Machine Learning | Confusion Matrix Explained With Example | Simplilearn

1. Confusion Matrix with Machine Learning

2. What’s in it for you? • Intro to Machine Learning • Need for Confusion Matrices • What is a Confusion Matrix? • Creating a Confusion Matrix • Confusion Matrix metrics • Comparing Confusion Matrices • 3 x 3 Confusion Matrix

3. What is Machine Learning? Machine Learning is a subset of Artificial Intelligence that trains machines with vast volumes of data to think and act like humans, without being explicitly programmed  Vast volumes of data Trains the model Perform human like tasks

4. Confusion Matrix with Machine Learning Click here to watch the video

5. Type of Machine Learning Machine Learning Unsupervised Learning Reinforcement Learning Supervised Learning Model-Free Classification Regression Dimensionality Reduction Model Clustering

6. What is Classification? Classification is the process of classifying data into different categories based on some of their common characteristicsExample: A system to classify spam emails Inbox Classifier Spam

7. What is Classification? Our classification algorithm may wrongly classify some values. Before using our model, we need to find out how accurate it is and what mistakes it is making Inbox Classifier Spam

8.    Need for Confusion Matrices Classification Models have multiple output categories. Most error measures will tell us the total error in our model, but we cannot use it to find out individual instances of errors in our model Total Error Score Input Classifier Error Measure Multiple outputs labels /categories

9.    Need for Confusion Matrices During classification, we also have to overcome the limitations of accuracy. Accuracy can be misleading for classification problems. If there is a significant class imbalance, a model might predict the majority class for all cases and have a high accuracy score  Inbox Our model predicted 8/10 instances and will have an accuracy of 80%. But, is it classifying correctly? Classifier Spam

10.     What is a Confusion Matrix? A confusion matrix presents a table layout of the different outcomes of prediction and results of a classification problem and helps visualize its outcomes The confusion matrix helps us identify the correct predictions of a model for different individual classes as well as the errors               Actual Predicted

11.      Creating Confusion Matrix A confusion matrix consists of rows that contain the values predicted by the classifier and columns which contain the actual values The actual values of our dataset are along the columns Actual The values predicted by our classifier are along the rows  Predicted

12.      Creating Confusion Matrix Consider a Confusion Matrix made from a classifier which classifies people based on whether they speak English or not  People who can speak English will be classified as positives Actual Positive Negative Positive People who cannot speak English will be classified as negatives Predicted Negative

13.      Creating Confusion Matrix Consider a Confusion Matrix made from a classifier which classifies people based on whether they speak English or not  Actual Positive Negative The number of times our actual positive values are equal to predicted positive values gives us True Positives (TP) True Positive Positive Predicted Negative

14.      Creating Confusion Matrix Consider a Confusion Matrix made from a classifier which classifies people based on whether they speak English or not  Actual Positive Negative True Positive Positive The number of times our actual negative values are equal to predicted negative values gives us True Negatives(TN) Predicted Negative True Negative

15.      Creating Confusion Matrix Consider a Confusion Matrix made from a classifier which classifies people based on whether they speak English or not  Actual Positive Negative The number of times our model wrongly predicts negative values as positives gives us False Positives (FP) True Positive False Positive Positive Predicted Negative True Negative

16.      Creating Confusion Matrix Consider a Confusion Matrix made from a classifier which classifies people based on whether they speak English or not  Actual Positive Negative True Positive False Positive Positive The number of times our model wrongly predicts positive values as negatives gives us False Negatives (FP) Predicted Negative False Negative True Negative

17. Confusion Matrix Metrics Confusion matrix metrics are performance measures which help us find the accuracy of our classifier. There are four main metrics : • Accuracy • Precision • Recall • F1-Score

18. Confusion Matrix Metrics Confusion matrix metrics are performance measures which help us find the accuracy of our classifier. There are four main metrics : Accuracy: The accuracy is used to find the portion of correctly classified values. It tells us how often our classifier is right It is the sum of all true values divided by total values

19. Confusion Matrix Metrics Confusion matrix metrics are performance measures which help us find the accuracy of our classifier. There are four main metrics : Precision : Precision is used to calculate the model’s ability to classify positive values correctly. It answers the question, “When the model predicts a positive value, how often is it right?” It is the true positives divided by total number of predicted positive values

20. Confusion Matrix Metrics Confusion matrix metrics are performance measures which help us find the accuracy of our classifier. There are four main metrics : Recall :  It is used to calculate the model's ability to predict positive values. “How often does the model actually predict the correct positive values?"  It is the true positives divided by total number of actual positive values

21. Confusion Matrix Metrics Confusion matrix metrics are performance measures which help us find the accuracy of our classifier. There are four main metrics : F1-Score: It is the harmonic mean of Recall and Precision. It is useful when you need to take both Precision and Recall into account

22. Comparison of Confusion Matrix Consider the following two confusion matrix, derived from two different classifier. To figure out which one performs better, we find the confusion matrix metrics for both of them Non English  Speaker Non English  Speaker English  Speaker English  Speaker English  Speaker English  Speaker 90 85 11 15 Non English  Speaker Non English  Speaker 40 40 19 20

23. Comparison of Confusion Matrix Non English  Speaker Non English  Speaker English  Speaker English  Speaker • Accuracy = (TP + TN) / (TF+TN+FP+FN) •                         = (90+40)/ (90+40+19+11) = 0.8125 • Precision = TP/(TP+FP)  •                      = 90/(90+ 11) = 0.891 • Recall = TP/(TP+FN) •                = 90/ (90 + 19) = 0.8256 • F1-Score = 2* Precision*Recall / ( Precision + Recall) •                    = 2*0.891*0.8256/ (0.8256+ 0.891) = 0.857 • Accuracy = (TP + TN) / (TF+TN+FP+FN)  •                   = (85+40)/ (85+40+15+20) = 0.781 • Precision = TP/(TP+FP)  •                   = 85/(85+ 15) = 0.85 • Recall = TP/(TP+FN) •                  = 85/ (85 + 20) = 0.809 • F1-Score = 2* Precision*Recall / ( Precision + Recall) •                   = 2*0.85*0.809/ (0.85+ 0.809) = 0.828 English  Speaker English  Speaker 90 85 11 15 Non English  Speaker Non English  Speaker 40 40 19 20

24. Comparison of Confusion Matrix From the above calculations we see that our first classifier performed better with a better Accuracy and  F1-Score Non English  Speaker Non English  Speaker English  Speaker English  Speaker English  Speaker English  Speaker 90 85 11 15 Non English  Speaker Non English  Speaker 40 40 19 20 Accuracy =  0.781  F1-Score =   0.828  Accuracy =  0.8125   F1-Score =  0.857

25. Demo on Heart Attack Prediction

26. 3x3 Confusion Matrix To scale up a confusion matrix, you simply extend the number of rows and columns. The true positives will be along the diagonal. All the other values will be FN or FP English  English  Hindi Hindi French English  English  Hindi Hindi French

27. Join us to learn more! simplilearn.com UNITED STATES Simplilearn Solutions Pvt. Limited 201 Spear Street, Suite 1100 San Francisco, CA 94105 Phone: (415) 741-3319 INDIA Simplilearn Solutions Pvt. Limited #53/1C, 24th Main, 2nd Sector HSR Layout, Bangalore 560102 Phone: +91 8069999471 UNITED STATES Simplilearn Solutions Pvt. Limited 801 Corporate Center Drive, Suite 138 Raleigh, NC 27607 Phone: (919) 205-5565