1 / 21

Feature Selection

Feature Selection. Dr. Gheith Abandah. Definition. Feature selection is typically a search problem for finding an optimal or suboptimal subset of m features out of original M features. Benefits: For excluding irrelevant and redundant features,

gerda
Download Presentation

Feature Selection

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Feature Selection Dr. Gheith Abandah

  2. Definition • Feature selection is typically a search problem for finding an optimal or suboptimal subset of m features out of original M features. • Benefits: • For excluding irrelevant and redundant features, • it allows reducing system complexity and processing time, • and often improves the recognition accuracy. • For large number of features, exhaustive search for best subset out of 2M possible subsets is infeasible.

  3. Feature Set

  4. Selection Algorithms • Generally be classified according to the criterion function used in searching for good features. • Wrapper algorithm: the performance of the classifier is used to evaluate the feature subsets. • Filter algorithm: some feature evaluation function is used rather than optimizing the classifier’s performance. • Wrapper methods are usually slower than filter methods but offer better performance.

  5. Best Individual Features • Select best individual features. A feature evaluation function is used to rank individual features, then the highest ranked m features are selected. • Although these methods can exclude irrelevant features, they often include redundant features. “The m best features are not the best m features”

  6. Best Individual Features • Examples: • Scatter criterion • Symmetric uncertainty

  7. Scatter Criterion

  8. Scatter Criterion

  9. Scatter Criterion

  10. Symmetric Uncertainty

  11. Symmetric Uncertainty

  12. Search Algorithms • Sequential < O(M2) • Forward selection, e.g., • Fast correlation-based filter (FCBF) • Minimal-redundancy-maximal-relevance (mRMR) • Backward selection • Bidirectional • Random • Genetic algorithm, e.g., • Multi-objective genetic algorithms (MOGA)

  13. Fast Correlation-Based Filter

  14. Fast Correlation-Based Filter

  15. Minimal-Redundancy-Maximal-Relevance

  16. Minimal-Redundancy-Maximal-Relevance

  17. Minimal-Redundancy-Maximal-Relevance

  18. Non-dominated Sorting Genetic Algorithm • Use NSGA to search for optimal set of solutions with two objectives: • Minimize the number of features used in classification. • Minimize the classification error.

  19. Pareto-Optimal Front

  20. Classification Accuracy

  21. Classification Accuracy

More Related