Download
1 / 21
210 likes | 392 Views
Interactive Interaction Analysis. Aleks Jakulin & Gregor Leban Faculty of Computer and Information Science University of Ljubljana Slovenia. Overview. Interactions : Correlation can be generalized to more than 2 attributes, to capture interactions - higher-order regularities.
E N D
Interactive Interaction Analysis Aleks Jakulin & Gregor Leban Faculty of Computer and Information Science University of Ljubljana Slovenia
Overview • Interactions: • Correlation can be generalized to more than 2 attributes, to capture interactions - higher-order regularities. • Information theory: • A non-parametric approach for measuring ‘association’ and ‘uncertainty’. • Applications: • Visualizations of the domain uncover previously unseen structure. • Software for interactive investigation of data assists the user in identifying interesting patterns. • Importance: • Understanding possible problems and assumptions in machine learning algorithms.
label C importance of attribute A importance of attribute B attribute attribute A B attribute correlation 3-Way Interaction: What is common to A, B and C together; and cannot be inferred from any subset of attributes. 2-Way Interactions Attribute Dependencies
Entropy given C’s empirical probability distribution (p = [0.2, 0.8]). H(C|A) = H(C)-I(A;C) Conditional entropy --- Remaining uncertainty in C after knowing A. H(A) Information which came with knowledge of A H(AB) Joint entropy I(A;C)=H(A)+H(C)-H(AC) Mutual information or information gain --- How much have A and C in common? Shannon’s Entropy A C
Interaction Information I(A;B;C) := I(AB;C) - I(A;C) - I(B;C) = I(A;B|C) - I(A;B) • Interaction information can be: • POSITIVE – synergy between attributes • NEGATIVE – redundancy among attributes • SMALL – nothing special about the 3-way relationship
The only type of odor that does not unambiguously predict the class of the mushroom (edible, inedible). Examples: A Useful Attribute Mutual information or information gain between the attribute and the label.
A Negative Interaction The proportion of information provided by either of the two attributes. This is the “overlap” between both mutual informations.
That’s the gain of s-p-c if we already know the odor. A Negative Interaction The only column where spore-print-color succeeded in providing some information in excess of what we already knew from odor.
A (Seemingly) Useless Attribute Stalk-shapeis totally uninformative, as the class distribution is similar at all attribute values. That’s why we cannot distinguish between classes using this attribute.
Surprise: A Positive Interaction! Information gained by holistic treatment of both attributes! Again, this is “new” mutual information arising from both attributes.
Why a Positive Interaction? Specific attribute value combinations that yield perfect label predictions, but only in combination of both attributes
a weakly negative interaction an unimportant interaction a positive interaction a cluster of negatively interacting attributes An Interaction Dendrogram
synergy redundancy Information Diagram A dissected Venn diagram helps investigate higher-order interactions.
Interactive Interaction Analysis Attributes of interest A sorted list of interactions, ordered by the interaction magnitude. An interaction graph
Summary • There are relationships exclusive to groups of n attributes. • Interaction information is a heuristic for quantification of relationships with entropy. • Visualization methods attempt to: • summarize the interactions in the domain (interaction graph, interaction dendrogram), • assist the user in exploring the domain and constructing classification models (interactive interaction analysis).
Work in Progress • Overfitting: the interaction information computations do not account for the increase in complexity. • Support for numerical and ordered attributes. • Inductive learning algorithms which use these heuristics automatically. • Models that are based on the real relationships in the data, not on our assumptions about them.
