320 likes | 459 Views
Optimization Problem Based on L 2,1 -norms. Xiaohong Chen 19-10-2012. Outline. Efficient and robust feature selection via joint l 2,1 -norm minimzation Robust and discriminative distance for multi-instance learning Its application…. Outline.
E N D
Optimization Problem Based on L2,1-norms Xiaohong Chen 19-10-2012
Outline • Efficient and robust feature selection via joint l2,1-norm minimzation • Robust and discriminative distance for multi-instance learning • Its application…
Outline • Efficient and robust feature selection via joint l2,1-norm minimization • Robust and discriminative distance for multi-instance learning • Its application…
Efficient and robust feature selection via joint l2,1-norm minimzation
Robust Feature Selection Based on l21-norm Given training data {x1, x2,…, xn} and the associated class labels {y1,y2,…, yn} Least square regression solves the following optimizaiton problem to obtain the projection matrix W Add a regularization R(W) to the robust version of LS,
Robust Feature Selection Based on l21-norm Possible regularizations
Robust Feature Selection Based on l21-norm Denote (14)
Robust Feature Selection Based on l21-norm Then we have (19)
The iterative algorithm to solve problem (14) Theorem1:The algorithm will monotonically decrease the objective of the problem in Eq.(14) in each iteration, and converge to the global optimum of the problem.
u u Proof of theorem1
(1)+(2) (1) (2)
Outline • Efficient and robust feature selection via joint l2,1-norm minimization • Robust and discriminative distance for multi-instance learning • Its application…
Robust and discriminative distance for multi-instance learning
Multi-instance learning 多示例学习中,训练集由若干个具有概念标记的包(bag)组成, 每个包包含若干个没有概念标记的示例。若一个包中至少有 一个正例,则该包被标记为正(positive),若一个包中所以示 例都是反例,则该包被标记为反(negative),通过对训练包的学 习,希望学习系统尽可能正确地对训练集之外的包的概念标 记进行预测。
Notations Given N training bags and K conceptual classes. Each bag contains a number of instances Given the class memberships of the input data, denoted as
Notations First, we represent every class as a super-bag that comprises the instances of all its training , where
Objective to learn class specific distance metrics For a given class, Ck,, we solve the following optimization problem:
Algorithm and its analysis On the other hand,
Algorithm and its analysis Therefore, the objective value of the problem of (6) is decreased in each iteration till convergences.
Outline • Efficient and robust feature selection via joint l2,1-norm minimzation • Robust and discriminative distance for multi-instance learning • Its application…
Its application For example:
Reference • [1]F.Nie, D.Xu, X.Cai, and C.Ding. Efficient and robust feature selection via • joint l2,1-norm minimzation. NIPS 2010. • [2] H.Wang, F.Nie and H.Huang. Robust and discriminative distance for multi- • instance learning, CVPR 2012: 2919-2924