330 likes | 453 Views
Relevance Feedback based on Parameter Estimation of Target Distribution. K. C. Sia and Irwin King Department of Computer Science & Engineering The Chinese University of Hong Kong 15 May IJCNN 2002. Agenda. Introduction to content based image retrieval (CBIR) and relevance feedback (RF)
E N D
Relevance Feedback based on Parameter Estimation of Target Distribution K. C. Sia and Irwin King Department of Computer Science & Engineering The Chinese University of Hong Kong 15 May IJCNN 2002
Agenda • Introduction to content based image retrieval (CBIR) and relevance feedback (RF) • Former approaches • Tackling the problem • Parameter estimation of target distribution • Experiments • Future works and conclusion Relevance Feedback Based on Parameter Estimation of Target Distribution
Content Based Image Retrieval • How to represent an image? • Feature extraction • Colour histogram (RGB) • Co-occurrence matrix texture analysis • Shape representation • Feature vector • Map images to points in hyper-space • Similarity is based on distance measure Relevance Feedback Based on Parameter Estimation of Target Distribution
R G B Feature Extraction Model Relevance Feedback Based on Parameter Estimation of Target Distribution
Relevance Feedback • Relevance feedback • Architecture to capture user’s target of search • Learning process • Two steps • Feedback – how to learn from the user’s relevance feedback • Display – how to select the next set of documents and present to user Relevance Feedback Based on Parameter Estimation of Target Distribution
1st iteration Display UserFeedback Feedbackto system Estimation & Display selection 2nd iteration Display UserFeedback Relevance Feedback Based on Parameter Estimation of Target Distribution
Former Approaches • Multimedia Analysis and Retrieval System (MARS) • Yong Rui et al. Relevance feedback: A powerful tool for interactive content-based image retrieval. - 1998 • Using weight to capture user’s preference • Pic-Hunter • Ingemar J. Cox et al. The Bayesian image retrieval system, pichunter, theory, implementation, and psychophysical experiments. - 2000 • Images are associated with a probability being the user’s target • Bayesian learning Relevance Feedback Based on Parameter Estimation of Target Distribution
Comparison Relevance Feedback Based on Parameter Estimation of Target Distribution
The Model • Feature Extraction • I - raw image data • - set of feature extraction method • f - feature extraction operation • Images data point in hyper-space (Rd) • Problem scope is narrowed down to a particular feature Relevance Feedback Based on Parameter Estimation of Target Distribution
Inconsistence in Feedback • User tells lies • Too many false positive or false negative • Conflict of feedback in each iteration by careless mistake Relevance Feedback Based on Parameter Estimation of Target Distribution
Resolving Conflicts • How to deal with inconsistent user feedback? • Maintain a relevance measure for each data points • Relevance measure > 0 counted as relevant and use in estimation Relevance Feedback Based on Parameter Estimation of Target Distribution
Red Data points selected as relevant Estimating Target Distribution • User’s target is a cluster • Assume it follows a Gaussian distribution • Model a distribution that fits the relevant data points • Based on the parameterof distribution, systemlearns what user wants Relevance Feedback Based on Parameter Estimation of Target Distribution
Expectation Maximization • Fitting a Gaussian distribution function using feedback data points • By expectation maximization • Distribution represent user’s target • Expectation function match the display model Relevance Feedback Based on Parameter Estimation of Target Distribution
Updating Parameters • Estimated mean is the average • Estimated variance by differentiation • Iterative approach Relevance Feedback Based on Parameter Estimation of Target Distribution
Maximum Entropy Display • Why maximum entropy display? • Reason: fully utilize information contained in user feedback to reduce number of feedback iteration • Result: near boundary images will be selected to fine tune parameters Relevance Feedback Based on Parameter Estimation of Target Distribution
Querytargetclustercenter Selectedby knnsearch Selectedby Max.Entropy Maximum Entropy Display • How to simulate maximumentropy display in ourmodel? • Data points 1.18 away from are selected • Why 1.18? • 2P(+1.18)=P() Relevance Feedback Based on Parameter Estimation of Target Distribution
Experiment • Synthetic data generated by Matlab • Mixture of Gaussians • Class label of data points shown for reference to give feedback • Dose it works and works better? Relevance Feedback Based on Parameter Estimation of Target Distribution
Convergence • Is the estimated parameter (mean and variance) converge to the actual parameter of target distribution? • Is the maximum entropy display correctly done? Relevance Feedback Based on Parameter Estimation of Target Distribution
Relevance Feedback Based on Parameter Estimation of Target Distribution
Relevance Feedback Based on Parameter Estimation of Target Distribution
Relevance Feedback Based on Parameter Estimation of Target Distribution
Performance • Compares to Rui’s intra-weight updating model • Nearest neighbour search performed after several feedbacks (6-7 iterations) • Data points outside 2 are discarded in our algorithm • Precision-Recall graph Relevance Feedback Based on Parameter Estimation of Target Distribution
Relevance Feedback Based on Parameter Estimation of Target Distribution
Relevance Feedback Based on Parameter Estimation of Target Distribution
Relevance Feedback Based on Parameter Estimation of Target Distribution
Relevance Feedback Based on Parameter Estimation of Target Distribution
Relevance Feedback Based on Parameter Estimation of Target Distribution
Relevance Feedback Based on Parameter Estimation of Target Distribution
Future Works • Modification to learn from information contained in non-relevant set • To capture correlation in different features • Apply in CBIR system for performance measurement Relevance Feedback Based on Parameter Estimation of Target Distribution
Conclusion • Proposed an approach to interpret the feedback information from user and learn his target of search • Compares our approach with Rui’s intra-weight updating method Relevance Feedback Based on Parameter Estimation of Target Distribution
END Presentation file available athttp://www.cse.cuhk.edu.hk/~kcsia/research/