200 likes | 421 Views
WEB MONITORING. E6125 Web enHanced Information Management. Presentation on Design of Web Monitoring applications. By Satyajeet Shaligram Columbia University MS CS Fall-08. WEB MONITORING. You are here!!. Introduction The web monitoring task Architectural components How it all works?
E N D
WEB MONITORING E6125 Web enHanced Information Management Presentation on Design of Web Monitoring applications. By Satyajeet Shaligram Columbia University MS CS Fall-08.
WEB MONITORING You are here!! Introduction The web monitoring task Architectural components How it all works? Applying machine learning Difference generation User notification Some examples Future improvements Questions?
What’s the task? “There could be countries out there giving away shit for free!!”
It’s different! RSS feeds..!! User driven. Involves Analytics. Supports a wide range of tasks. Customization! Search Engines..!! Understanding what web monitoring deals with..
How do I design one?? • Client Server model. • Scalable! Scalable! Scalable!! • Efficient & Accurate. • User friendly and non-obtrusive.
WEB MONITORING Introduction The web monitoring task Architectural components How it all works? Applying machine learning Difference generation User notification Some examples Future improvements Questions? You are here!!
Show me the design!! Change Detector - Whizbang! Labs.
Detailed Design! WebCQ – developed at Georgia Tech
WEB MONITORING Introduction The web monitoring task Architectural components How it all works? Applying machine learning Difference generation User notification Some examples Future improvements Questions? You are here!!
Machine Learning! Taxonomy driven crawling Text classification Entity detection Identifying interesting changes Building smarter systems to do the job!!
Difference Generation Developing algorithms with linear running time. Sub dividing the problem into many parts…
Content based chunking… Smart techniques like Rabin’s fingerprinting algorithm make this possible.
WEB MONITORING Introduction The web monitoring task Architectural components How it all works? Applying machine learning Difference generation User notification Some examples Future improvements Questions? You are here!!
Notification Displaying relevant changes!
Future trends… Improving feeds and updates currently provided… E.g. Facebook. Integrating with existing applications such as browsers Developing more intuitive user interfaces to web monitors
Questions!? “Sometimes questions are more important than answers.” - Nancy Willard.
Thank you! …and have a nice day!!