70 likes | 170 Views
HYPERTEXT DATA BASES. Session 7 16:10 - 16:35 Dr Soumen Chakabarti, IIT Bombay. Database evolution. Structured Operational Data. Additional data capture: free text from customer forms, phone conversation logs, etc. OLTP Operations. Semi- structured Database. SQL3: Text and
E N D
HYPERTEXT DATA BASES Session 7 16:10 - 16:35 Dr Soumen Chakabarti, IIT Bombay
Database evolution StructuredOperational Data Additional data capture: free text from customer forms, phone conversation logs, etc. OLTP Operations Semi- structuredDatabase SQL3: Text and multimedia support Indexing, search and retrieval Data analysis and mining Collaboration and resource discovery
Basic text indexing and retrieval • Inverted index mapping terms to (document, position) • Boolean search • socks and not “network* protocol*” • Phrase and proximity search • socks near(5) network* • Relevance ranking • Vector space model loan inflation bank
Text mining • Problems • Very high dimensionality • Vocabulary mismatch • Available technologies • Relevance feedback • Semantic indexing • Semi-automatic thesaurus construction • Topic directory construction (clustering) • Topic directory maintenance (classification)
From text to hypertext • Email, newsgroup, intranet web pages • Hyperlinks are much more than a browsing mechanism • Citations describe collective opinion • Hyperlinks implicitly form online communities • Hyperlinks enable collaboration and autonomous resource discovery
Example: clustering and classification Organize resources used by employees into topic directory Classify news, email, office circulars, etc. into topic areas Discover resources on specific topics of interest on the Web Enable multiple users to collaborate without explicit effort
Research problems • Represent and query semi-structured data • Mine distributed, hyperlinked data with both structured and unstructured fields • Locate Internet resources, understand their data schema, and enable data integration • Answer “why,” “how-to,” “where can I” questions in restricted domains • Unify text corpora with semi-structured thesaurus