210 likes | 300 Views
Making Sense of Twitter Search. Gene Golovchinsky / FXPAL Miles Efron / UIUC. 100,000,000 people can ’ t be wrong. All the cool researchers are doing it. There is Science in everything. We can build something cool. Maybe we can build something useful!. How do we know we’ve succeeded?.
E N D
Making Sense of Twitter Search Gene Golovchinsky / FXPAL Miles Efron / UIUC
Test collections (data sets) are one way to evaluate systems
Test collection to-do list • Research questions • Data collection methodology • Metrics • Data!
What do people look for on Twitter? • Trending topics • Events • Specific tweets • Catching up • Etc.
Evaluation in Information Retrieval • Need a collection • Need tasks • Need metrics • Ground Truth helps
How do we sample the data? • By time, by topic, randomly? …
Credits Title slide http://www.flickr.com/photos/matthamm/3383916444/ Matt Hamm http://www.flickr.com/photos/nixart/370553161/ nixArt “10,000 people can’t be wrong” http://www.flickr.com/photos/luc/1824234195/ luc legay “All the cool researchers are doing it” Gene Golovchinsky “There is science in everything” Gene Golovchinsky “We can build something cool” http://www.flickr.com/photos/donpezzano/3230179951/ urban don “Maybe we can build something useful!” http://www.flickr.com/photos/herzogbr/2274372747/ herzogbr “How do we know we’ve succeeded?” http://www.flickr.com/photos/14696088@N04/1606128341/ rstrawser “Test collections (data sets) are one way to evaluate systems” http://www.flickr.com/photos/_after8_/4052028795/ Régis Gaidot “The fire hose is not a test collection” http://www.flickr.com/photos/magh/2565471904/ Magh “Test collection to-do list” http://www.flickr.com/photos/ario/3362388566/ ario_ “What are we trying to study? The network?” http://www.flickr.com/photos/meg/3533025291 Meg Pickard “Or the person?” Gene Golovchinsky We are interested in interactive search http://www.flickr.com/photos/dullhunk/3389581452/ dullhunk What do people look for on Twitter? http://www.flickr.com/photos/kevglobal/3170506186/ kevglobal First steps (3) Gene Golovchinsky Evaluation in Information Retrieval http://www.flickr.com/photos/ozyman/155517365/ Ozyman How do we sample the data? http://www.flickr.com/photos/red_devil/18540700/ SeenyaRita http://www.flickr.com/photos/texasrobo/4185851361/ texasrobo How do we measure? http://www.flickr.com/photos/jurvetson/17945646/ jurvetson