1 / 39

Snap2Eat: Enhancing Dining Experiences with Image Recognition and Recommendations

This thesis proposal focuses on creating Snap2Eat, a mobile application that utilizes image recognition and user-generated data to provide restaurant recommendations. The project includes a human computation game and a smart server for merging duplicate entries. The proposed solution also incorporates multidimensional recommender systems and multicriteria ratings, aiming to offer context-aware restaurant suggestions. The system architecture involves features like login, comments, ratings, search, and photo classification. The scheduling timeline covers data collection, network connection, GPS integration, GUI development, and server connectivity. References include related studies on recommendation systems and image matching technologies.

lkrause
Download Presentation

Snap2Eat: Enhancing Dining Experiences with Image Recognition and Recommendations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Thesis proposal Chi-Chia Huang 28.10.2008

  2. Motivation

  3. http://www.pixnet.net

  4. “美食雜誌上介紹了一家花月嵐拉麵，不知道評價如何？”“美食雜誌上介紹了一家花月嵐拉麵，不知道評價如何？”

  5. “118巷附近有甚麼好吃的？” “明天要去花蓮玩，那邊有甚麼好吃的？”

  6. “今天臨時被拉到光華新天地當駝獸，第一次來不知道有啥可以吃的？”“今天臨時被拉到光華新天地當駝獸，第一次來不知道有啥可以吃的？”

  7. Digital Concierge 2Go

  8. StreetFlow • Restaurant list • Comments, ratings, photos • Location or text input searching • Friend list

  9. 你喜歡的就是我喜歡的嗎？

  10. 噁心… 蛋糕吃到飽耶！

  11. 硬肚人不吃牛~~~~~~ 小沙彌不吃肉~~~~~~

  12. 推薦系統解決了問題了嗎？

  13. 老闆，我要一份紅豆、粉圓、花生、芋頭、芒果、牛奶冰不要紅豆、粉圓、花生、芋頭、牛奶，網路上說這個很好吃老闆，我要一份紅豆、粉圓、花生、芋頭、芒果、牛奶冰不要紅豆、粉圓、花生、芋頭、牛奶，網路上說這個很好吃 少年耶，十月芒果不好吃，沒有芒果冰啦！

  14. “眼前這家看不懂名字的店到底好不好吃，有甚麼好吃的？”“眼前這家看不懂名字的店到底好不好吃，有甚麼好吃的？”

  15. 스타벅스커피 Starbucks Coffee 星巴克

  16. 每個人都在分享…

  17. 歐姆萊斯亭 オムライス亭 おむらいす亭 Shiawase no omuraisu

  18. 三燔本家 三番本家 Minhan Honke

  19. Challenge • Information at anywhere, anytime • Searching information by photo • Ratings from different users

  20. Background & Objective • MoSAIC (Mobile Search and Annotation using Images in Context) • IPAL (Image Perception, Access, and Language)(France, Singapore) • Snap2Tell[4], Snap2Play[5] • National Taiwan University(Taiwan) • PhotoSlap[1], ImageHunter • National Institute of Informatics(Janpan) • MICA (Multimedia, Information, Communications and Application)(France, Vietnam) • LIRIS(France) • LIG(France)

  21. Problem definition • Input: A set of user created restaurant data R • Output: A set of user created restaurant data R’ where duplicate entries of one identical restaurant are merged. |R| ≧ |R’|

  22. Problem definition • Input: • R : user created restaurant data • Date, time, GPS • C : comments • Date, time • S : ratings • Date, time • U : user profile • Output: • Context awareness recommendation on restaurants • Date, time, location, profile

  23. Problem definition • Input: • Restaurant photos with GPS information • Output: • A set of classified photo catalogs

  24. Proposed Solution • Project: Snap2Eat • A mobile application • A web embedded with human computation game • PhotoSlap, ImageHunter • A smart server which can merge duplicate entries of a identical restaurant(image recognition) • Snap2Tell • PhotoSlap, ImageHunter • A smart server which provide recommendation • Multidimensional recommender system[3] • Multicriteria rating[2]

  25. System architecture • Login • New restaurants/foods information/comments • Rating • Search request • Location • Registration • Error report/correction • Photos classification by human Server Mobile client Web client • Restaurants/food/comments • Search result

  26. Schedule • Data collection • 3 areas, each has at least 10 restaurants (11/14) • At lest 10 photos for a restaurant at both launch time and diner time (11/14) • 10 users (01/10) • Mobile phone application • Network connection (11/14) • GPS (11/21) • GUI (11/28)

  27. Schedule • Server • Connect to Snap2Tell (12/05) • Duplicate entry problem (12/12) • Recommender system [3](12/26) • Web • User registration (01/02) • Human computation game (?) • Error report/correctness(01/07)

  28. Reference • Chien-Ju Ho, Tsung-Hsiang Chang, and Jane Yung-jen Hsu. Photo-slap: A multi-player online game for semantic annotation. In Proceedings of Twenty-Second Conference on Articial Intelligence(AAAI-07), pages 1359-1364. AAAI Press, July 2007. • Gediminas Adomavicius and YoungOk Kwon. New recommendation techniques for multicriteria rating systems. IEEE Intelligent Systems, 22(3):48-55, 2007. • Gediminas Adomavicius, Ramesh Sankaranarayanan, Shahana Sen, and Alexander Tuzhilin. Incorporating contextual information in recommender systems using a multidimensional approach. ACM Transactions on Information Systems (TOIS), 23(1):103-145, January 2005. • Jean-Pierre Chevallet, Joo-Hwee Lim, and Mun-Kew Leong. Object iden-tication and retrieval from ecient image matching. snap2tell with the stoic dataset. Information Processing and Management: an International Journal, 43(2):515-530, March 2007. • Tat-Jun Chin, Yilun You, Jean-Pierre Chevallet, Celine Coutrix, Joo-Hwee Lim and Laurence Nigay, Snap2Play: A Mixed-Reality Game based on Scene Identification, ICME 2007

  29. 小公主不要看啟嘉，趕快決定 晚餐要吃甚麼? 你喜歡的，就是我喜歡的？ 照相的人決定

  30. Sports Tracker

  31. Extension • Eating diary • To reduce weight • To control diabetes • For balanced diet

  32. ImageHunter time limit candidate images a target image

More Related