Using media related user profiles to personalize multimedia access over social networks

1. Using media related user profiles to personalize multimedia access over social networks Lemonia Argyriou, (ICCS - National Technical University of Athens)Charalampos Z. Patrikakis, (Technological Education Institute of Piraeus)Stuart CM Porter, (CTVC) Nikolaos Papaoulakis, (ICCS - National Technical University of Athens)Christina Androulaki, (ICCS - National Technical University of Athens) The research leading to these results presented here, has received funding from the European Union's Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 248474 Socially Aware, collaboRative, scAlable Coding mEdia. SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

2. What this paper is about • Personalized media access over the internet • Use of media related, dynamic user profiles, - created and updated through the user behavior, - related to access to multimedia resources, - related also to tagging and commenting • Mechanisms for implementing and deploying the profiles SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

3. The situation • The amount of time spent on the internet is approaching (and in many countries, has surpassed) the number of hours spent watching television, or consuming more traditional media types. • There is too much content. • Personalized media platforms based on sophisticated recommendation engines and social networks are now starting to enter the market. • The “orange film” problem. • Recommendations (taking also into account social networking) • Context awareness SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

4. SARACEN project concept • Design and implement a prototype platform for offering Quality of Experience and personalisation, • using scalable media streaming, • over P2P networks, • and making use of social networking • An end to end platform covering all aspects of the media value chain (from production to consumption of media) • Accommodating both broadcaster originating and user generated content SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

5. SARACEN platform general architecture SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

6. Personalising the streaming experience User-centric design of the media access experience Technological capabilities of the supporting infrastructures and devices Personal preferences and general context (i.e. location) Implicit and explicit methods and harnessing information and functionality of social networks (SARACEN will in fact implement a media centric social network) SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

7. Personalising the streaming experience • Personalized streams, through the integration of metadata related to the streamed media • Produced by users or user groups and destined to users belonging to the same interest group • Filter media related information such as tags, comments and metadata that will be streamed to the user • Allow for the effective formation of a P2P distribution network so as to link users with similar preferences SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

8. The SARACEN user profile • Consists of a range of public and a private data. • The level of privacy of each entry in his profile can be set/edited • Data accessible by some specific “friends” or “groups”, or completely public • Data making up the profile is gathered and, importantly, kept up to date, through both explicit and implicit data collection techniques SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

9. Explicit profiles • Provision of particular information contributed directly by the user. • Targeted information can be used: Movies, Music, Books, TV series/shows, Sports and Other Interests. • Import information from the user’s social networks • Use of auto complete function • Automatic creation of new records SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

10. Implicit profiles • After the creation of a user profile, SARACEN creates a record where all the keywords produced according to his activity are stored • The user can select which of those keywords he wants to add to his profile • The user activities that allow the system to update his personal record are: • 1) Movies and music that he accesses, • 2) Filtering information (addition and deletion of terms) • 3) Comments and tagging of media that he performs SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

11. Dynamic update of profiles • Each time a user selects to watch or interact with a video, the personalization module of the system increases or reduces a value parameter that represents the importance of the keywords that are being stored in his personal user record • Refreshing a “keyword memory” mechanism similar to that of a cache • When the user next selects to update his profile by selecting the “Suggest me” button the system ask him if he wishes to delete the keywords whose values has reached zero and if he wishes to add the new keywords the values of which have been increased and reached a threshold value for addition SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

12. Filtering Mechanism SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

13. Use of keywords for characterising a movie SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

14. Multimedia uploading and sharing • Two types of users: SARACEN users and the SARACEN broadcasters • Common users are offered the opportunity to make their uploaded video public and available to anyone who uses the SARACEN or characterize it as private and only send it to a group of friends that they select. • Keywords presented in a depictive way, that can easily visualize the profile (like in the different templates of wordlets/ slidebar for managing focus) SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

15. Content access and media browsing • Metadata is stored in a separate multimedia record at the SARACEN database and is translated into keywords that characterize the certain multimedia • Keywords come out from the filtering of the users’ performing over the content but also from the rights owner’s page describing the multimedia content. • User is also able to interact with the content, in other words he/she can tag the multimedia while he/she is watching it and make comments, producing metadata that are being stored following the record of the page in SARACEN database SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

16. Making it happen • The ideas presented here are being implemented. • A LoFi prototype is available • A first version of a HiFi prototype has just been released • Demos of P2P streaming are being prepared • User generated content with the ability to tag it is also being tested • 3D is introduced since Oct 2011. SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

17. SARACEN profile creation - SN Back to SARACEN home page Social networks & subscriptions Indicatecredentialstoacesstothedifferent social networksyou’re in User name myId Password ***** Check You didn’t indicate a valid user name or password to access to Primetel system Invite to use Saracen (we can place this somwhere else) Me My favorite Profileinformation SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

18. SARACEN profile creation - info Back to SARACEN home page Social networks & subscriptions Me My favorite Profileinformation Tags from your profile Tags from your profile that matches with your actions Alfred Hitchcock Presents blues Tagsextractedfromyour actions comedy classic The String Hung Up sailing beach Audrey Hebpburn Mediterranean sea war rock Formentera India Waka Waka Maria Callas The Doors bollywood Ibiza pop Save SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

19. SARACEN player - info Back to SARACEN home page Info Title: Haan Maine Bhi Pyaar Kiya Duration: 113 min. Date: 2002 Directed by: Dharmesh Darshan Country: India Distributed byShree Krishna Productions Written by: Dharmesh Darshan Raj Sinha Genres: Romantic Select extra info you want to see Comments Friendstagged Camera info …. Comment: Rian:Best scene from the whole movie Friends tagged: Natalia is on this scene Camera info: diafragma 5.6 Raiting & Social Networks Tags & Comments Recommend something else Back to search Review SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

20. SARACEN player - rateSN Back to SARACEN home page Info Raiting & Social Networks Mine & friends boxes (when you can put content) Set Boxes General Bollywood Football Allow other people know that I’m watchin this movie and that I’ve watched it Raiting Yours : Your friends: --- General : 7.8 6 Social network chat share social networks Select extra info you want to see Which of you friends have watched/is going to watch this movie Comments Friendstagged Camera info …. Comment: Rian:Best scene from the whole movie Friends tagged: Natalia is on this scene Camera info: diafragma 5.6 Tags & Comments Recommend something else Back to search Review SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

21. SARACEN player – tags - tag Back to SARACEN home page Info Raiting & Social Networks Tags & Comments Drag & Drop the icon to add a new Click to view/edit existing Comment Friend beeing tagged Tag love Romeo&julia dance music belly-dancing Bombay comedy Select extra info you want to see Timeline Comments Friendstagged Camera info …. Comment: Rian:Best scene from the whole movie Friends tagged: Natalia is on this scene Camera info: diafragma 5.6 Location Link Folk clothes Review SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

22. SARACEN player - recommend Back to SARACEN home page Info Title: Haan Maine Bhi Pyaar Kiya Duration: 113 min. Date: 2002 Directed by: Dharmesh Darshan Country: India Distributed byShree Krishna Productions Written by: Dharmesh Darshan Raj Sinha Genres: Romantic Haan Maine Bhi Pyaar Kiya Mother India Sangam Select extra info you want to see Comments Friendstagged …. Comment: Rian:Best scene from the whole movie Friends tagged: Natalia is on this scene Raiting & Social Networks Tags & Comments Recommend something else Back to search Review SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

23. Search result Back to SARACEN home page Search Search blues Sort Affinity Best rating Most watched Newest Only my firends content Only my friends affinity, rating, watched Include 3D content in restult Only 3D in result More info Uploader: name4/id4 Uploader: name1/id1 Type: trailer Type: movie Title: Blues Brothers 2000 play Title: Genghis Blues Date: 1998 Date: 1999 Duration: 10 min. do not show again Duration: 88 min. 3D Affinity with you: 82% Affinity with you: 93% Uploader: name2/id2 Type: home recording Uploader: name4/id4 Title: Dubee dubee in concert Type: movie Date: 20-july-2012 Title: Blues Brothers 2000 Duration: 23 min. Date: 1998 Affinity with you: 85% Duration: 123 min. Remarks: one of your facebook firends is related to the video Affinity with you: 82% Uploader: Primetel Type: TV program / documentary Uploader: name5/id5 Title: Red, white & blues Type: movie Date: 18:00 20-july-2012 Title: The Blues Brothers Duration: 60 min. Date: 1980 3D Duration: 132 min. Affinity with you: 84% 3D Remarks: TV program only for primetel suscribers Affinity with you: 70% SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

24. DetailedInformation Blues Brothers 2000 Movie - rateSN Back to SARACEN home page Info Raiting & Social Networks Yours : --- Your friends: 8.2 General : 7.8 Set boxes Raiting play General Bollywood Football Deatiled votes and review from you friends add to queue John Smith: 7.5 Rita Mosoyan: 5.6 Peter Hamilton: 8.1 It's a mix of loving homage to rhythm and blues, scattershot comedy, on-the-spot musical, and the most outrageous car chase film ever put to celluloid. share social networks Patrik Monterde: 7.4 The Blues Brothers ranks among my favorite films of all time -- not the best, just a favorite. With inspiration that never remotely comes close to ceasing, Landis' (The Kentucky Fried Movie, An American Werewolf in London) film is more an experience than a conventional movie. Which of you friends have watched/is going to watch this movie Affinity with you: 82% Deatiled votes and review from people that have same affinity than you Lurta Nikolas: 8.5 Uploader: name4/id4 A musical comedy action fantasy should not work, especially when one considers that it is the first SNL skit-to-screen adventure (which, history has shown us, is a decidedly mixed bag). But this one does. See what others are watching Type: movie Frida Robertsson: 7.4 Dani Goya: 6.5 Back to search Tags SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

25. Boxes Back to SARACEN home page Your boxes Your friends boxes Bollywood General Football Favorite Family Create new box SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

26. Configure Box - search Back to SARACEN home page Bollywood Searchformoviestoadd Search bollywood movies Already added movies Sort Affinity Best rating Most watched Newest Only my firends content Only my friends affinity, rating, watched Include 3D content in restult Only 3D in result Uploader: name1/id1 Type: movie add Title: Haan Maine Bhi Pyaar Kiya Title: Sangam Title: Hum Aapke Hain Kaun Date: 1999 Duration: 88 min. Affinity with you: 93% Uploader: name1/id1 Type: movie add Title: Mughal-e-Azam Date: 1999 Duration: 88 min. Title: Mother India Affinity with you: 86% Privacy Go to boxes SBNMA workshop, ACM multimedia 2011, Scottsdale AZ

27. Publish - description Back to SARACEN home page File /user/files/movies/lordofrings/twoTowers.avi Browse Description Preview image Title The Lord of the Rings: The Two Towers Use image from uploaded video Description 3/10 Frodo and Sam continue on to Mordor in their mission to destroy the One Ring. Whilst their former companions make new allies and launch an assault on Isengard. Upload image for preview /user/files/movies/lordofrings/t Browse Duration The published file has a duration of 223 minutes. 3D It has been detected this content is 3D (please confirm ir can be detected) Info & Tags Privacy & License

28. Taggedcontent Back to SARACEN home page Select from filter Select All Select None Filter Alex Unselect from filter Marianne, did the following review for Vertical Limit meets real life in this deceptively simple documentary. The story of two British mountain climbers who stared death in the face while climbing a treacherous peak in the Peruvian Andes, Touching The Void is a gripping tale of ordinary men coping with extraordinary circumstances. Alex added location to Location: Time 1:34-1:45, Es Pujols Play that scene Alex added a comment to Comment: Spent there my last summer, beautiful place. Play that scene Approve selected SBNMA workshop, ACM multimedia 2011, Scottsdale AZ Reject selected

29. Thank you and … http://saracen-p2p.eu/ twitter.com/saracenP2P www.youtube.com/user/SaracenP2P#p/u www.facebook.com/pages/Saracen/126085287431720 . . .stay tuned SBNMA workshop, ACM multimedia 2011, Scottsdale AZ