Agent Technology for e-Commerce

1. Agent Technology for e-Commerce Chapter 6: Recommender Systems Maria Fasli http://cswww.essex.ac.uk/staff/mfasli/ATe-Commerce.htm

2. Recommender systems: The problem • Too much information: information overload – consumers have too many options • A recommender system is a system which provides recommendations to a user • Applications: Books, music CDs, movies. Even documents, services and other products such as software games

3. Requester Recommender Request for service Sorted description of P1,..Pn Advertisement of capabilities Service delegation  Results of service request Provider 1 Provider n Recommender

4. Information needed Information used for recommendations can come from different sources: • browsing and searching data • purchase data • feedback explicitly provided by the users • textual comments • expert recommendations • demographic data

5. Providing recommendations Recommendations can take the following forms: • Attribute-based recommendations: based on syntactic attributes of products (e.g. science fiction books) • Item-to-item correlation (as in shopping basket recommendations) • User-to-user correlation (finding users with similar tastes) • Non-personalized recommendations (as in traditional stores, i.e. dish of the day, generic book recommendations etc.)

6. Recommendation technologies Information retrieval (IR) systems: • allow users to express queries to retrieve information relevant to a topic of interest or fulfil an information need • they are not useful in the actual recommendation process • they cannot capture any information about the users’ preferences • they cannot retrieve documents based on opinions or quality as they are text-based To address these issues two techniques have been developed: • Content-based filtering (Information filtering) • Collaborative-based filtering

7. Content-based filtering The system processes information from various sources and tries to extract useful elements about its content • keyword-based search (keywords sometimes in boolean form) • semantic-information extraction by using associative networks of keywords, or directed graphs of words

8. Each user is assumed to act independently and the system requires a profile of the user’s needs or preferences • The user has to provide information on her personal interests on starting to use the system for the profile to be built • The profile includes information about the items of interest, i.e. movies, books, CDs etc. • Content-based filtering techniques try to identify similar items which are returned as recommendations • They do not depend on having other users in the system

9. Issues • Pure content-based filtering systems are not capable of exploring new items and topics • Over-specialization: one is restricted in viewing similar items • Difficult to apply in situations where the desirability of an item is determined in part by aesthetic qualities that are difficult to quantity – it is difficult to apply content-based analysis to such items

10. The user profiles • For the system to produce accurate recommendations, the user has to provide constant feedback on the returned suggestions – users do not like providing feedback • Consist entirely of ratings of items and topics of interest: the fewer the ratings, the more limited the set of possible recommendations • As the user’s interests change, these changes need to be tracked

11. Collaborative filtering • Collaborative-based filtering systems can produce recommendations by computing the similarity between a user’s preferences and the preferences of other people • Such systems do not attempt to analyse or understand the content of the items being recommended • They are able to suggest new items to user who have similar preferences with others

12. Basic mechanism • A large group of people’s preferences are registered • A subgroup of people is located whose preferences are similar of the user who seeks the recommendation • An average of the preferences for that group is calculated • The resulting preference function is used to recommend options to the user who seeks the recommendation • The concept of similarity needs to be defined in some way

13. Example user-item matrix What would be the recommendation for user D?

14. Neighbourhood-based algorithms Three steps (i) The degree of similarity of the active user and the others in the database is calculated (positive or negative) (ii) A set of users is chosen as the basis for making the prediction. This is determined based on the degree of similarity and differs from system to system (iii) The set of users chosen in the previous step is used to make the recommendation. A user with high degree of similarity may be assigned higher weight

15. Pearson’s correlation coefficients It reflects the degree of linear relationship between two variables and ranges –1 to +1 The degree of correlation between an active user a and another user u is:

16. Next, the neighbourhood of users based on which the recommendation will be provided is selected • The weighted average of the ratings of the neighbourhood of users for the item of interest is then calculated as follows:

17. Example Using Pearson’s correlation coefficients: wD,A= 0.9 wD,B= - 0.7 wD,C= 0 pD,Item4= 4.5

18. Issues • A critical mass of users is needed in order to create a database of preferences: first-rater or cold start problem • New items cannot be recommended until someone has rated them • The scarcity of ratings (the user profiles are sparse vectors of ratings) also presents a problem • Recommendations will come from users with which the active user shares ratings (or votes) – this presents a problem to methods such as Pearson’s correlation coefficients; potential solutions: default voting

19. Scalability: in systems with a large number of items and users, computation grows linearly; appropriate algorithms that scale up are needed • Reliability, especially in reputation systems: content providers inflate their ratings • Lack of transparency: the user is given no indication whether to trust a recommendation – incorporating explanation systems would help address this concern • Privacy – once a system has built your profile, who else can have access to it?

20. Combing collaborative and content-based filtering • The underlying idea is that the content is also taken into account when attempting to identify similar users for collaborative recommendations • A number of systems have been developed: Fab, Tango, the Recommender system, GroupLens’ approach

21. Recommender systems in e-commerce • Turning browsers into customers: they can stimulate the users’ needs (need identification stage) • Cross-selling: suggest additional products which may match the user’s interests or current shopping basket • Personalization: personalized services, or the site can be personalized to the user’s liking – unique shopping experience • Keeping customers informed • Retaining customer loyalty

22. Personalization • Vendors can identify exactly who is visiting their store through registration, cookies, spyware • Vendors can personalize their websites for their customers • They can keep track of preferences, actions, they can build profiles of their users. These can be used for marketing • Vendors can measure the users’ desires – dynamic pricing • When the consumer is unaware, then problems arise, possible breaches of the user’s privacy. Who else gains access to these profiles? • Negative impact on consumer confidence