A Proposed Methodology for E-Business Intelligence Measurement Using Data Mining Techniques

1. A Proposed Methodology for E-Business Intelligence Measurement Using Data Mining Techniques Stavros Valsamidis, Ioannis Kazanidis, Sotirios Kontogiannis Alexandros Karakos {svalsam@teikav.edu.gr, kazanidis@teikav.edu.gr, skontog@gmail.com, karakos@ee.duth.gr } PCI 2014

2. Outline Introduction Method Results Discussion Limitations Conclusions PCI 2014

3. Introduction (1/7) E-business Business Intelligence Knowledge Data Discovery Data Mining PCI 2014

4. Introduction (2/7) E-business E-business refers to any business that uses the Internet and related technologies. E-business is the conducting of business on the Internet, not only buying and selling but also servicing customers and collaborating with business partners Intelligence Luhn defined intelligence as: "the ability to apprehend the interrelationships of presented facts in such a way as to guide action towards a desired goal“. PCI 2014

5. Introduction (3/7) Business Intelligence Business Intelligence (BI) is the emerging discipline that aims at combining corporate data with textual user-generated content (UGC) to let decision-makers analyze their business based on the trends perceived from the environment PCI 2014

6. Introduction (4/7) Knowledge Data Discovery The term Knowledge Data Discovery (KDD) was coined in 1989 to refer to the broad process of finding knowledge in data, and to emphasize the “high-level” application of particular data mining (DM) methods PCI 2014

7. Introduction (5/7) Data Mining Data mining main goal is the search for relationships and distinct patterns that exist in datasets but they are “hidden" among the vast amount of data. PCI 2014

8. Introduction (6/7) Indexes and metrics proposed by authors for the usage of web applications. There are not metrics specifically for measuring e-business usage in terms of BI This study contributes to the area of web usage analysis for e-business intelligence by ‘marrying’ e-business with data mining Four metrics, applied innovatively for the first time in the field of e-business PCI 2014

9. Introduction (7/7) This paper proposes an iterative method for designing and maintaining BI applications that reorganizes the activities and tasks normally carried out by practitioners is completed by a case study to the consumer goods area, aimed at proving that the adoption of a structured methodology positively impacts on the project success PCI 2014

10. Method PCI 2014

11. Method – Steps (1/5) logging data: logging of specific data from e-business systems Specifically eleven (11) fields (request_time_event, remote_host, request_uri, remote_logname, remote_user, request_method, request_time, request_protocol, status, bytes_sent, referer, agent) and user requests from different products Pre-processing: The data contain noise such as URLs, emoticons, symbols, like asterisks, hashes, etc. PCI 2014

12. Method – Steps (2/5) Indexes, metrics and rates: PCI 2014

13. Method – Steps (3/5) Indexes, metrics and rates: Enrichment = 1- (Unique Pages/Total Pages) Disappointment= Sessions/Total Pages Interest=1-Disappointment Homogeneity =Unique pages/Total Sessions Mean rate = (Enrichment + Homogeneity + Interest) /3 Score = Mean rate * UPPS PCI 2014

14. Method – Steps (4/5) Data mining techniques: data mining techniques are applied so that relevant data can be analyzed. Classification, clustering and association rule mining are used, based on the metrics of the third step. During this step the classification the algorithm 1R may be applied Product clustering is included in the clustering step, this is established by the Purchases attribute Clustering of user visits is performed with the use of k-means algorithm PCI 2014

15. Method – Steps (5/5) Data mining techniques: Association rule mining enables relationships to be found amongst attributes in databases, revealing if-then statements regarding attribute-values An association rule X  Y shows a close correlation among items in a database. This occurs when transactions in the database in which X occurs, there is also a high probability of having Y. In an association rule X and Y are respectively named the antecedent and consequent of the rule. PCI 2014

16. Results (1/6) Study population and context The data of 40 products are ranked in descending order according to the column Score PCI 2014

17. Results (2/6) Data pre-processing and calculation of the metrics and rates The data are in ASCII form and are obtained from the Apache server log file. Application of data mining techniques the column Score The attributes of the table were inserted in .cvs format into Weka The attributes Product ID and Disappointment were removed Product_ID is different for each instance and Disappointment is the complement to the Interest attribute. All the remaining attributes were disretized. PCI 2014

18. Results (3/6) Classification In the classification step, the algorithm 1R is applied. The attribute Purchases is used as class. The best attribute which describes the classification is Score PCI 2014

19. Results (4/6) Clustering The clustering step contains products clustering, based on the Purchases attribute with the use of the SimpleKmeans algorithm PCI 2014

20. Results (5/6) Association rule mining The Apriori algorithm was used to find association rules over the discretized data Because of the obvious dependencies of the attributes Sessions, Pages and Unique Pages with the attributes Enrichment, Interest and Homogeneity, the latter group of attributes was removed from the data table Weka shows a list of 6 rules with the support of the antecedent and the consequent (total number of items) at 0.1 minimum, and the confidence of the rule at 0.9 PCI 2014

21. Results (6/6) Association rule mining There is an uninteresting rule, like rule 1. There are some similar rules, rules with the same element in antecedent and consequent but interchanged, such as the couples of rules 3, 4 and 5, 6 It is proven that purchases of the products are dependent on the scores PCI 2014

22. Discussion (1/2) The indication that many pages within useful paths contribute to increased usage is fairly obvious. The more and better content on a site, the more a user might visit it. So the administrators should add some useful and helpful pages to a site. If there is an essentially blank site but it is required for the customers to visit it every day and contribute a comment, then the usage will be necessarily high. On the other hand, if there is a very elaborate web site with rich content but is not required reading, limited usage of the site would be expected PCI 2014

23. Discussion (2/2) Rule 2 offers to the administrators a lot of action ability, since they can pay more attention to the products with low values of Score and Sessions. An increase in sessions results in more users (customers) using the e-business system Of course, it cannot be denied that a certain number of customers only attempt to read the product information just before doing their purchases PCI 2014

24. Limitations The fact that only 40 products in one e-business system were investigated is a limitation to the study. Especially for the data mining techniques which demand large datasets. However, this was ineluctable since the e-business system of the case study had this number of active online products. PCI 2014

25. Conclusions (1/3) The proposed iterative method uses existing tools and techniques in a novel way to perform e-business systems usage analysis. The metrics enrichment, homogeneity, disappointment and interest are used. It incorporates clustering, classification and association rule mining. PCI 2014

26. Conclusions (2/3) Advantages It is independent of a specific e-business system, since it is based on the Apache log files and not the e-business system itself. Thus, it can be easily implemented for every e-business system. It uses indexes and metrics in order to facilitate the evaluation of each product. It offers useful information for a company to have to determine which parts of its web site to improve. PCI 2014

27. Conclusions (3/3) This approach may be applied after a long time period of data tracking The proposed approach may also be applied to other web applications such as e-government, e-learning, e-banking, blogs, social networks etc. PCI 2014

28. Thank You! Stavros Valsamidis, Ioannis Kazanidis, Sotirios Kontogiannis Alexandros Karakos svalsam@teikav.edu.gr, kazanidis@teikav.edu.gr, skontog@gmail.com, karakos@ee.duth.gr TEI of Kavala Kavala, Greece PCI 2014