WP 4 : Plugins and installation modules

1. WP4: Plugins and installation modules Status Report ETICS All-Hands – 23 May 2007 Presenter: Eva Takács (4D SOFT)

2. Location of the plugins • Location of the plugins: \ETICS_HOME\etics\lib\python2.3\site-packages\plugins directory of the etics client installation. ETICS 3rd All-Hands Meeting - University of Wisconsin, Madison - 23-25/05/2007

3. Location of plugins • The \ETICS_HOME\etics\lib\python2.3\site-packages\plugins directory has additional subdirectories: • \plugins_externals – containing the tools and their dependencies • \plugins_buildfiles – containing the ant scripts with the core functionality of the implemented plugins. These ant scripts are wrapped by the execute() function of the PluginManager itself. • \plugins_properties –containing the property files of the ant scripts • \plugins_docs –containing the brief description of the implemented plugins

4. Included tools as plugins • At present the following tools are included as plugins: • \cobertura – containing the libraries of the java coverage tool cobertura • \emma – containing the libraries of the java coverage tool emma • \junit –containing junit library • \pmd – containing the libraries of the pmd tool • \findbugs – containing the libraries of FindBugs • \javaccn – containing the libraries of javancss • \jdepend – containing the libraries of jdepend tool ETICS 3rd All-Hands Meeting - University of Wisconsin, Madison - 23-25/05/2007

5. Naming convention • Naming convention of the plugins: JtoolnamePlugin.pywhere • ‘J’ – indicates that java programs are under analysis • ‘toolname’ – refers to the integrated tool • ‘Plugin’ – should be in the name of the module to enable the PluginManager.py to load the plugin during the execution of the build ETICS 3rd All-Hands Meeting - University of Wisconsin, Madison - 23-25/05/2007

6. JUnit plugin • Definition:this plugin is responsible for running JUnit test cases/suites using the widely-used ant mechanism. • Input directory: • {workspaceDir}/repository/{projectName}/{moduleName}/{version}/noarch/lib • Output: xml, html reports located in • {workspaceDir}/reports/JUnitReports ETICS 3rd All-Hands Meeting - University of Wisconsin, Madison - 23-25/05/2007

7. Emma JUnit plugin • Definition: this plugin is responsible for calculating EMMA coverage metrics for JUnit test cases. • Input directory: • {workspaceDir}/repository/{projectName}/{moduleName}/{version}/noarch/lib • Output junit results: xml, html reports located in • {workspaceDir}/reports/JUnitReports-emma • Output emma coverage results: xml, html reports located in • {workspaceDir}/reports/coverage-emma ETICS 3rd All-Hands Meeting - University of Wisconsin, Madison - 23-25/05/2007

8. Cobertura JUnit plugin • Definition: this plugin is responsible for calculating coverage metrics for JUnit test cases using Cobertura. • Input directory: • {workspaceDir}/repository/{projectName}/{moduleName}/{version}/noarch/lib • Output junit results: xml, html reports located in • {workspaceDir}/reports/JUnitReports-cobertura • Output emma coverage results: xml, html reports located in • {workspaceDir}/reports/coverage-cobertura ETICS 3rd All-Hands Meeting - University of Wisconsin, Madison - 23-25/05/2007

9. JDepend plugin • Definition: this plugin is responsible for scanning Java byte code and generating dependency info. The open source jdepend tool is used in this plugin. • Input directory:{workspaceDir}/repository/{projectName}/{moduleName}/{version}/noarch/lib • Output: • Html(jdepend-report.html) and xml(jdepend-report.xml) reports are created in {workspaceDir}/reports directory.

10. JPmd plugin Definition: this plugin is responsible for scanning Java source code and looking for potential problems like: • Possible bugs - empty try/catch/finally/switch statements • Dead code - unused local variables, parameters and private methods • Suboptimal code - wasteful String/StringBuffer usage • Overcomplicated expressions - unnecessary if statements, for loops that could be while loops The open source pmd tool is used in this plugin.

11. JPmd plugin • Input directory:{workspaceDir}/repository/{projectName}/{moduleName}/{version}/noarch/src • Output: • Html(pmdReport.html) and xml(pmdReport.xml) reports are created in {workspaceDir}/reports directory.

12. Interpreting the results • PMD works by scanning Java code and checks for violations in three major areas: ETICS 3rd All-Hands Meeting - University of Wisconsin, Madison - 23-25/05/2007

13. Interpreting the results • Compliance with coding standards such as: • Naming conventions - class, method, parameter and variable names • Class and method length • Existence and formatting of comments and JavaDocs

14. Interpreting the results • Coding antipatterns such as: • Emptytry/catch/finally/switch blocks • Unused local variables, parameters and private methods • Empty if/while statements • Overcomplicated expressions - unnecessary ifstatements, for loops that could be while loops • Classes with high Cyclomatic Complexity measurements

15. Interpreting the results • More precisely, PMD defines rulesets and inside each ruleset a set of rules in order to verify the source code. • The default of rulesets are as follows: • # Basic (rulesets/basic.xml) – looks for some basic problems, for example, catch blocks shouldn't be empty, override hashCode() anytime you override equals(), etc. • # Naming (rulesets/naming.xml) -- Tests for the standard Java naming conventions: variable names should not be too short; method names should not be too long; class names should begin with an uppercase letter, method and field names should begin with a lowercase letter, etc.

16. Interpreting the results • # Import statements (rulesets/imports.xml) -- Checks for minor issues with import statements, such as importing the same class twice or importing a class from java.lang. • # JUnit tests (rulesets/junit.xml) -- Looks for specific issues with test cases and test methods, such as correct spelling of method names and whether suite() methods are static and public.

17. Interpreting the results • # Strings (rulesets/string.xml) -- Identifies common problems that when working with strings, such as duplicate string literals, calling the String constructor, and calling toString() on String objects. • # Braces (rulesets/braces.xml) -- Checks whether for, if, while, and else statements use braces. • # Code size (rulesets/codesize.xml) -- Tests for overly long methods, classes with too many methods, and similar candidates for refactoring. • # Javabeans (rulesets/javabeans.xml) -- Inspects JavaBeans components for violations of JavaBeans coding conventions, such as unserializable bean classes.

18. Interpreting the result • # Strict exceptions (rulesets/strictexception.xml) -- A few more tests for exceptions: methods should not be declared to throw java.lang.Exception, exceptions should not be used for flow control, Throwable should not be caught. • # Controversial (rulesets/controversial.xml) -- This rule set contains some of the more questionable checks, including assigning null to a variable, multiple return points from a method, and importing from the sun packages. • # Logging (rulesets/logging-java.xml) -- Searches for strange uses of java.util.logging.Logger, including nonfinal, nonstatic loggers, and more than one logger in a class.

19. JFindBugs plugin • Definition: this plugin is responsible for scanning Java byte code and looking for potential bugs. The open source findbugs tool is used in this plugin. • Input directory:{workspaceDir}/repository/{projectName}/{moduleName}/{version}/noarch/lib • Output: Html(findbugsReport.html) and xml(findbugsReport.xml) reports are created in {workspaceDir}/reports directory.

20. Interpreting results • FindBugs results are stored in html/xml reports. • The html report contains a summary of the bug categories indicating their severity. • Then each bug is detailed and the location of it in the source file is also specified. • Lastly, an explanation of the bugs found are put at the end of the report. One example of such a report can be found

21. Interpreting results • FindBugs looks for potential problems by matching the bytecodes against a list of bug patterns. • Bug patternsare code idioms that are often errors. • FindBugs doesn't focus on style or formatting; it specifically tries to find real bugs or potential performance problems. • Bug patterns in principally fall in the following categories: • Single-threaded correctness issue • Thread/synchronization correctness issue • Performance issue • Security and vulnerability to malicious untrusted code

22. OS - Open Stream • When a program opens an input or output stream, it is good practice to ensure that the stream is closed when it becomes unreachable. Although finalizers ensure that Java I/O streams are automatically closed when they are garbage collected, there is no guarantee that this will happen in a • timely manner. There are two reasons why streams should be closed as early as possible. First, operating system file descriptors are a limited resource, and running out of them may cause the program to misbehave. Another reason is that if a buffered output stream is not closed, the data stored • in the stream’s buffer may never be written to the file (because Java finalizers are not guaranteed to be run when the program exits).

23. OS - Open Stream • An example of an open stream found by the bug detector is as follows: • private static File _parsePackagesFile(File packages, File destDir) { • try { • FileReader fr =new FileReader(packages); • BufferedReader br =new BufferedReader(fr); • ... • // fr/br are never closed • }

24. NP – Null pointer dereference • NP – Null pointer dereference • Calling a method or accessing an instance field of a null value results in a NullPointerException at runtime. This detector looks for instructions where a null reference might be dereferenced. • An example of a null pointer dereference found by the NP bug detector is as follows: • if (entry == null) { • IClasspathContainer container=JavaCore.getClasspathContainer ( • entry.getPath(), // entry is null! • root.getJavaProject()); • ..... • }

25. UR - Uninitialized Read In Constructor • When a new object is constructed, each field is set to the default value for its type. In general, it is not useful to read a field of an object before a value is written to it. This detector finds fields that are read in constructors before they're initialized. • public class Thing { • private List actions; • public Thing(String startingActions) { • StringTokenizer tokenizer = new StringTokenizer(startingActions); • while (tokenizer.hasMoreTokens()) { • actions.add(tokenizer.nextToken()); • } • } • } • In this example, line red will cause a null pointer exception because the variable actions has not been initialized.

26. Installation modules • The aim of installation modules is to assist users in their installation procedure by providing some general purpose installation modules, for example, for tomcat, mysql, wscore

27. Installation modules • In order to be more flexible, it was intended to implement a general installation procedure for all software packages while the individual features such as the commands for testing the installation should be included in a separate config file.

28. Steps of installation • To have (download) the requested package • The implemented automatism guess the file type from the mime type and unpacks or installs the packages accordingly • Then the package is installed in the specified location • The last step is to check the installation using the testing command specified in the config file. By default the testing command is provided, but the user can change it.

29. Config file • [TOMCAT] • tomcat.url = http://xenia.sote.hu/ftp/mirrors/www.apache.org/tomcat/tomcat-6/v6.0.10/bin/apache-tomcat-6.0.10-deployer.tar.gz • tomcat.destdir = /home/takacs/Applications/tomcat • tomcat.installcmd = mkdir -p %(tomcat.destdir)s && rm -fr %(tomcat.destdir)s/* && cp -r %(tomcat.unpackpath)s/* %(tomcat.destdir)s/ • tomcat.javahome = $(dirname $(dirname $(which java))) • tomcat.startcmd = (JAVA_HOME=%(tomcat.javahome)s %(tomcat.destdir)s/apache-tomcat-6.0.10/bin/startup.sh) • tomcat.checkcmd = (sleep 21 && wget http://localhost:8080 -O /tmp/TOMCAT.TEST && cat /tmp/TOMCAT.TEST) • tomcat.stopcmd = (JAVA_HOME=%(tomcat.javahome)s %(tomcat.destdir)s/apache-tomcat-6.0.10/bin/shutdown.sh) • tomcat.uninstallcmd = echo -e '\ntomcat uninstall command\n'

30. Etics • 1. etics-show-module-structure externals | grep tomcat • Result: • >Component : tomcat • 2. etics-checkout --project-config externals tomcat • Result: • >Downloading configuration 'tomcat v. 5.0.28' from • >repository 'http://eticssoft.web.cern.ch/eticssoft/repository//externals/tomcat/5.0.28/noarch/tomcat5->5.0.28.tar.gz' • At this point ’tomcat5-5.0.28.tar.gz’ is located in {workspaceDir}/repository/{projectName}/{moduleName}/{version}/noarch.

31. YUM • If the ETICS repository does not contain the required tool, then the next option is to use the YUM mechanism. In order to successfully use YUM the /etc/yum.repos.d/*.repo files should be present and filled with the necessary data. • which yum • yum update (updating just metadata not libraries) • yum list all | grep tomcat • yum install tomcat

32. Apt • Lastly, if no YUM mechanism is provided then apt is used. The following commands are run: • which apt • apt-search tomcat (the biggest version of the component is searched) • apt-get update (updating metadata) • apt-get upgrade (upgrading the libraries) • apt-get install