JaCoCo is an open source toolkit for measuring and reporting Java code coverage. JaCoCo is distributed under the terms of the Eclipse Public License. It was developed as a a eplacement for EMMA under the umbrella of the EclEmma eclipse project, and is currently the only byte code coverage tool that works with Java 8.
JaCoCo instruments the bytecode while running the code. To do this it runs as a Java agent,and can be configured to store the collected data in a file, or send it via TCP. Files from multiple runs or code parts can be merged easily. The main motivation to move away from EMMA and go with Jacoco due to fact that Jacoco supports for Java 7 and Java 8.
To enable class instrumentation jacoco agent is attached to carbon server before the server startup. All jar files need to be included for coverage generation is read from instrumentation.txt file which should be available in test module resource directory. And jar file patterns to exclude from instrumentation should be specified in filters.txt in the same resource location. These jar list files will be used in instrumentation and coverage report generation phases.
To introduce Jacoco based code coverage generation to WSO2 products, you can following the following steps below.
- Include jacoco agent as a dependency to root pom. And add this dependency into your sub test modules.
Root pom
<dependency>
<groupId>org.jacoco</groupId>
<artifactId>org.jacoco.agent</artifactId>
<version>${jacoco.agent.version}</version>
</dependency>
<jacoco.agent.version>0.7.4.201502262128</jacoco.agent.version>
|
Sub test module pom
<dependency>
<groupId>org.jacoco</groupId>
<artifactId>org.jacoco.agent</artifactId>
</dependency>
|
- Edit your test module pom and replace emma copy dependency execution configuration with below (if emma copy dependency is already exists)
<execution>
<id>copy-jacoco-dependencies</id> <phase>compile</phase> <goals> <goal>copy-dependencies</goal> </goals> <configuration> <outputDirectory>${project.build.directory}/jacoco</outputDirectory> <includeTypes>jar</includeTypes> <includeArtifactIds>org.jacoco.agent</includeArtifactIds> </configuration> </execution> |
- Remove following system properties from surefire plugin. If those properties are already exits.
<emma.home>${basedir}/target/emma</emma.home>
<emma.output>${basedir}/target/emma</emma.output>
|
- Add following system properties to surefire configuration element. Note that you need to keep instrumentation and filter txt files at src/test/resources location of your test module.
- Remove emma.properties from your test module if it alredy exists at src/test/resources.
- Sample instrumentation file is as follows. This will include all jars matching the pattern for on fly bytecode instrumentation and jars matching will be included only for coverage report generation as well.
Note that you need to use same instrumentation and filter files throughout all test modules.
org.wso2.carbon.aarservices*
org.wso2.carbon.addressing*
org.wso2.carbon.application.deployer*
org.wso2.carbon.application.deployer.webapp*
org.wso2.carbon.authenticator.proxy*
org.wso2.carbon.bam.service.data.publisher*
org.wso2.carbon.caching*
org.wso2.carbon.jaxws.webapp.deployer*
org.wso2.carbon.redirector.servlet*
org.wso2.carbon.repomanager.axis2*
org.wso2.carbon.service.mgt*
org.wso2.carbon.springservices*
org.wso2.carbon.statistics*
org.wso2.carbon.throttle*
org.wso2.carbon.tomcat*
|
- Sample filters.txt file is given below. All jar names matching the file patterns defined in filters.txt file will be ignored from coverage generation. Note that you cannot filter out specific class files from coverage generation. You can only exclude jar files matching the given pattern.
-*.stub*
-*.stub_
-org.eclipse.*
-*.equinox.*
-samples.*
-*.log4j*
-*.axis2*
-*.ui*
-*.tenant*
-*.stratos*
-*.eventing*
-*transports*
-org.wso2.carbon.mediation.statistics*
-*startup*
|
- Coverage report will be generated at the end of the test module execution. Coverage data dump into $basedir/target/jacoco/coverage directory at JVM exit. And at the time of report generation, jacoco coverage data files are merged back to a single file (jacoco-data-merge.exec) and report will be generated using this merged data file.
- You need to enable coverage in automation.xml if it is already disabled.
- Carbon server extension class has been moved to carbon-platform-integration repository. Hence you need to use org.wso2.carbon.automation.extensions.servers.carbonserver.CarbonServerExtension class instead of old class. To use the new class edit automation.xml and set the class name as below.
<platformExecutionManager>
<extentionClasses>
<class>
<name>org.wso2.carbon.automation.extensions.servers.carbonserver.CarbonServerExtension</name>
<!--<parameter name="portOffSet" value="2" />
<parameter name="cmdArg" value="debug 5005" />-->
</class>
<class>
<name>org.wso2.carbon.integration.common.extensions.usermgt.UserPopulateExtension</name>
</class>
</extentionClasses>
</platformExecutionManager>
|
- You will find following log entries in console with required details to view the report once coverage generation completed. Available report formats are XML, HTML and CSV.
INFO [org.wso2.carbon.automation.extensions.servers.carbonserver.CarbonServerManager] - Jacoco coverage dump file path : /Users/krishantha/git/wso2/product-as/modules/integration/tests-integration/tests/target/jacoco.exec
INFO [org.wso2.carbon.automation.extensions.servers.carbonserver.CarbonServerManager] - Jacoco class file path : /Users/xxx/product-as/modules/integration/tests-integration/tests/target/wso2as-6.0.0-NAPSHOT/repository/components/plugins
INFO [org.wso2.carbon.automation.extensions.servers.carbonserver.CarbonServerManager] - Jacoco coverage report path : /Users/krishantha/git/wso2/product-as/modules/integration/tests-integration/tests/target/jacoco/coverage
|
Merge Coverage report of multi module test projects (Optional)
To merge coverage reports of multiple modules, you need to introduce new module at the end of all modules execution. This mould should be executed at the end of test module execution. As a convention let's name this modules as “reports”. In the reports module, create a new pom file and add below content. Make sure to modify the parent and pom configurations according to your product group and artifact naming conventions.
Also you need to add instrumentation and filters text files and set the patch correctly in arguments section of exec maven plugin. Note that you need to use same instrumentation and filter files throughout all test modules and report module.
project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<groupId>org.wso2.appserver</groupId>
<artifactId>test-integration-appSever</artifactId>
<version>5.3.0-SNAPSHOT</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
<name>WSO2 AppServer - Integration Test Report Module</name>
<artifactId>org.wso2.appserver.integration.tests.reports</artifactId>
<packaging>jar</packaging>
<profiles>
<profile>
<id>with-tests</id>
<activation>
<activeByDefault>true</activeByDefault>
<property><name>!maven.test.skip</name></property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.2.1</version>
<executions>
<execution>
<id>my-exec</id>
<phase>install</phase>
<goals>
<goal>exec</goal>
</goals>
</execution>
</executions>
<configuration>
<executable>java</executable>
<arguments>
<argument>-Dbasedir=${basedir}</argument>
<argument>
-Dcarbon.zip=${basedir}/../../../distribution/target/wso2as-${project.version}.zip
</argument>
<argument>-Dinstr.file=${basedir}/instrumentation.txt</argument>
<argument>-Dfilters.file=${basedir}/filters.txt</argument>
<argument>-classpath</argument>
<classpath/>
<argument>
org.wso2.carbon.automation.engine.frameworkutils.TestCoverageGenerator
</argument>
</arguments>
<workingDirectory>${basedir}/target</workingDirectory>
</configuration>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>without-tests</id>
<activation>
<activeByDefault>true</activeByDefault>
<property><name>maven.test.skip</name></property>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.2.1</version>
<executions>
<execution>
<id>my-exec</id>
<phase>install</phase>
<goals>
<goal>exec</goal>
</goals>
</execution>
</executions>
<configuration>
<skip>true</skip>
<executable>java</executable>
<arguments>
<argument>-Dbasedir=${basedir}</argument>
<argument>
-Dcarbon.zip=${basedir}/../../../distribution/target/wso2as-${project.version}.zip
</argument>
<argument>-Dinstr.file=${basedir}/instrumentation.txt</argument>
<argument>-Dfilters.file=${basedir}/filters.txt</argument>
<argument>-classpath</argument>
<classpath/>
<argument>
org.wso2.carbon.automation.engine.frameworkutils.TestCoverageGenerator
</argument>
</arguments>
<workingDirectory>${basedir}/target</workingDirectory>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
<dependencies>
<dependency>
<groupId>org.wso2.carbon.automation</groupId>
<artifactId>org.wso2.carbon.automation.engine</artifactId>
<scope>compile</scope>
</dependency>
</dependencies>
</project>
|
The generate coverage report can be found at - $basedir/target/jacoco/coverage/index.html and reports available in XML and CSV formats as well.
Reports module file structure
Reports module should look like below as applying all changes.
Class include and exclude patterns
'*' matches zero or more characters
'?' matches one character.
Examples:
"**\*.class" matches all .class files/dirs in a directory tree.
"test\a??.java" matches all files/dirs which start with an 'a', then two more characters and then ".java", in a directory called test.
"**" matches everything in a directory tree.
"**\test\**\XYZ*" matches all files/dirs which start with "XYZ" and where there is a parent directory called test (e.g. "abc\test\def\ghi\XYZ123").
Case sensitivity may be turned off if necessary. By default, it is turned on.
e.g **/*AddressingInFaultHandler*.*
Troubleshooting
If you got following exception while generating reports. The cause of the issue would be having different asm dependency in your classpath. This exception means that JaCoCo sees the wrong version of the ASM
library. To avoid the exception, need to exclude ams dependency appropriately. To do that, get the maven dependency:tree and exclude the dependency.
Caused by: java.lang.IncompatibleClassChangeError: class org.jacoco.core.internal.flow.ClassProbesVisitor has interface org.objectweb.asm.ClassVisitor as super class
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at org.wso2.carbon.automation.engine.frameworkutils.ReportGenerator.analyzeStructure(ReportGenerator.java:144)
at org.wso2.carbon.automation.engine.frameworkutils.ReportGenerator.create(ReportGenerator.java:77)
at
<exclusion>
<groupId>asm</groupId>
<artifactId>asm</artifactId>
</exclusion>
|
No comments:
Post a Comment