Natural and NaturalONE provide code coverage functionality. Using code coverage, it is possible to exactly examine the statements of a Natural application that have been executed during a specific run and those that have not been executed. Tests can thus be improved so that they raise the number of touched statements of a Natural application when executed.
In many situations the Jenkins automation server is used for controlling the build and testing sequences. With the Natural Coverage Plugin for Jenkins, you can visualize the outcome of a Natural coverage cycle directly in the Jenkins job result pages. The plugin comes with the following features:
Display a trend graph reflecting the changes in coverage over the last builds.
Display a detailed view of all executed Natural objects and their covered and missed number of statements during the build.
Support the detailed view also throughout the past builds in the job's build history.
This document covers the following topics:
When running a Natural session with the coverage functionality enabled, a result file (so-called Natural coverage file) with extension .ncvf is generated. This result file contains the collected events that describe the statements in a Natural application and the information about the exact statements that have been executed during the Natural session.
The Natural Coverage Plugin enables an existing Jenkins job to read such .ncvf files and to visualize the information stored within. Thus, it is possible to view the coverage of a Natural session from a higher per-application level down to a per-object level. However, for visualizing an even more detailed source code level the usage of NaturalONE is mandatory.
Jenkins Version 1.625.3 or above must be available to run the Natural Coverage Plugin. The .ncvf files to be processed must be in ASCII format.
The Natural Coverage Plugin is provided with the file NaturalCoverage.hpi in the naturalone/jenkins subdirectory of a NaturalONE installation. In order to install the plugin perform the following steps:
Login as a Jenkins user who is allowed to install plugins.
In the Manage Jenkins / Manage Plugins configuration page select the Advanced section.
In the Upload Plugin dialog section browse to the NaturalCoverage.hpi file and press the button.
Follow the instructions that are given by Jenkins.
When the installation has finished the new plugin Natural Code Coverage Publisher should be visible in the Installed view of the Manage Plugins page.
An existing Jenkins job can be configured to post-process .ncvf files by performing the following steps:
Login as a Jenkins user who is allowed to configure Jenkins jobs.
On the job main page select the Configure action.
In the Post-build Actions section at the end of the job configuration page select the Add post-build action box.
From the available actions select Natural code coverage publisher. If the action is not available, check if the installation of the plugin has been performed successfully.
In the new dialog the .ncvf file path can be entered. Base for all file paths is the Jenkins workspace directory.
The new post-build action gets executed when running the next build.
A pipeline job can be enhanced with calls to the Natural Coverage Plugin.
Such a call has a format like
step([$class: 'NaturalCoveragePublisher',
coverageReportFiles: 'MyProject/*.ncvf', showFloatingGraph: true])
where the path to the .ncvf file can be specified in the second parameter.
The pipeline syntax snippet generator can be used to generate such a call by selecting step: General Build Step as the Sample Step and then by selecting Natural code coverage publisher as the Build Step.
When the Natural code coverage publisher has been activated and when the .ncvf files have been detected at the specified location, a trend graph will be shown on the job's main page. Please note that the trend graph requires at least two consecutive builds to be correctly drawn. That same graph on a separate page is also shown when selecting the Coverage Trend Graph link on the left hand side of the job page.
When moving the mouse pointer across the graph, some base information will pop up as tool tips. Clicking inside the graph within the range of some specific build will open a detailed result page for this specific build. The same result page can be reached by selecting the link Coverage Build Result from within a specific build result page.