Kurtosis Using a Third Party Library

Kurtosis is a statistical measure of 'peakedness' in the values for a dataset compared to a normal distribution. This indicates how closely the distribution matches the rounded bell shape of a normal distribution.

In this example, we will use an implementation of kurtosis provided in the Apache Commons Math library, version 2.2. The method to calculate kurtosis, in the DescriptiveStatistics class in the Apache Library, expects the values to use as the probability distribution to be primitive values in an array.

To support this, the aggregate method builds an array from the column values for records in a group, partition or window. The getAggregate method then uses this array to perform the calculation. As always, the state object is used to hold state for both methods.

package com.raql.samples;

import java.io.Serializable;

import org.apache.commons.math.stat.descriptive.DescriptiveStatistics;

import com.jackbe.jbp.raql.udx.loader.RaqlFunc;

import de.rtm.push.adapters.UserDefinedAggregationFunctionAdapter;

@RaqlFunc(name="myKurtosisFunction")

/**

* This adapter computes the kurtosis for a group, partition or window.

*/

public class KurtosisAdapter implements

UserDefinedAggregationFunctionAdapter<double[]> {

/**

* Returns the types of the input parameters.

*/

@Override

public Type[] getParameterTypes() {

return new Type[] {Type.DOUBLE};

}

/**

* Returns the type of the result.

*/

@Override

public Type getAggregateType() {

return Type.DOUBLE;

}

/**

* Creates the initial state of the kurtosis. Note that

* the state solely stores the input values. Therefore

* the initial state is an empty array.

* @return the initial state of the kurtosis

*/

@Override

public double[] createInitialState() {

return new double[] {};

}

/**

* Aggregates the current internal state with the new input values

* and derives a new internal state. As the kurtosis aggregate is

* computed by an external library, the state solely stores

* the incoming values.

* @param state the current internal state

* @param values the new input values

* @return the new internal state

*/

@Override

public double[] aggregate(double[] state, Serializable[] values) {

if (values[0] != null) {

double[] newState = new double[state.length+1];

System.arraycopy(state, 0, newState, 0, state.length);

newState[newState.length-1] = (Double) values[0];

return newState;

}

else

return state;

}

/**

* Derives the return value from the current internal state.

* The kurtosis aggregate is computed by calling an external

* library with all input values.

* @param state the internal state

* @return the kurtosis value

*/

@Override

public Object getAggregate(double[] state) {

DescriptiveStatistics ds = new DescriptiveStatistics(state);

double kurtosis = ds.getKurtosis();

if (Double.isNaN(kurtosis))

return 0.0;

else

return kurtosis;

}

/**

* Removes the input values from the internal state.

* This step is required for the support of the

* positive/negative approach.

* @param state the current internal state

* @param values the input values

* @return the new internal state

*/

@Override

public double[] negativeCall(double[] state, Serializable[] values) {

if (values[0] == null)

return state;

else {

boolean gotValue = false;

double[] newState = new double[state.length-1];

double value = (Double)values[0];

for (int i = 0; i < state.length; i++) {

gotValue |= state[i] == value;

newState[i] = state[i + (gotValue ? 1 :0)];

}

return newState;

}

}

/**

* Indicates support for positive/negative approach, which allows

* adding or removing values from the internal state;

* This approach provides a more efficient evaluation

* when windows are used.

* @return indicates whether PN approach is supported

*/

@Override

public boolean supportsPN() {

return true;

}

/**

* Indicates whether the internal state is empty.

* @param state the internal state

* @return indicates whether the internal state is empty

*/

@Override

public boolean isEmptyAggregate(double[] state) {

return state.length == 0;

}

}

To compile this example, you must include the Apache Commons Math library, version 2.2, in the classpath. You may include the jar file for this library in the lib folder for the user-defined function library. This specific library is also used in MashZone NextGen, so you also simply add the jar file for this library to the classpath.