Use Case 2: Load testing a non-http protocol

With the rise of connected objects in our day-to-day life, performance engineers need to increasingly take IoT (Internet of Things) oriented applications into account. But how is performance testing different with IoT? As a performance engineer, the most significant difference between traditional and IoT performance testing lies in the communication protocol. While traditional performance testing uses standard protocols (HTTP, SOAP…) to communicate, IoT also introduces new and sometimes non-standard protocols such as MQTT, CoAP, AMQP, etc.

NeoLoad has the ability to test MQTT out-of-the-box (for more information, see MQTT). For that reason we will spend some time here on CoAP, and more specifically on building an Advanced action using CoAP.

CoAP (Constrained Application Protocol) is, like HTTP, a document transfer protocol. CoAP is however designed for use with resource constrained devices.

As CoAP is designed to interoperate with HTTP, both protocols have a lot in common:

  • They both have a client/server model

  • They both use methods like GET, PUT, POST, DELETE

The main challenge we will have with NeoLoad is communicating with CoAP. It is very close to HTTP, but uses UDP instead of TCP. This is the reason we will have to develop our own Advanced action in order to communicate with a CoAP server.

As we will be using Java for the implementation of CoAP, we will use Californium. You can find information about Californium here: https://github.com/eclipse/californium

As we will be building CoAP Advanced actions, we will need an application to test it against. You can either:

  • build your own server with the demo servers provided on the GitHub page, or

  • use the test server provided by Californium: coap://californium.eclipse.org:5683/

We will use the provided test server at coap://californium.eclipse.org:5683.

The code of the Advanced action in this use case is on GitHub here: https://github.com/ttheol/NeoLoadTutorials/tree/master/neotys-advanced-action-coap-request

Prepare the action

As you are now familiar with eclipse and creating custom actions, we will go straight into the code. This tutorial will describe the implementation of the CoAP GET method.

  • Create a new Maven project.

  • Add the Californium dependencies in the "pom.xml" file.

    Copy 
    <dependency>
        <groupId>org.eclipse.californium</groupId>
        <artifactId>californium-core</artifactId>
        <version>1.0.4</version>
    </dependency>

  • Exclude the manifest signatures on creation of the jar.

  • Click on Edit > Search/Replace.

  • Search for maven-shade-plugin.

  • Update the section to match the following one:

    Copy 
    <plugin>
      <artifactId>maven-shade-plugin</artifactId>
      <version>2.1</version>
      <executions>
        <execution>
          <phase>package</phase>
          <goals>
            <goal>shade</goal>
          </goals>
          <configuration>
            <artifactSet>
              <excludes>
                <exclude>*:neotys-custom-action-api</exclude>
              </excludes>
            </artifactSet>
            <filters>
              <filter>
                <artifact>*:*</artifact>
                <excludes>
                  <exclude>META-INF/*.SF</exclude>
                  <exclude>META-INF/*.DSA</exclude>
                  <exclude>META-INF/*.RSA</exclude>
                </excludes>
              </filter>
            </filters>
          </configuration>
        </execution>
      </executions>
    </plugin>

We now have our skeleton for the Advanced action.

Configure the interface with NeoLoad

We will now configure how the Advanced action is presented. As shown in the previous action, you can change the default name, icon, etc. For more information, see Configure the interface of the Advanced action). We will be specifying the parameters, as well as a description for the Advanced action.

  • Specify the default parameters:

    Copy 
    @Override
    public List<ActionParameter> getDefaultActionParameters() {
        final List<ActionParameter> parameters = new ArrayList<ActionParameter>();
        parameters.add(new ActionParameter("Method","GET"));
        parameters.add(new ActionParameter("Server",""));
        parameters.add(new ActionParameter("Port","5683"));
        parameters.add(new ActionParameter("Path",""));
        return parameters;
    }

  • Specify the description:

    Copy 
    @Override
    public String getDescription() {
    final StringBuilder description = new StringBuilder();
    description.append("This advanced action sends a CoAP request.\n\n");
    description.append("Possible parameters are:\n");
    description.append("  - Method (required): Method to use to send the request to the server. Possible value are GET, POST, PUT, DELETE\n");
    description.append("  - Server (required): address of the CoAP server\n");
    description.append("  - Port (required): Port of the CoAP endpoint\n");
    description.append("  - Path (required): Path to the URI\n");
    description.append("  - Confirmable (optional): Set to true or false.\n    If true, request is confirmable. If false, request is non-confirmable.\n");
    description.append("  - Payload (optional): Send a payload along a POST or PUT request\n");
    return description.toString();
    }

  • We will also add a function to count the action as a request by default. You will need to place those lines in the methods of your ActionEngine class:

    Copy 
    @Override
    public boolean getDefaultIsHit() {
        return true;
    }

The file can be found on GitHub: https://github.com/ttheol/NeoLoadTutorials/blob/master/neotys-advanced-action-coap-request/src/main/java/com/neotys/ps/iot/coap/client/CoAPClientAction.java

Code the behavior of the Advanced action

For the time being we will stick to sending simple requests to see how Californium works. We have four types of requests: GET, POST, PUT and DELETE.

  • Declare the parameters we need in the ActionEngine class attributes:

    Copy 
    private static String server;
    private static String port;
    private static String path;
    private static String method;
    private static String scheme;
    private static boolean confirmable;
    private static String payload;

  • Parse the parameters from NeoLoad in ActionEngine. We will create a function called "parseParameters" in the class definition of ActionEngine.

    Copy 
    private static void parseParameters(List<ActionParameter> parameters){
            //Setting default value
            scheme="coap";
            confirmable=true;        
            payload="";
            //parameterList=null;
            parameterList=new StringBuilder();
            for (ActionParameter temp:parameters){
                switch (temp.getName().toLowerCase()) {
                case "method":
                    method = temp.getValue().toUpperCase();
                    break;
                case "server":
                    server = temp.getValue();
                    break;
                case "port":
                    port = Integer.parseInt(temp.getValue());
                    break;
                case "path":
                    path = temp.getValue();
                    break;
                case "confirmable":
            confirmable=Boolean.parseBoolean(temp.getValue().toLowerCase());    
                    break;
                case "payload":
                    payload=temp.getValue();
                default :                
                    break;
                }
            }
        }

  • Create four functions to call the different CoAP methods:

    Copy 
    private static CoapResponse sendGet(CoapClient client) {
            //Initialise the response
            CoapResponse response=null;        
            //Send the request
            response = client.get();
            //Return the response
            return response;
        }
        private static CoapResponse sendPost(CoapClient client) {
            //Initialise the response
            CoapResponse response=null;
            //Send the request
            response = client.post(payload, MediaTypeRegistry.TEXT_PLAIN);
            //Return the response
            return response;
        }
        private static CoapResponse sendPut(CoapClient client) {
            //Initialise the response
            CoapResponse response=null;
            //Send the request
            response = client.put(payload, MediaTypeRegistry.TEXT_PLAIN);
            //Return the response
            return response;
        }
        private static CoapResponse sendDelete(CoapClient client) {
            //Initialise the response
            CoapResponse response=null;
            //Send the request
            response = client.delete();
            //Return the response
            return response;
        }

  • Code the execute method:

    Copy 
    @Override
    public SampleResult execute(Context context, List<ActionParameter> parameters) {
        final SampleResult sampleResult = new SampleResult();
        final StringBuilder requestBuilder = new StringBuilder();
        final StringBuilder responseBuilder = new StringBuilder();
        parseParameters(parameters);
        //Instantiate the client
        CoapClient client = new CoapClient(scheme,server,port,path);
        if (confirmable) {
            client.useCONs();
        } else {
            client.useNONs();
        }
        //Log the request for the check VU
        appendLineToStringBuilder(requestBuilder,String.format("%s %s",method,client.getURI()));
        appendLineToStringBuilder(requestBuilder,String.format("\n%s",payload));
        sampleResult.setRequestContent(requestBuilder.toString());
        sampleResult.sampleStart();
        CoapResponse response=null;
        //Send the request
        switch (method) {
        case "GET":
            response = sendGet(client);
            break;
        case "POST":
            response = sendPost(client);
            break;
        case "PUT":
            response = sendPut(client);
            break;
        case "DELETE":
            response = sendDelete(client);
            break;
        default:
            response = sendGet(client);
            break;
        }
        //If there's a response
        if (response != null) {
            //Log the response in the check VU
            appendLineToStringBuilder(responseBuilder, Utils.prettyPrint(response));
        } else {
            //Log the error in the checkVU
            return getErrorResult(context,sampleResult,"No response","NL-CoAPClient-01",null);
        }
        sampleResult.sampleEnd();
        //Log the response in the check VU
        sampleResult.setResponseContent(responseBuilder.toString());
        return sampleResult;
    }

  • Change the getErrorResult function to be able to customize the error codes:

    Copy 
    private static SampleResult getErrorResult(final Context context, final SampleResult result, final String errorMessage, final String statusCode, final Exception exception) {
        result.setError(true);
        result.setStatusCode(statusCode);
        result.setResponseContent(errorMessage);
        if(exception != null){
            context.getLogger().error(errorMessage, exception);
        } else{
            context.getLogger().error(errorMessage);
        }
        return result;
    }

The final action engine can be found here: https://github.com/ttheol/NeoLoadTutorials/blob/master/neotys-advanced-action-coap-request/src/main/java/com/neotys/ps/iot/coap/client/CoAPClientActionEngine.java

Push the action to NeoLoad

Once you have finished developing your Advanced action, you can push it to NeoLoad and try it as explained in Check the behavior of the Advanced action.