Manager 2024.2 Installation Guide for Kubernetes

The following chapters walk you through the installation of qTest Manager 2024.2 and its additional applications on a Kubernetes cluster, using Helm.

Prerequisites

The following requirements apply to proceed with the installation:

  • Kubernetes 1.18 (or higher)

  • Support for PersistentVolume (PV) in the underlying infrastructure (PV must set persistentVolumeReclaimPolicy to Retain)

  • Helm 3.x

  • PostgreSQL 13.x

  • Elasticsearch 7.x

For more details on prerequisites, see chapter "2024.2 OnPremises Prerequisite Software for Linux and Kubernetes".

What's included

The qTest Helm repository includes the qTest applications listed below. You must first install qTest Manager to then choose which other applications to deploy. Note that the applications you can install are tied to your qTest license tier.

  • qtest-mgr-1.6.0 

  • qtest-session-1.4.0

  • qtest-parameters-1.2.0

  • qtest-insights-1.5.0

  • qtest-launch-1.3.0

  • qtest-pulse-1.1.0

  • qtest-scenario-1.2.0

  • qtest-insights-etl-1.3.0

Architecture overview

The diagram below offers an overview of the pod and deployment structure of qTest Manager and its applications using Kubernetes and Docker.

Docker Hub Repository Authentication

qTest uses private Docker images and requires authentication via Docker. To begin with the authentication process, perform the following steps:

  1. Initiate the Docker login process:

    docker login
  2. When prompted, enter your Docker ID and access token. The login process creates or updates a config.json file that holds an authorization token, typically located under $HOME/.docker/config.json.

  3. Create a corresponding Kubernetes secret named regcred. This can be created from existing credentials as follows:

    kubectl create secret generic regcred \
    
    --from-file=.dockerconfigjson=<path/to/.docker/config.json> \
    
    --type=kubernetes.io/dockerconfigjson
  4. If you do not wish to use the Docker config.json file, the secret can also be created directly from the command line:

    kubectl create secret docker-registry regcred --docker-server=<your-registry-server> --docker-username=<your-name> --docker-password=<your-password> --docker-email=<your-email>

    Replace the parameters above as follows:

    • <your-registry-server> is your Private Docker Registry FQDN. Use https://index.docker.io/v1/ for DockerHub.

    • <your-name> is your Docker username.

    • <your-password> is your Docker access token.

    • <your-email> is your Docker email.

Installation

To install qTest Manager 2024.2 and its additional applications, follow the steps below:

  1. Add the qTest Helm repository to your environment:

    helm repo add qtest https://tricentis.github.io/qTest.Charts
    helm repo update
  2. Install the qTest Manager chart with the release name qtest-manager in the qtest namespace:

    helm install qtest-manager qtest/qtest-mgr -f <values.yaml> -n qtest --create-namespace

    You can use the values-aws-eks.yaml file for reference values.

  3. Install other qTest applications as required, using the syntax below:

    helm install <qTest Application> qtest/<qTest Application> -n qtest

    Replace both <qTest Application> strings with one of the values below:

    • Sessions: qtest-sessions

    • Insights: qtest-insights

    • Parameters: qtest-parameters

    • Launch: qtest-launch

    • Pulse: qtest-pulse

    • Scenario: qtest-scenario

Configuration

The following sections list the configurable parameters for qTest Manager, its applications, and your qTest license. At a minimum, the postgres and elasticSearch parameters should be provided to match your environment.

You can use the provided sample YAML files as a reference during the deployment process.

Ingress Controller

qTest relies on an IngressController to route ingress traffic into the cluster. We recommend SSL offloading/termination to be done at the IngressController. Specific TLS setup instructions depend on the IngressController you have.

To define an IngressController for qTest, specify the IngressControllerClass name via the Helm value ingressClass.controller.

Below is the configuration for AWS ALB IngressController:

ingressClass:
   enabled: true
   controller: ingress.k8s.aws/alb

The configuration for NginX IngressController is as follows:

ingressClass:
   enabled: true
   controller: k8s.io/ingress-nginx
qTest Manager and its applications use the new IngressClass provided in Kubernetes versions 1.18 and higher, which deprecates the preceding annotation syntax in the Kubernetes Ingress resource.

qTest Manager

At a minimum, the postgres and elasticSearch parameters should be provided to match your environment.

Parameter

Description

Default

attachment.folder.path

qTest attachment folder path

/mnt/data/qtest/attachments

blobstorage.type

qTest Manager attachments folder type

disk_storage (accepted values disk_storage, amazon_s3_access_key)

blobstorage.region

S3 bucket region

us-east-1 (needed when blobstorage.type is amazon_s3_access_key)

blobstorage.sharedBucket

S3 bucket name

qtest (needed when blobstorage.type is amazon_s3_access_key)

client.jdbc.postgresUrl

PSQL database URL

jdbc:postgresql://host.docker.internal/qtest
client.jdbc.postgresUserName

PSQL username

postgres
client.jdbc.postgresPassword

PSQL password

cG9zdGdyZXM= (postgres, base64-encoded)

client.jdbc.sslEnable

Enable SSL connections

false

client.jdbc.postgres.readonly.url

PSQL read-only database URL

jdbc:postgresql://host.docker.internal/qtest

client.jdbc.postgres.readonly.username

PSQL username

postgres

client.jdbc.postgres.readonly.password

PSQL password

cG9zdGdyZXM= (postgres, base64-encoded)

client.jdbc.sslMountPath

PostgreSQL SSL certificate mount directory

\etc\ssl

client.jdbc.sslPath

PostgreSQL SSL connection string (changes to certificate path only)

?ssl=true&sslmode=verify-full&sslrootcert=/etc/ssl/root.crt

client.jdbc.cert

PostgreSQL client certificate

`` (postgres client certificate, base64-encoded)

cors.allowed.all

Cors allowed

true

cors.allowed.domains

Cors allowed domains

``

elasticSearch.host

Elasticsearch host

host.docker.internal
elasticSearch.port

Elasticsearch port

9200
elasticSearch.scheme

Elasticsearch scheme

http

license.folder.path

qTest Manager license folder path

/mnt/data/qtest/license

mail.host

SMTP hostname

smtp.local

mail.port

SMTP port number

465

mail.username

SMTP username

qtest

mail.password

SMTP password

cG9zdGdyZXM= (postgres, base64-encoded)

mail.supportEmailAddress

qTest Support e-mail address

support@tricentis.com

notification.urlExternal

Notification URL

https://nephele.qtest.local

preUrl

qTest Manager URL

http://<sitename>.qtest.local

preUrlHttps

qTest Manager URL

https://<sitename>.qtest.local

qtest.request.nonce.disabled

qTest request nonce disabled

true

qtest.request.nonce.mode

qTest request nonce mode

HighPrecision

s3.accessKey

S3 access key

""

s3.secretKey

S3 secret key

""

s3.region

S3 region

""

s3.bucketName

S3 bucket name

""

s3.folder

S3 folder

""

security.csrf.source.trust.pattern

qTest Manager CSRF security pattern

``

vera.auto.testrun.beta.clients

SMTP port number

-1

autoscaling.enabled

Set minimum replicas

true

autoscaling.minReplicas.ui

Minimum replicas

1

autoscaling.maxReplicas.ui

Maximum replicas

3

autoscaling.minReplicas.api

Minimum replicas

1

autoscaling.maxReplicas.api

Maximum replicas

3

autoscaling.minReplicas.notification

Minimum replicas

1

autoscaling.maxReplicas.notification

Maximum replicas

1

autoscaling.minReplicas.poller

Minimum replicas

1

autoscaling.maxReplicas.poller

Maximum replicas

1

ingress.enabled

Enable Ingress

false

ingress.class

Name of IngressClass

nginx

ingress.host

qTest Manager hostname

nephele.qtest.local

ingress.tls.hosts

Configuration for TLS on the Ingress

mgr.qtest.local

limitRange.enabled

Set resource limits

true

limitRange.limits.max.cpu

Maximum CPU

2

limitRange.limits.max.memory

Maximum memory

16Gi

limitRange.limits.default.cpu

Default CPU

2

limitRange.limits.default.memory

Default memory

16Gi

limitRange.limits.defaultRequest.cpu

Requested CPU

2

limitRange.limits.defaultRequest.memory

Requested memory

4Gi

persistence.enabled

Enable PersistentVolume

true

persistence.existingClaim

Existing PersistentVolumeClaim

""

persistence.storageClass

Existing StorageClass

""

persistence.accessMode

Storage Access Mode

ReadWriteOnce

persistence.size

Storage size

10Gi

service.port

Service port for other pods

8080

service.targetPort

Container port to serve traffic

8080

testconductor.environment.singleInstance

Runs in qTest Manager in a single pod

true

serverAppSSLRequired

Runs pod of qTest Manager on SSL

false

server.sslMountPath

Pod certificate mounting path

/mnt/secrets/tls

For file persistence, only one of the following options is needed:

  • AWS S3: file attachments are stored into the specified S3 bucket or folder.

  • StorageClass: used to carve out a PersistentVolumeClaim with the specified size and access mode. If empty, the default StorageClass will be used.

  • existingClaim: uses PersistentVolumeClaim as file persistence for qTest if specified.

qTest Parameters

Parameter

Description

Default
qTestParametersDBName

qTest Parameters PSQL database name

parameters
qTestParametersDBUserName

PSQL username

postgres
qTestParametersDBHostName

PSQL database hostname

host.docker.internal
qTestParametersDBSSLEnable

Enable DB SSL connections

false
qTestParametersDBSSLMountPath

PostgreSQL SSL certificate mount directory

\etc\ssl

qTestParametersDBSSL

SSL Connection which appends for SSL Connection (changes only to the certificate name)

?ssl=true&sslmode=verify-full&sslrootcert=/etc/ssl/root.crt

qTestParametersDBRootCRT

PostgreSQL client certificate

`` (postgres client certificate, base64-encoded)

targetHttpsPort

Port used for SSL connections.

5443

qTestParametersSSLRequired

Enable SSL connections

false

qTestParametersSSLMountPath

Pod certificate mounting path

/mnt/secrets/tls

qTest Launch

Parameter

Description

Default
qTestLaunchDBName

qTestManager PSQL database name

qtest
qTestLaunchDBUserName

PSQL username

postgres
qTestLaunchDBHostName

PSQL database hostname

host.docker.internal
qTestLaunchRootURL

qTest Launch URL

https://launch.qtest.local
qTestLaunchDBSSLEnable

Enable SSL connections

false

qTestLaunchDBSSLMountPath

PostgreSQL SSL certificate mount directory

\etc\ssl
qTestLaunchDBSSL

SSL Connection which appends for SSL Connection (changes only to the certificate name)

?ssl=true&sslmode=verify-full&sslrootcert=/etc/ssl/root.crt

qTestLaunchDBRootCRT

PostgreSQL client certificate

`` (postgres client certificate, base64-encoded)

qTest Pulse

Parameter

Description

Default
qTestPulseDBName

qTest Pulse PSQL database name

pulse
qTestPulseDBUserName

PSQL username

postgres
qTestPulseDBHostName

PSQL database hostname

host.docker.internal
qTestPulseRootURL

qTest Pulse URL

https://pulse.qtest.local
qTestPulseScenarioURL

qTest Scenario URL

https://scenario.qtest.local

qTestPulseDBSSLEnable

Enable SSL connections

false
qTestPulseDBSSLMountPath

PostgreSQL SSL certificate mount directory

\etc\ssl

qTestPulseDBSSL

SSL Connection which appends for SSL Connection (changes only to the certificate name)

?ssl=true&sslmode=verify-full&sslrootcert=/etc/ssl/root.crt

qTestPulseDBRootCRT

PostgreSQL client certificate

`` (postgres client certificate, base64-encoded)

qTest Scenario

Parameter

Description

Default
qTestScenarioDBName

qTest Scenario PSQL database name

pulse
qTestScenarioDBUserName

PSQL username

postgres
qTestScenarioDBHostName

PSQL database hostname

host.docker.internal
qTestScenarioLocalBaseURL

qTest Pulse URL

https://pulse.qtest.local
qTestScenarioQTestURL

qTest Launch URL

https://nephele.qtest.local

qTestScenarioDBSSLEnable

Enable SSL connections

false
qTestScenarioDBSSLMountPath

PostgreSQL SSL certificate mount directory

\etc\ssl

qTestScenarioDBSSL

SSL Connection which appends for SSL Connection (changes only to the certificate name)

?ssl=true&sslmode=verify-full&sslrootcert=/etc/ssl/root.crt

qTestScenarioDBRootCRT

PostgreSQL client certificate

`` (postgres client certificate, base64-encoded)

qTest Sessions

Parameter

Description

Default
qTestSessionDBName

qTest Sessions PSQL database name

sessions
qTestSessionDBUserName

PSQL username

postgres
qTestSessionDBHostName

PSQL database hostname

host.docker.internal

qTestStorageBucketName

Amazon S3 bucket name

aws-session-bucket-name

qTestSessionClamavURL

qTest clama url

https://clam.qtest.local
qTestSessionStorageType

qTest Sessions storage type

`amazon_s3

qTestSessionDBSSLEnable

Enable SSL connections

false
qTestSessionDBSSLMountPath

PostgreSQL SSL certificate mount directory

\etc\ssl

qTestSessionDBSSL

SSL Connection which appends for SSL Connection (changes only to the certificate name)

?ssl=true&sslmode=verify-full&sslrootcert=/etc/ssl/root.crt

qTestSessionDBRootCRT

PostgreSQL client certificate

`` (postgres client certificate, base64-encoded)

qTest Insights

Parameter

Description

Default
qTestInsightsDBName

qTest Manager PSQL database name

qTest
qTestInsightsDBUser

PSQL username

postgres
qTestInsightsDBHost

PSQL database hostname

host.docker.internal

qTestInsightsWriteQTestDBName

PSQL database name of the qTest manager

qTest

qTestInsightsWriteQTestDBUser

PSQL username

postgres
qTestInsightsWriteQTestDBPassword

PSQL password

cG9zdGdyZXM= (postgres, base64-encoded)

qTestInsightsWriteQTestDBHost

PSQL database hostname

host.docker.internal
qTestInsightsSessionDBName

PSQL database name of the session

sessions

qTestInsightsSessionDBUser

PSQL username

postgres

qTestInsightsSessionDBPassword

PSQL password

cG9zdGdyZXM= (postgres, base64-encoded)

qTestInsightsSessionDBHost

PSQL database hostname

host.docker.internal

qTest Insights ETL

Parameter

Description

Default
qTestInsightsEtlDBName

qTest Manager PSQL database name

qTest
qTestInsightsEtlDBUser

PSQL username

postgres
qTestInsightsEtlDBHost

PSQL database hostname

host.docker.internal

qTestInsightsEtlWriteQTestDBName

qTest Manager PSQL database name

qTest

qTestInsightsEtlWriteQTestDBUser

PSQL username

postgres

qTestInsightsEtlWriteQTestDBHost

PSQL database hostname

host.docker.internal
qTestInsightsEtlSessionDBName

PSQL database name of the session

sessions

qTestInsightsEtlSessionDBUser

PSQL username

postgres

qTestInsightsEtlSessionDBHost

PSQL database hostname

host.docker.internal

qTest License

The final step is to configure your qTest License. To do so, perform the following steps:

  1. Once the server is configured and running, copy the Server ID from Administrator -> License.

  2. Contact Tricentis Support and provide the Server ID from the previous step to request a license file.

  3. Using the user interface, import the license you received.

  4. The screen below demonstrates an example of a successful deployment:
$ helm install qtest-mgr qtest/qtest-mgr -f values-aws-eks.yaml -n
qtest –create-namespace
NAME: qtest-mgr
LAST DEPLOYED: Thu Jan 13 17:49:00 2022
NAMESPACE: qtest
STATUS: deployed
REVISION: 1

$ kubectl -n qtest get pods
NAME					READY	STATUS		RESTARTS	AGE
mgr-ui-deployment-6b77987c79-q2msb	1/1	Running		0		18m

Configure SSL

This section shows how you can set up SSL connections for different qTest applications. Note that additional SSL validation at the pod level may impact the performance of qTest.

qTest Manager

To configure qTest Manager to use SSL, follow these steps:

  1. Create your certificates and deploy them as a secret in the same namespace where qTest will be deployed.

    The name of the secret must be qtest-ssl-root-secret.

  2. Set the parameter serverAppSSLRequired to true.

  3. By default, the pod certificate mounting path is /mnt/secrets/tls. To mount a different directory, change the parameter server.sslMountPath accordingly.

qTest Parameters

To configure qTest Parameters to use SSL, follow these steps:

  1. Make sure you have configured qTest Manager to use SSL.

  2. In your qTest Parameters values, set the parameter qTestParametersSSLRequired to true.

  3. To make changes to the default HTTPS port used for the connection, modify the parameter targetHttpsPort.

  4. By default, the pod certificate mounting path is /mnt/secrets/tls. To mount a different directory, change the parameter qTestParametersSSLMountPath accordingly.

Autoscaling

qTest Manager and its applications can be automatically scaled in and out by setting the autoscaling.enabled parameter in the respective values.yaml file to true (default). You must also install the Metrics server on the Kubernetes cluster to provide the required CPU and memory metrics for autoscaling. For AWS EKS, see the Amazon EKS documentation.

qTest uses Kubernetes Horizontal Pod Autoscaler (HPA) to autoscale its pods, as seen in the section below.

HPA Autoscaling in qTest Manager

By default, qTest Manager is deployed in a single pod (UI) which provides all the qTest Manager functionality needed within one deployment. You can scale it as described in the HPA example that follows. The advantage of doing so is less CPU memory footprint and a simpler deployment process.

For finer-grained control, qTest Manager additionally allows autoscaling for each of its 4 “mini-services”, each running in its own pod:

  • UI: User interface and core functionality

  • API: API server

  • Notification: Web sockets

  • Poller: Integration for Rally and VersionOne

To enable the autoscaling for these services, set testconductor.environment.singleInstance to false.

autoscaling:
  enabled: true
  minReplicas:
    ui: 1
    api: 1
    notification: 1
    poller: 1
    default: 1
  maxReplicas:
    ui: 3
    api: 3
    notification: 3
    poller: 1
  targetCPUUtilizationPercentage: 70
  targetMemoryUtilizationPercentage: 70

The example above creates a horizontal pod autoscaler in Kubernetes which is configured to:

  • Create at least one pod for the component.

  • Scale the component to a maximum of maxReplicas pods.

  • Observe the CPU usage of all replicas and try to scale between one and maxReplicas pods.

  • Observe the memory usage of all replicas and try to scale between one and maxReplicas pods.

Limit Ranges and ResourceQuotas

qTest supports ResourceQuotas to limit the amount of resources consumed by the qTest namespace. The default settings are presented below and can be configured to fit your workload:

resourceQuota:
  enabled: true
  hard:
    limits.cpu: "80"
    limits.memory: 400Gi
    requests.cpu: "64"
    requests.memory: 160Gi
    pods: "25"
    persistentvolumeclaims: "50"
    requests.storage: 500Gi

In addition, qTest applies LimitRanges to all containers running in the qTest namespace. The default settings are as follows:

limitRange:
  enabled: true
  limits:
    -max:
       cpu: "2"
       memory: 16Gi
    default:
       cpu: "2"
       memory: 16Gi
    defaultRequest:
       cpu: "2"
       memory: 4Gi      
    type: Container

Uninstall

To uninstall qTest Manager and other qTest applications, follow the steps below:

  1. Using the same <qTest Application> string from the installation steps, uninstall every qTest application which you installed after qTest Manager:

    helm uninstall <qTest Application>
  2. Once you have uninstalled all your qTest applications, uninstall qTest Manager:

    helm uninstall qtest-manager

Configure and Test Connectivity to qTest Applications

Follow the Configure qTest Applications guide to configure qTest Manager to connect with the other qTest applications you installed. You should also verify that the applications can be accessed and working properly before performing the next step.

What's next

Refer to the following chapters for more information on the qTest containerization with Kubernetes: