Exposing workflow base metrics to Prometheus

SonataFlow generates metrics that can be consumed by Prometheus and visualized by dashboard tools, such as OpenShift, Dashbuilder, and Grafana.

This document describes how you can enable and expose the generated metrics to Prometheus.

Enabling metrics in SonataFlow

You can enable the metrics in your workflow application.

Prerequisites

A workflow application is created.

For more information about creating a workflow, see Creating your first workflow service.

Procedure

To add the metrics to your workflow application, add the org.kie.kogito:kogito-addons-quarkus-monitoring-prometheus dependency to the pom.xml file of your project:
Dependency to be added to the pom.xml file to enable metrics
<dependency> <groupId>org.kie.kogito</groupId> <artifactId>kogito-addons-quarkus-monitoring-prometheus</artifactId> </dependency>
Rebuild your workflow application.

The metrics is available at /q/metrics endpoint.

Metrics consumption in SonataFlow

After enabling the metrics in SonataFlow, the generated metrics can be consumed from OpenShift, Kubernetes, and Prometheus to visualize on different dashboard tools.

Consuming metrics from OpenShift

If your workflow server is running on OpenShift, then you can use the server to monitor your workflow application. Also, you can perform the task of consuming metrics from OpenShift.

Prerequisites

Metrics is enabled in SonataFlow.

For more information, see Enabling metrics in SonataFlow.

Procedure

To consume metrics from OpenShift, enable monitoring for user-defined projects.

For more information, see Enabling monitoring for user-defined projects in OpenShift documentation.

When you enable monitoring for user-defined projects, the Prometheus Operator is installed automatically.

Create a service monitor as shown in the following configuration:

Example configuration in service-monitor.yaml

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    k8s-app: prometheus-app-monitor
  name: prometheus-app-monitor
  namespace: my-project
spec:
  endpoints:
  - interval: 30s
    targetPort: 8080
    path: /q/metrics
    scheme: http
  selector:
    matchLabels:
      app-with-metrics: 'serverless-workflow-app'

Run the following command to apply the service monitor:
Apply service monitor
oc apply -f service-monitor.yaml

In the previous procedure, a service monitor named prometheus-app-monitor is created, which selects applications containing the label as app-with-metrics: serverless-workflow-app. Ensure that your workflow application contains the same label.

After that, Prometheus sends request to the /q/metrics endpoint for all the services that are labeled with app-with-metrics: serverless-workflow-app every 30 seconds. For more information about monitoring Quarkus application using Micrometer and Prometheus into OpenShift, see Quarkus - Micrometer Metrics.

Consuming metrics from Kubernetes is similar to OpenShift. However, you need to install the Prometheus Operator project manually.

For more information about installing Prometheus Operator, see Prometheus Operator website.

Consuming metrics from Prometheus

If your workflow server is running on Prometheus, then you can perform the task of consuming metrics from Prometheus and visualize the workflow on different dashboard tools.

Prerequisites

Metrics is enabled in SonataFlow.

For more information, see Enabling metrics in SonataFlow.

Procedure

Use the following configuration to enable Prometheus to remove metrics directly from the workflow application:

Example Prometheus configuration

- job_name: 'Serverless Workflow App'
    scrape_interval: 2s
    metrics_path: /q/metrics
    static_configs:
        - targets: ['localhost:8080']

Replace the values of job_name and scrap_interval in the previous configuration with your own values.
Ensure that target under static_configs parameter in Prometheus configuration matches with your workflow application location.

For more information about configuring Prometheus, see Configure Prometheus to monitor the sample targets in Prometheus Getting Started document.

Example metrics in SonataFlow

In SonataFlow, you can check the following example metrics:

kogito_process_instance_completed_total: Completed workflows
kogito_process_instance_started_total: Started workflows
kogito_process_instance_running_total: Running workflows
kogito_process_instance_duration_seconds_sum: Workflows total duration

Internally, workflows are referred as processes. Therefore, the processId and processName is workflow ID and name respectively.

Each of the metrics mentioned previously contains a label for a specific workflow ID. For example, the kogito_process_instance_completed_total metric contains labels for jsongreet, yamlgreet, and foreach workflows:

Example kogito_process_instance_completed_total metric

# HELP kogito_process_instance_completed_total Completed Process Instances
# TYPE kogito_process_instance_completed_total counter
kogito_process_instance_completed_total{app_id="default-process-monitoring-listener",artifactId="kogito-serverless-workflow-demo",node_name="2",process_id="jsongreet",version="1.0.0-SNAPSHOT",} 154.0
kogito_process_instance_completed_total{app_id="default-process-monitoring-listener",artifactId="kogito-serverless-workflow-demo",node_name="2",process_id="yamlgreet",version="1.0.0-SNAPSHOT",} 218.0
kogito_process_instance_completed_total{app_id="default-process-monitoring-listener",artifactId="kogito-serverless-workflow-demo",node_name="2",process_id="foreach",version="1.0.0-SNAPSHOT",} 162.0

Internally, SonataFlow uses Quarkus Micrometer extension, which also exposes built-in metrics. You can disable the Micrometer metrics in SonataFlow. For more information, see Quarkus - Micrometer Metrics.

Found an issue?

If you find an issue or any misleading information, please feel free to report it here. We really appreciate it!