9. Micrometer

Register time and log applications that are pre-built with io.micrometer:micrometer-registry-influx. The next version of SCSt App Initializr allows adding Micrometer registry dependencies as well.

app register --name time2 --type source --uri file://<path-to-your-time-app>/time-source-kafka-2.0.0.BUILD-SNAPSHOT.jar --metadata-uri file://<path-to-your-time-app>/time-source-kafka-2.0.0.BUILD-SNAPSHOT-metadata.jar

app register --name log2 --type sink --uri file://<path-to-your-log-app>/log-sink-kafka-2.0.0.BUILD-SNAPSHOT.jar --metadata-uri file://<path-to-your-log-app>/log-sink-kafka-2.0.0.BUILD-SNAPSHOT-metadata.jar

Create InfluxDB and Grafana Docker containers

docker run -d --name grafana -p 3000:3000 grafana/grafana:5.1.0

docker run -d --name influxdb -p 8086:8086 influxdb:1.5.2-alpine

Create and deploy the following stream

dataflow:>stream create --name t2 --definition "time2 | log2"

dataflow:>stream deploy --name t2 --properties "app.*.management.metrics.export.influx.db=myinfluxdb"

The app.*.management.metrics.export.influx.db=myinfluxdb instructs the time2 and log2 apps to use the myinfluxdb database (created automatically).

By default, the InfluxDB server runs on localhost:8086. You can add the app.*.management.metrics.export.influx.uri={influxbb-server-url} property to alter the default location.

You can connect to the InfluxDB and explore the measurements

docker exec -it influxdb /bin/bash
root:/# influx
> show databases
> use myinfluxdb
> show measurements
> select * from spring_integration_send limit 10

Configure Grafana

Register time and log applications that are pre-built with io.micrometer:micrometer-registry-prometheus. The next version of SCSt App Initializr allows adding Micrometer registry dependencies as well.

app register --name time2 --type source --uri file://<path-to-your-time-app>/time-source-kafka-2.0.0.BUILD-SNAPSHOT.jar --metadata-uri file://<path-to-your-time-app>/time-source-kafka-2.0.0.BUILD-SNAPSHOT-metadata.jar

app register --name log2 --type sink --uri file://<path-to-your-log-app>/log-sink-kafka-2.0.0.BUILD-SNAPSHOT.jar --metadata-uri file://<path-to-your-log-app>/log-sink-kafka-2.0.0.BUILD-SNAPSHOT-metadata.jar

Create and deploy the following stream

dataflow:>stream create --name t2 --definition "time2 | log2"

dataflow:>stream deploy --name t2 --properties "app.*.management.endpoints.web.exposure.include=prometheus,app.*.spring.autoconfigure.exclude=org.springframework.boot.autoconfigure.security.servlet.SecurityAutoConfiguration"

The deployment properties make sure that the prometheus actuator is enabled and the Spring Boot security is disabled

Build and start the SCDF Prometheus Service Discovery application

Build the spring-cloud-dataflow-prometheus-service-discover project form: github.com/spring-cloud/spring-cloud-dataflow-samples/micrometer/spring-cloud-dataflow-prometheus-service-discovery

cd ./spring-cloud-dataflow-samples/micrometer/spring-cloud-dataflow-prometheus-service-discovery
./mvnw clean install

For convenience, the final spring-cloud-dataflow-prometheus-service-discovery-0.0.1-SNAPSHOT.jar artifact is provided with this sample.

Start the service discovery application:

java -jar ./target/spring-cloud-dataflow-prometheus-service-discovery-0.0.1-SNAPSHOT.jar \
--metrics.prometheus.target.discovery.url=http://localhost:9393/runtime/apps \
--metrics.prometheus.target.file.path=/tmp/targets.json \
--metrics.prometheus.target.refresh.rate=10000 \
--metrics.prometheus.target.mode=local

It will connect to the SCDF runtime url, and generates /tmp/targets.json files every 10 sec.

Create Prometheus configuration file (prometheus-local-file.yml)

global:
  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
  # scrape_timeout is set to the global default (10s).

# A scrape configuration containing exactly one endpoint to scrape:
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
  - job_name: 'scdf'
    metrics_path: '/actuator/prometheus'
    file_sd_configs:
      - files:
        - targets.json
        refresh_interval: 30s

Configure the file_sd_config discovery mechanism using the generated targets.json:

Start Prometheus

docker run -d --name prometheus \
 -p 9090:9090 \
 -v <full-path-to>/prometheus-local-file.yml:/etc/prometheus/prometheus.yml \
 -v /tmp/targets.json:/etc/prometheus/targets.json \
 prom/prometheus:v2.2.1

Pass the prometheus.yml and map the /tmp/targets.json into /etc/prometheus/targets.json

Use the management UI: localhost:9090/graph to verify that SCDF apps metrics have been collected:

# Throughput
rate(spring_integration_send_seconds_count{type="channel"}[60s])

# Latency
rate(spring_integration_send_seconds_sum{type="channel"}[60s])/rate(spring_integration_send_seconds_count{type="channel"}[60s])

Start Grafana Docker containers

docker run -d --name grafana -p 3000:3000 grafana/grafana:5.1.0

Configure Grafana