search string side menu

BRIX On-Premises > BRIX On-Premises Enterprise > Install add-on components for BRIX / Install Tempo

Install Tempo

Tempo is a high-performance and scalable system for storing and analyzing traces. It provides complete visibility into interactions between components of distributed systems. Tempo eliminates the complexities associated with storing and retrieving traces from numerous sources in real-time.

Beforehand, install the monitoring tools.

The installation of Tempo consists of five steps:

  1. Download the Helm chart and the configuration file.
  2. Create buckets in Minio.
  3. Fill in the configuration file.
  4. Install the Tempo chart using Helm in the Kubernetes cluster.
  5. Configure the connection to Tempo in Grafana data sources.

Начало внимание

When installing Loki + Tempo, start with the Loki addon. This allows preserving cross-linking settings, as they are initially configured in the Datasource settings.

Конец внимание

Step 1: Download the Helm chart and the configuration file

To install via the internet, obtain the configuration file values-tempo.yaml by executing the command:

helm repo add elma365 https://charts.elma365.tech
helm repo update
helm show values elma365/tempo > values-tempo.yaml

 

Getting configuration file for installation in a closed environment without internet access

  1. On a computer with internet access, download the archive of the latest version of the Tempo chart from the BRIX repository using the command:

helm repo add elma365 https://charts.elma365.tech
helm repo update
helm pull elma365/tempo

  1. Copy the obtained tempo-X.Y.Z.tgz chart archive to the server where the installation will take place.
  2. Unpack the chart and copy the default configuration file values.yaml to values-tempo.yaml

tar -xf tempo-X.Y.Z.tgz
cp tempo/values.yaml values-tempo.yaml

Step 2: Create buckets in Minio

1. Create alias for MinIO:

mc alias set my_alias http://minio.local accessKey secretKey

2. Create a bucket with the name tempo:

mc mb -p my_alias/tempo --region=ru-central-1

Step 3: Fill in the configuration file

Fill in the configuration file for Tempo installation

 
 tempo:

   tempo:

     retention: 24h

     storage:

       trace:

         # tempo storage backend

         # refer https://grafana.com/docs/tempo/latest/configuration/

         ## Use s3 for example

         backend: s3

         # store traces in s3

         s3:

           bucket: tempo                        # store traces in this bucket

           endpoint: minio.local:9000           # api endpoint

           access_key: access_key               # optional. access key when using static credentials.

           secret_key: secret_key               # optional. secret key when using static credentials.

           insecure: true                       # optional. enable if endpoint is http

         #backend: local

         local:

           path: /var/tempo/traces

         wal:

           path: /var/tempo/wal

 jaeger:

   enabled: false

   fullnameOverride: "jaeger"

   provisionDataStore:

       cassandra: false

   storage:

     type: badger

   collector:

     enabled: false

   agent:

     enabled: true

     extraEnv:

       MEMORY_MAX_TRACES: 100000

       SPAN_STORAGE_TYPE: badger

       BADGER_EPHEMERAL: false

       BADGER_DIRECTORY_VALUE: /badger/data

       BADGER_DIRECTORY_KEY: /badger/key

       REPORTER_TYPE: grpc

       REPORTER_GRPC_HOST_PORT: tempo:14250

       REPORTER_GRPC_RETRY_MAX: 100

 

Filling in connection parameters to a private registry for installation in a closed environment without internet access

 
To connect to a private registry, follow these steps:

  1. Download BRIX images and upload them to the local image registry. For more details, refer to Download BRIX images.
  1. Specify the address and path for the image.repository parameter.
  1. Specify the name of the secret with access rights to the private registry in the image.pullSecrets parameter. The secret must be created manually and encoded in Base64.

 tempo:
 # connection parameters for the private registry
 # address and path for the private registry
   repository: registry.example.com/tempo
 #   tag: "2.3.0"
 #   pullPolicy: IfNotPresent
   pullSecrets:
     - myRegistryKeySecretName
 jaeger-all-in-one:
 # connection parameters for the private registry
   image:
 # address and path for the private registry
     repository: registry.example.com/all-in-one
 #     versionOverride: "1.47"
 #     pullPolicy: IfNotPresent
   imagePullSecrets:
     - myRegistryKeySecretName

Where the repository format is: address — registry.example.com.

Step 4: Install the Tempo chart using Helm in the Kubernetes cluster

Install Tempo chart in namespace monitoring.

For online installation:

helm upgrade --install -n monitoring tempo elma365/tempo -f values-tempo.yaml

For offline installation without internet access, navigate to the directory with the downloaded chart and execute:

helm upgrade --install tempo ./tempo -f values-tempo.yaml -n monitoring

Step 5: Set the parameters for sending traces

To display traces, set the jaegerAgentAddress or jaegerCollectorEndpoint parameters on the BRIX application side.

elma365:
  global:
    monitoring:
      jaegerAgentAddress: "jaeger-agent.monitoring.svc.cluster.local:5775"
      jaegerCollectorEndpoint: "http://tempo.monitoring.svc.cluster.local:14268/api/traces"

  1. The preferred method is to specify the HTTP endpoint of the collector using the global.monitoring.jaegerCollectorEndpoint parameter. In this case, traces will be sent directly to the collection system, bypassing intermediate components.
  2. An alternative option is to use Jaeger Agent, specifying its address in the global.monitoring.jaegerAgentAddress parameter. Traces are then transmitted to the agent via the UDP protocol, after which it forwards them to the storage system.
  3. If none of the parameters are specified, traces are automatically sent to port 5775 of the local node, where the Jaeger agent is expected to run in DaemonSet mode.

Step 6: Configure the connection to Tempo in Grafana data sources

The Tempo data source is added during installation. Verify it using the path:  Home > Administration > Data source > Tempo.

URL: http://tempo:3100

clip0086

For convenient viewing, a cross-link to Tempo is added to the Loki data source.

(Query: ${__value.raw}; Regex: (?:trace\.traceid|traceid)(?:":|=)"(\w+)")

clip0087

Configuring Derived Fields will facilitate viewing of the trace's reference connection in Tempo.

clip0088

install-mimir.html install-longhorn.html