14. Telemetry
A beginner’s guide to OpenTelemetry
Last updated
Was this helpful?
A beginner’s guide to OpenTelemetry
Last updated
Was this helpful?
is a collection of tools, APIs, and SDKs.
Instrument the application using OpenTelemetry SDK.
Traces are sent to a (collector) agent.
The collector exposes ports 4317(gRPC) and/or 4318(http) using OTLP.
Traces are then exported (in this case to ) and stores the traces on the backend
The Jaeger Query retrieves the traces and expose them to the Jaeger UI.
Jaeger UI provides a web based user interface that can be used to analyse traces.
The storage backend can be , , , , , , a vendor (, , , , etc..).
This is good, because that means that we are not tied to any tool (or vendor). Not only can we use any programming language we want, but we can also pick and choose the storage backend, thus avoiding a potential buy in from commercial vendors.
It also means that developers can instrument their application without having to know where the data will be stored.
Defines how OpenTelemetry is used.
Defines the specific implementation of the API for a language.
As you can see from the image (at the top), in order to get trace data, we first need to instrument the application. To collect the trace data, we can use the OpenTelemetry SDK.
The trace data can be generated using either automatic or manual (or a mix) instrumentation.
One of the best ways to instrument applications is to use OpenTelemetry automatic instrumentation (auto-instrumentation). This approach is simple, easy, and doesn’t require many code changes.
Using auto-instrumentation libraries, means that you don’t need to write code for the trace information to be collected. In fact, OpenTelemetry offers an API and SDK that allows for easy bootstrapping of distributed tracing into your software.
This is good to know if don’t have the necessary knowledge (or time) to create a tracing framework tailored for your application.
For example, running the following command will automatically instrument your python code.
opentelemetry-instrument python app.py
When you use auto instrumentation, a predefined sets off spans will be created for you and populated with relevant attributes.
Manual instrumentation is when you write specific code for your application. It’s the process of adding observability code to your application. This can more effectively suit your needs. For example, you can add attributes and events.
Once you have collected trace data, you need to send it somewhere.
The OTLP protocol describes how to encode and transmit telemetry data, which makes it a natural choice for data transport. Each language SDK provides an OTLP exporter you can configure to export data over OTLP. The OpenTelemetry SDK then transforms events into OTLP data.
The collector can be setup as an agent or as a gateway.
We usually first send traces to a (collector) agent. This (collector) agent handles the trace data from the instrumented application.
The (collector) agent can offload responsibilities that the client instrumentation otherwise need to handle. This includes batching, retry, encryption, compression and more.
You can also perform sampling here depending on the amount of traces/traffic you want to send. (ie. take only 10% of the traces).
You need to configure receivers (how data gets into the collector), which then transform the data (process) before sending it to one or more backends using exporters.
Here we configure a receiver (on the collector) that accepts OTLP data on port 4317 (gRPC). (You also need to configure your application to export over OTLP to use it)
The exporters converts OpenTelemetry protocol (OTLP) formatted data to their respective predefined back-end format and exports this data to be interpreted by the back-end or system.
It’s important to know that the OpenTelemetry collector does not provides their own backend.
gives us the tools to create trace data. It provides a vendor agnostic standard for observability as it aims to standardise the generation of traces.
To instrument your application with , go to the OpenTelemetry , and pick the language for your application and follow the instructions.
On you can search for libraries, plugins, integrations, and other useful tools for extending OpenTelemetry.
OpenTelemetry Protocol (OTLP) specification describes the encoding, transport, and delivery mechanism of telemetry data between telemetry sources, intermediate nodes such as collectors and telemetry backends.
The data from your instrumented application can be sent to an collector.
The collector is a component of that collects trace data (spans, metrics, logs etc) process (pre-processes data), and exports the data (sends it off to a backend that you want to talk to).
The collector can receive telemetry data in multiple formats.
In our article, we will talk about this in more detail and also show how you can deploy this on a local Kubernetes cluster.
Once you’ve instrumented your code, you need to get the data out in order to do anything useful with it. comes with a variety of exporters.
For example, for metrics, there is a exporter that allows sending metrics so that Prometheus can consume it.
A common scenario (especially during testing), is to export all data directly to using the jaeger-exporter.
In our article, we will talk about this in more detail and also show how you can deploy this on a local Kubernetes cluster.
You can read more about exporters
The storage backend can be , , , , , , a vendor (, , , , etc..).
For a full list of storage alternatives, please checkout the repository.
This OpenTelemetry provides a complete demo on how you can deploy OpenTelemetry on Kubernetes.
Please check this on how to deploy this on a local Kubernetes cluster.
Checkout to quickly get started with OpenTelemetry. This repo contains a big list of helpful resources.
