This is the protocol and semantic conventions documentation for Red Hat OpenShift Logging's OTEL support starting with Logging v6.1 which is considered Tech-Preview. This document should be considered as a work in progress and is subject to change until OTEL support graduates to General Acceptance.
Red Hat OpenShift Logging provides a log collection and forwarding solution that is capable of writing logs to OpenTelemetry endpoints using OTLP. OTLP is the protocol for encoding, transporting, and delivering telemetry data. This product additionally is capable of providing a Loki storage deployment that provides an OTLP endpont to ingest log streams. This document defines the semantic conventions associated with the logs collected from the various sources of an OpenShift cluster.
Note: Logs are forwarded using OTLP/HTTP as defined by the OpenTelemetry Observability Framework. It uses Protobuf payloads encoded in JSON format.
The log collector provided by this solution collects the following log streams:
- Container logs
- Cluster node journal logs
- Cluster node auditd logs
- Kubernetes and OpenShift API server logs
- OpenShift Virtual Network (OVN) logs
These streams are forwarded using the sementatic conventions defined by OpenTelemetry semantic attributes. The semantic conventions in OpenTelemetry define a Resource as an immutable representation of the entity producing telemetry as Attributes. For example, a process producing telemetry that is running in a container has a container_name, a cluster_id, a pod_name, a namespace, and possibly a deployment or app_name. All of these Attributes are included in the Resource object. This grouping and reducing of common attributes is a powerful tool when sending logs as telemetry data.
The following sections define the attributes that are generally forwarded.
All log streams include the following log data fields
The "Applicable Sources" column shows which log sources this field applies to:
allis a field that is present on all logscontaineris a field that is present on Kubernetes Container logs (both application and infrastructure)auditis a field that is present on Kubernetes and OpenShift API and OVN Logsauditdis a field that is present on Node auditd logsjournalis a field that is present on Node journal logs
| Name | Applicable Sources | Comment |
|---|---|---|
body |
all | |
observedTimeUnixNano |
all | |
timeUnixNano |
all | |
severityText |
container, journal | |
attributes |
all | Optional. Present when forwarding stream specific attributes |
Log entries will have a set of resource, scope and log attributes depending on their source described by the following table.
The "Location" column can be one of the following:
resourcefor a resource attributescopefor a scope attributelogfor a log attribute
The "Storage" column shows whether the attribute is stored into a LokiStack using the default openshift-logging tenancy mode and where the attribute is stored:
stream labelstructured metadata
| Name | Location | Applicable Sources | Storage (LokiStack) | Comment |
|---|---|---|---|---|
log_source |
resource | all | stream label | (DEPRECATED) Compatibility attribute, contains same information as openshift.log.source |
log_type |
resource | all | stream label | (DEPRECATED) Compatibility attribute, contains same information as openshift.log.type |
kubernetes.container_name |
resource | container | stream label | (DEPRECATED) Compatibility attribute, contains same information as k8s.container.name |
kubernetes.host |
resource | all | stream label | (DEPRECATED) Compatibility attribute, same information as k8s.node.name |
kubernetes.namespace_name |
resource | container | stream label | (DEPRECATED) Compatibility attribute, contains same information as k8s.namespace.name |
kubernetes.pod_name |
resource | container | stream label | (DEPRECATED) Compatibility attribute, contains same information as k8s.pod.name |
openshift.cluster_id |
resource | all | stream label | (DEPRECATED) Compatibility attribute, contains same information as openshift.cluster.uid |
openshift.cluster.uid |
resource | all | stream label | |
openshift.log.source |
resource | all | stream label | |
openshift.log.type |
resource | all | stream label | |
openshift.labels.* |
resource | all | stream label | |
k8s.node.name |
resource | all | stream label | |
k8s.namespace.name |
resource | container | stream label | |
k8s.container.name |
resource | container | stream label | |
k8s.pod.labels.* |
resource | container | structured metadata | |
k8s.pod.name |
resource | container | stream label | |
k8s.pod.uid |
resource | container | stream label | |
k8s.cronjob.name |
resource | container | structured metadata | Conditionally forwarded based on creator of Pod |
k8s.daemonset.name |
resource | container | structured metadata | Conditionally forwarded based on creator of Pod |
k8s.deployment.name |
resource | container | structured metadata | Conditionally forwarded based on creator of Pod |
k8s.job.name |
resource | container | structured metadata | Conditionally forwarded based on creator of Pod |
k8s.replicaset.name |
resource | container | structured metadata | Conditionally forwarded based on creator of Pod |
k8s.statefulset.name |
resource | container | structured metadata | Conditionally forwarded based on creator of Pod |
log.iostream |
log | container | structured metadata | |
k8s.audit.event.level |
log | audit | structured metadata | |
k8s.audit.event.stage |
log | audit | structured metadata | |
k8s.audit.event.user_agent |
log | audit | structured metadata | |
k8s.audit.event.request.uri |
log | audit | structured metadata | |
k8s.audit.event.response.code |
log | audit | structured metadata | |
k8s.audit.event.annotation.* |
log | audit | structured metadata | |
k8s.audit.event.object_ref.resource |
log | audit | structured metadata | |
k8s.audit.event.object_ref.name |
log | audit | structured metadata | |
k8s.audit.event.object_ref.namespace |
log | audit | structured metadata | |
k8s.audit.event.object_ref.api_group |
log | audit | structured metadata | |
k8s.audit.event.object_ref.api_version |
log | audit | structured metadata | |
k8s.user.username |
log | audit | structured metadata | |
k8s.user.groups |
log | audit | structured metadata | |
process.executable.name |
resource | journal | structured metadata | |
process.executable.path |
resource | journal | structured metadata | |
process.command_line |
resource | journal | structured metadata | |
process.pid |
resource | journal | structured metadata | |
service.name |
resource | journal | structured metadata | |
systemd.t.* |
log | journal | structured metadata | |
systemd.u.* |
log | journal | structured metadata |
Note: Attributes marked as "Compatibility attribute" are added to support minimal backwards compatibility with the ViaQ data model. These attributes should be considered deprecated and will be removed one release after General Acceptance of Red Hat OpenShift Logging.
Note: Loki changes the attribute names when persisting them to storage. They will be lower-cased and all characters in the set: (.,/,-) will be replaced by underscores (_). For example, k8s.namespace.name will become k8s_namespace_name.