[Bug]: strimzi operator doesn't expose content-type in /metrics endpoint

[KyriosGN0]

Bug Description

after upgrading to Prometheus v3, we noticed it started to alert us on TargetDown for strimzi operator pods
the error from the logs are non-compliant scrape target sending blank Content-Type and no fallback_scrape_protocol specified for target

Steps to reproduce

1. deploy strimzi operator
2. deploy prometheus v3
3. create pod/service monitor and try to scrape the /metrics endpoint

Expected behavior

prometheus v3 should scrape without any issues

Strimzi version

0.44.0

Kubernetes version

1.30.5

Installation method

helm chart

Infrastructure

GKE

Configuration files and logs

apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
  name: cluster-operator-metrics
  labels:
    app: strimzi
spec:
  namespaceSelector:
    matchNames:
    - {{ .Release.Namespace }}
  podMetricsEndpoints:
  - path: /metrics
    port: http
  selector:
    matchLabels:
      name: strimzi-cluster-operator

podmonitor config we used

Additional context

No response

[scholzj]

So, what is the correct content type?

[scholzj]

The Prometheus JMX Exporter in the Kafka brokers uses Content-type: text/plain; version=0.0.4; charset=utf-8 -> so is text/plain ok for Prometheus 3?

[KyriosGN0]

from what i see in curl

the endpoint doesn't return any
from the docs it says that the accept values are

# The protocols to negotiate during a scrape with the client.
# Supported values (case sensitive): PrometheusProto, OpenMetricsText0.0.1,
# OpenMetricsText1.0.0, PrometheusText0.0.4, PrometheusText1.0.0.
[ scrape_protocols: [[<string>](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#string), ...] | default = <global_config.scrape_protocols> ]

[scholzj]

the endpoint doesn't return any

Because that is the operator. It is different for the other components such as the Kafka pods.

from the docs it says that the accept values are

# The protocols to negotiate during a scrape with the client.
# Supported values (case sensitive): PrometheusProto, OpenMetricsText0.0.1,
# OpenMetricsText1.0.0, PrometheusText0.0.4, PrometheusText1.0.0.
[ scrape_protocols: [[<string>](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#string), ...] | default = <global_config.scrape_protocols> ]

Not sure how the protocols translate to content types. But I guess we can assume text/plain is ok if Prometheus's own client uses it maybe?

Update: Seems to be mentioned here as well: https://prometheus.io/docs/instrumenting/exposition_formats/#basic-info

[KyriosGN0]

then yeah text/plain; version=0.0.4 should be good

[scholzj]

Ok, I opened a PR for it. But I do not have any Prometheus 3 installation. So I cannot really test it with that. I only verified that my old Prometheus still works and that the Content-Type is there when trying with curl.

[KyriosGN0]

@scholzj could i build the operator docker image from your PR and then run it with my prom v3 ?