Skip to content

OpenMetrics content negotiation: escaping=underscores causes name mismatch between HELP/TYPE metadata and sample lines for metrics with colons #1177

@nabil-dbz

Description

@nabil-dbz

Description

When exposing metrics in the OpenMetrics format using content negotiation (via Accept: application/openmetrics-text; version=1.0.0), metrics containing colons (:) have their names escaped in the sample lines but remain unescaped in the # HELP and # TYPE lines.

This produces mismatched exposition output that violates the OpenMetrics standard, causing strict standard parsers (including promtool check metrics and downstream ingesters) to fail or treat them as completely separate metrics (an orphaned metadata set and an untyped metric).

Steps to Reproduce

  1. Create a simple Python server using prometheus_client:
from http.server import BaseHTTPRequestHandler, HTTPServer
from prometheus_client import CollectorRegistry, Gauge
from prometheus_client.exposition import choose_encoder
  
registry = CollectorRegistry()
token_usage = Gauge(
    name="sglang:token_usage",
    documentation="Total token usage.",
    registry=registry,
)
token_usage.set(42.0)
  
class MetricsHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        if self.path == "/metrics":
            accept_header = self.headers.get("Accept", "")
            encoder, content_type = choose_encoder(accept_header)
            data = encoder(registry)
            self.send_response(200)
            self.send_header("Content-Type", content_type)
            self.end_headers()
            self.wfile.write(data)
  
if __name__ == "__main__":
    server = HTTPServer(("0.0.0.0", 8000), MetricsHandler)
    server.serve_forever()
  1. Query the endpoint requesting OpenMetrics formatting:
curl -i -H "Accept: application/openmetrics-text; version=1.0.0" http://localhost:8000/metrics

Actual Output

Notice that the Content-Type is negotiated with escaping=underscores, but the comment lines mismatch the sample line:

HTTP/1.0 200 OK
Content-Type: application/openmetrics-text; version=1.0.0; charset=utf-8; escaping=underscores
  
# HELP sglang:token_usage Total token usage.
# TYPE sglang:token_usage gauge
sglang_token_usage 42.0
# EOF

Expected Output

The metric name inside the # HELP and # TYPE comment blocks should match the name used in the sample line:

# HELP sglang_token_usage Total token usage.
# TYPE sglang_token_usage gauge
sglang_token_usage 42.0
# EOF

Root Cause

In prometheus_client/openmetrics/exposition.py:

  • The comment generator uses the unescaped metric.name directly (since a colon is treated as legacy-valid under default validation checks).
  • The sample line generator, however, strictly sanitizes the sample's name against the narrower OpenMetrics name requirements, replacing the colon with an underscore under escaping=underscores.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions