Skip to content

PO to GMP Migration Tool: Add Draft Generation with TODO Annotations for Errors - #2061

Draft
karthunni wants to merge 7 commits into
mainfrom
karthunni/po-migrate-guardrails-todos
Draft

PO to GMP Migration Tool: Add Draft Generation with TODO Annotations for Errors#2061
karthunni wants to merge 7 commits into
mainfrom
karthunni/po-migrate-guardrails-todos

Conversation

@karthunni

Copy link
Copy Markdown
Collaborator

This PR enhances the Prometheus Operator to GMP migration tool by implementing graceful draft generation for conversion errors instead of aborting the entire migration. The converter produces manifests with TODO annotations, placeholder values, and a safety guardrail label.

  • Missing Secret / ConfigMap Keys & References:
    • Emits TODO_MISSING_KEY_<KEY>_IN_<KIND>_<NAME> and TODO_SET_SECRET_NAME placeholders with [ERROR] TODO annotations instead of returning hard errors.
  • Duration Parsing & Scrape Timeouts:
    • Defaults invalid scrape intervals to "30s" with [ERROR] TODOs to preserve schema validity.
    • Drops invalid scrape timeouts and caps timeouts exceeding scrape intervals with actionable TODOs.
  • Endpoint & Service Port Resolution:
    • Emits port: "TODO_SET_PORT" when an endpoint omits both port and targetPort.
    • Resolves unmapped Service ports to port: "TODO_RESOLVE_PORT_<PORT>" with [WARNING] TODOs.
  • Missing Backing Services for ServiceMonitor:
    • When a ServiceMonitor has no matching backing Service in the inputs, generates a draft PodMonitoring with selector: {app: "TODO_SET_POD_SELECTOR"} and port: "TODO_RESOLVE_PORT".
  • Selector & Keep Rule Conflicts:
    • Preserves base selectors and records actionable TODOs when relabel keep rules conflict with existing match labels.
  • Safety Guardrail Label:
    • Injects gmp.googleapis.com/migration-review-required: "true" into spec.selector.matchLabels whenever [ERROR] TODO items are present, preventing unintentional scraping until user review.

@karthunni karthunni self-assigned this Aug 7, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a mechanism to inject sequential TODO annotations and safety guardrails (gmp.googleapis.com/migration-review-required: "true") into migrated resources when configuration issues are encountered, and updates the migration report to track these action items. The review feedback highlights several violations of migration rules where the code falls back to default values or placeholders (such as for invalid proxy URLs, invalid scrape intervals, or unresolvable Service ports) instead of returning a fatal error. Malformed configurations that would lead to failed scrapes must result in errors rather than falling back to defaults or logging warnings.

Comment thread pkg/migrate/helpers.go Outdated
Comment thread pkg/migrate/helpers.go
Comment thread pkg/migrate/servicemonitor.go
Comment thread pkg/migrate/podmonitor.go
Comment thread pkg/migrate/servicemonitor.go
@karthunni
karthunni requested a review from dashpole August 7, 2026 21:21

@bwplotka bwplotka left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a reasonable approach if you can make it a pattern for all errors/warnings (e.g. we don't forget about something).

Not sure about blocking scrape logic - it feels odd, but it is one of the least bad options if we assume people or agents will apply without looking

@karthunni
karthunni force-pushed the karthunni/po-migrate-guardrails-todos branch from a88cc48 to 460df7d Compare August 10, 2026 15:38
portMap[k] = intstr.FromString("TODO_RESOLVE_PORT")
}
dummySvc := &corev1.Service{
ObjectMeta: metav1.ObjectMeta{

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When backing Services are missing, using sm.Namespace for dummySvc ignores targetNamespaces (e.g. from namespaceSelector.matchNames), causing draft PodMonitoring manifests to be generated in sm.Namespace instead of the targeted namespaces. Consider creating dummy Services for each namespace in targetNamespaces.

Comment thread pkg/migrate/helpers.go
return true
if strings.HasPrefix(s, "__meta_kubernetes_node_") && s != "__meta_kubernetes_node_name" {
logger.Warn(fmt.Sprintf("Relabeling rule referencing node metadata %q is unsupported in GMP (only node name is supported). The rule has been dropped.", s))
return true, false, ""

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When dropping a keep or drop rule on node metadata, scopeExpanded is returned as false. Consider returning scopeExpanded: true if action == relabel.Keep || action == relabel.Drop (like lines 926-929 for pod annotations) so a TODO and guardrail label are attached.

Comment thread pkg/migrate/migrate.go
fmt.Fprintln(m.Stderr, "\nNOTE: Some resources were migrated with action items and contain TODO annotations.")
fmt.Fprintln(m.Stderr, "These resources include the safety guardrail label:")
fmt.Fprintln(m.Stderr, " 'gmp.googleapis.com/migration-review-required: \"true\"'")
fmt.Fprintln(m.Stderr, "Review the TODO annotations in the generated manifests and remove this label when ready to activate scraping.")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using the GuardrailLabelKey and GuardrailLabelValue constants here instead of hardcoded strings:

Suggested change
fmt.Fprintln(m.Stderr, "Review the TODO annotations in the generated manifests and remove this label when ready to activate scraping.")
fmt.Fprintf(m.Stderr, " '%s: \"%s\"'\n", GuardrailLabelKey, GuardrailLabelValue)

},
},
wantErr: true,
expected: nil,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that TestServiceMonitorConverter_Convert currently guards its output length assertion with if tc.expected != nil, which causes tests with expected: nil to skip verifying that outputs is actually empty.

Comment thread pkg/migrate/helpers.go
Comment on lines +1427 to +1436
if parsed.User != nil {
if _, hasPass := parsed.User.Password(); hasPass {
c.todos = append(c.todos, todoItem{
category: "ERROR",
reason: "Proxy URL contains embedded plaintext credentials. Credentials were removed.",
action: "Configure proxy authentication via Kubernetes Secret or proxy server configuration.",
})
parsed.User = nil
return parsed.String()
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the proxy URL contains a username without a password (such as http://user@host), hasPass is false so credentials aren't stripped and the resulting URL still contains @ which is rejected by GMP. Consider checking if parsed.User != nil directly without requiring a password:

Suggested change
if parsed.User != nil {
if _, hasPass := parsed.User.Password(); hasPass {
c.todos = append(c.todos, todoItem{
category: "ERROR",
reason: "Proxy URL contains embedded plaintext credentials. Credentials were removed.",
action: "Configure proxy authentication via Kubernetes Secret or proxy server configuration.",
})
parsed.User = nil
return parsed.String()
}
c.todos = append(c.todos, todoItem{
category: "ERROR",
reason: "Proxy URL contains embedded plaintext credentials. Credentials were removed.",
action: "Configure proxy authentication via Kubernetes Secret or proxy server configuration.",
})
parsed.User = nil
return parsed.String()

@karthunni

Copy link
Copy Markdown
Collaborator Author

Looks like a reasonable approach if you can make it a pattern for all errors/warnings (e.g. we don't forget about something).

Not sure about blocking scrape logic - it feels odd, but it is one of the least bad options if we assume people or agents will apply without looking

Reconsidered potential options and decided to take the following approach:

On a default CLI call (i.e no flags), we output the manifests without TODO items, while noting in the stderr output summary: (such that users can safely blindly apply)

X manifests with action items were omitted from Stdout as they contain best-effort draft configurations with TODO annotations and placeholders.
Run with '--all' to output all manifests for review.

When the --all flag is added, all manifests are written to stdout including the ones with TODOs (still in the annotations but no longer blocking scraping logic with the matchLabels), and the output summary states: (such that an explicit call is needed to output "dangerous" CRs)

X manifests contain best-effort draft configurations with TODO annotations and placeholders.
Review the inline 'gmp.googleapis.com/todo-*' annotations in the generated manifests before applying to a cluster.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants