PO to GMP Migration Tool: Add Draft Generation with TODO Annotations for Errors - #2061
PO to GMP Migration Tool: Add Draft Generation with TODO Annotations for Errors#2061karthunni wants to merge 7 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a mechanism to inject sequential TODO annotations and safety guardrails (gmp.googleapis.com/migration-review-required: "true") into migrated resources when configuration issues are encountered, and updates the migration report to track these action items. The review feedback highlights several violations of migration rules where the code falls back to default values or placeholders (such as for invalid proxy URLs, invalid scrape intervals, or unresolvable Service ports) instead of returning a fatal error. Malformed configurations that would lead to failed scrapes must result in errors rather than falling back to defaults or logging warnings.
bwplotka
left a comment
There was a problem hiding this comment.
Looks like a reasonable approach if you can make it a pattern for all errors/warnings (e.g. we don't forget about something).
Not sure about blocking scrape logic - it feels odd, but it is one of the least bad options if we assume people or agents will apply without looking
a88cc48 to
460df7d
Compare
| portMap[k] = intstr.FromString("TODO_RESOLVE_PORT") | ||
| } | ||
| dummySvc := &corev1.Service{ | ||
| ObjectMeta: metav1.ObjectMeta{ |
There was a problem hiding this comment.
When backing Services are missing, using sm.Namespace for dummySvc ignores targetNamespaces (e.g. from namespaceSelector.matchNames), causing draft PodMonitoring manifests to be generated in sm.Namespace instead of the targeted namespaces. Consider creating dummy Services for each namespace in targetNamespaces.
| return true | ||
| if strings.HasPrefix(s, "__meta_kubernetes_node_") && s != "__meta_kubernetes_node_name" { | ||
| logger.Warn(fmt.Sprintf("Relabeling rule referencing node metadata %q is unsupported in GMP (only node name is supported). The rule has been dropped.", s)) | ||
| return true, false, "" |
There was a problem hiding this comment.
When dropping a keep or drop rule on node metadata, scopeExpanded is returned as false. Consider returning scopeExpanded: true if action == relabel.Keep || action == relabel.Drop (like lines 926-929 for pod annotations) so a TODO and guardrail label are attached.
| fmt.Fprintln(m.Stderr, "\nNOTE: Some resources were migrated with action items and contain TODO annotations.") | ||
| fmt.Fprintln(m.Stderr, "These resources include the safety guardrail label:") | ||
| fmt.Fprintln(m.Stderr, " 'gmp.googleapis.com/migration-review-required: \"true\"'") | ||
| fmt.Fprintln(m.Stderr, "Review the TODO annotations in the generated manifests and remove this label when ready to activate scraping.") |
There was a problem hiding this comment.
Consider using the GuardrailLabelKey and GuardrailLabelValue constants here instead of hardcoded strings:
| fmt.Fprintln(m.Stderr, "Review the TODO annotations in the generated manifests and remove this label when ready to activate scraping.") | |
| fmt.Fprintf(m.Stderr, " '%s: \"%s\"'\n", GuardrailLabelKey, GuardrailLabelValue) |
| }, | ||
| }, | ||
| wantErr: true, | ||
| expected: nil, |
There was a problem hiding this comment.
Note that TestServiceMonitorConverter_Convert currently guards its output length assertion with if tc.expected != nil, which causes tests with expected: nil to skip verifying that outputs is actually empty.
| if parsed.User != nil { | ||
| if _, hasPass := parsed.User.Password(); hasPass { | ||
| c.todos = append(c.todos, todoItem{ | ||
| category: "ERROR", | ||
| reason: "Proxy URL contains embedded plaintext credentials. Credentials were removed.", | ||
| action: "Configure proxy authentication via Kubernetes Secret or proxy server configuration.", | ||
| }) | ||
| parsed.User = nil | ||
| return parsed.String() | ||
| } |
There was a problem hiding this comment.
If the proxy URL contains a username without a password (such as http://user@host), hasPass is false so credentials aren't stripped and the resulting URL still contains @ which is rejected by GMP. Consider checking if parsed.User != nil directly without requiring a password:
| if parsed.User != nil { | |
| if _, hasPass := parsed.User.Password(); hasPass { | |
| c.todos = append(c.todos, todoItem{ | |
| category: "ERROR", | |
| reason: "Proxy URL contains embedded plaintext credentials. Credentials were removed.", | |
| action: "Configure proxy authentication via Kubernetes Secret or proxy server configuration.", | |
| }) | |
| parsed.User = nil | |
| return parsed.String() | |
| } | |
| c.todos = append(c.todos, todoItem{ | |
| category: "ERROR", | |
| reason: "Proxy URL contains embedded plaintext credentials. Credentials were removed.", | |
| action: "Configure proxy authentication via Kubernetes Secret or proxy server configuration.", | |
| }) | |
| parsed.User = nil | |
| return parsed.String() |
Reconsidered potential options and decided to take the following approach: On a default CLI call (i.e no flags), we output the manifests without TODO items, while noting in the stderr output summary: (such that users can safely blindly apply) When the |
This PR enhances the Prometheus Operator to GMP migration tool by implementing graceful draft generation for conversion errors instead of aborting the entire migration. The converter produces manifests with TODO annotations, placeholder values, and a safety guardrail label.
TODO_MISSING_KEY_<KEY>_IN_<KIND>_<NAME>andTODO_SET_SECRET_NAMEplaceholders with[ERROR]TODO annotations instead of returning hard errors."30s"with[ERROR]TODOs to preserve schema validity.port: "TODO_SET_PORT"when an endpoint omits bothportandtargetPort.port: "TODO_RESOLVE_PORT_<PORT>"with[WARNING]TODOs.ServiceMonitor:ServiceMonitorhas no matching backingServicein the inputs, generates a draftPodMonitoringwithselector: {app: "TODO_SET_POD_SELECTOR"}andport: "TODO_RESOLVE_PORT".gmp.googleapis.com/migration-review-required: "true"intospec.selector.matchLabelswhenever[ERROR]TODO items are present, preventing unintentional scraping until user review.