Parse observeFullEvaluationData and hash targeting_key [FFL-2963] - #6178
Parse observeFullEvaluationData and hash targeting_key [FFL-2963]#6178vjfridge wants to merge 2 commits into
Conversation
…g_key [FFL-2963] Make the no-PII path the default for EVP flagevaluation events. The SDK hashes the targeting key with the sha256_ prefix and omits the evaluation context when the top-level UFC boolean observeFullEvaluationData is false (the default). When the value is true, the SDK emits the raw targeting key and the full context. The consent value travels on the event, not in live config. The evaluator reads the value from the UFC it evaluated against and stamps it onto the evaluation metadata. The hook and the aggregator read only this value. Changes: - native_evaluator: parse observeFullEvaluationData from the UFC root (sibling of environment). Absent, null, or wrong-typed values return false. Malformed JSON does not raise. - evaluation_engine: expose observe_full_evaluation_data. - provider: stamp consent onto evaluation metadata. - flag_eval_evp_hook: read consent from metadata. Skip the context copy on the hot path when consent is off. Capture the error code. - aggregator: put consent in the bucket key. Drop the context dimension from the key when consent is off. Do not store context_attrs when off. - writer: hash the targeting key and omit the context when consent is off. Redact the error message to the error code when consent is off. Add a pre-queue capacity check before the bounded copy. Add three telemetry signals: pre-queue overflow, context truncated, enqueue drop. The kill switch DD_FLAGGING_EVALUATION_COUNTS_ENABLED still wins over everything and emits nothing. DoLog has no effect on this track. Canonical cross-SDK vector: "jane.doe@datadoghq.com" -> sha256_b4698f9b6d186781fa8dc59e533578fa2d8379a46b1cf6db85cda6aa9c99e51b Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
|
👋 Hey @DataDog/ruby-guild, please fill "Change log entry" section in the pull request description. If changes need to be present in CHANGELOG.md you can state it this way **Change log entry**
Yes. A brief summary to be placed into the CHANGELOG.md(possible answers Yes/Yep/Yeah) Or you can opt out like that **Change log entry**
None.(possible answers No/Nope/None) Visited at: 2026-08-11 15:10:31 UTC |
Typing analysisNote: Ignored files are excluded from the next sections. Untyped methodsThis PR introduces 1 untyped method and 23 partially typed methods, and clears 23 partially typed methods. It increases the percentage of typed methods from 69.52% to 69.58% (+0.06%). Untyped methods (+1-0)❌ Introduced:Partially typed methods (+23-23)❌ Introduced:If you believe a method or an attribute is rightfully untyped or partially typed, you can add |
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: da353b2 | Docs | Datadog PR Page | Give us feedback! |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 008eb827dc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # Stamp the consent value from the engine that did the evaluation. The hook reads | ||
| # this value from metadata, not from live config. This is the authoritative stamp: | ||
| # the C extension result has no flag_metadata setter, so consent goes here. | ||
| flag_meta[Ext::METADATA_OBSERVE_FULL_EVALUATION_DATA] = engine.observe_full_evaluation_data |
There was a problem hiding this comment.
Bind consent to the evaluator result
If Remote Config changes from observeFullEvaluationData: false to true between fetch_engine_value and this separate accessor call, the result was evaluated under the old configuration but is stamped with the new consent; the hook can then emit its raw targeting key and context despite the configuration used for that evaluation forbidding it. Capture the evaluator and its consent atomically inside EvaluationEngine#fetch_value, and carry that snapshot with the result.
AGENTS.md reference: AGENTS.md:L112-L112
Useful? React with 👍 / 👎.
| full_key = [ | ||
| flag_key, variant, allocation_key, runtime_default, error_message, | ||
| targeting_key, false | ||
| ] |
There was a problem hiding this comment.
Key redacted errors by their emitted code
When consent is off, serialization replaces error_message with error_code, but this bucket key still uses the raw message. Evaluations for the same flag and targeting key with messages that differ only in embedded input therefore create separate buckets that serialize as duplicate, schema-identical rows, wasting the per-flag cap and fragmenting counts; conversely, equal messages with different codes merge and report the first code. Use the emitted error code as the consent-off aggregation dimension, including in the degraded key.
Useful? React with 👍 / 👎.
| snapshot[:degraded].each do |key, entry| | ||
| flag_key, variant, allocation_key, _runtime_default, _error_message = key | ||
| flag_key, variant, allocation_key, _runtime_default, _error_message, _consent = key | ||
| event = build_event( | ||
| flag_key: flag_key, variant: variant, allocation_key: allocation_key, | ||
| targeting_key: nil, entry: entry, flush_time_ms: flush_time_ms, tier: :degraded, | ||
| observe_full_evaluation_data: false, |
There was a problem hiding this comment.
Preserve consent and error codes after degradation
When a cardinality cap routes an error-bearing event to the degraded tier, add_to_degraded stores neither its consent nor its error code, and this call hardcodes consent to false. Consequently, degraded consent-on errors lose the raw error message, while degraded consent-off errors also lose the replacement error code, so both paths silently omit the error field that was emitted before this change. Carry the degraded key's consent and the event's error code into the entry and pass the consent through here.
AGENTS.md reference: lib/datadog/open_feature/AGENTS.md:L185-L187
Useful? React with 👍 / 👎.
| dropped_pre_queue = take_dropped_pre_queue_overflow | ||
| context_truncated = take_context_truncated_count | ||
|
|
||
| emit_drop_counts(dropped_queue, dropped_overflow) | ||
| emit_drop_counts(dropped_queue, dropped_overflow, dropped_pre_queue) | ||
| emit_context_truncated_counts(context_truncated) |
There was a problem hiding this comment.
Increment context-truncation telemetry
When pruning drops fields because there are more than 256 entries or a string exceeds 256 characters, nothing increments @context_truncated_count; repository-wide uses only initialize, read, and reset it. Thus this value is always zero and flagevaluation.context.truncated is never emitted (and max_field_length is never distinguishable), leaving the newly added observability path dead. Have the pruning step report both truncation reasons to the writer's counter.
AGENTS.md reference: lib/datadog/open_feature/AGENTS.md:L185-L187
Useful? React with 👍 / 👎.
| def hash_targeting_key(targeting_key) | ||
| TARGETING_KEY_HASH_PREFIX + Digest::SHA256.hexdigest(targeting_key) |
There was a problem hiding this comment.
Encode targeting keys as UTF-8 before hashing
When a targeting key uses a non-UTF-8 Ruby encoding, Digest::SHA256 hashes its current byte representation rather than the UTF-8 bytes promised by this cross-SDK format. For example, the logical key é encoded as ISO-8859-1 hashes byte e9, while other SDKs and a UTF-8 Ruby string hash bytes c3 a9, producing different identifiers for the same subject. Encode the string to UTF-8 before hashing and define how invalid byte sequences are handled.
Useful? React with 👍 / 👎.
RuboCop enforces trailing commas after the last item of a multiline array literal. Add the missing commas in the two full_key arrays. Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com>
Strech
left a comment
There was a problem hiding this comment.
I would like to request few changes before it goes into master
- Remove unnecessary AI generated comments explaining usage of something, it's not helpful and duplicates code
- Complete typespecs, some types are
untyped, even tho they are known and easy to type - Remove unnecessary comments in tests
- Remove unnecessary programming in tests
| # Literal prefix for the SHA-256 hash of the targeting key. The hash is | ||
| # unsalted SHA-256 over the raw UTF-8 bytes, lowercase hex, 71 chars total. | ||
| # This prefix is the cross-SDK contract: every SDK emits the same digest. |
There was a problem hiding this comment.
This comment is not relevant to the simple constant with sha256_ value (i.e AI slop)
| # Pre-queue capacity check: a full queue is an O(1) drop + counter, not | ||
| # a copy-then-discard. Do the bounded context copy only when the queue | ||
| # has room. |
There was a problem hiding this comment.
3 lines of comments vs 3 lines of same-logic code (i.e AI slop)
| # Read-and-reset the pre-queue overflow counter. This counter fires when | ||
| # the queue is full before the bounded context copy, so a full queue is | ||
| # an O(1) drop, not a copy-then-discard. |
There was a problem hiding this comment.
This comment explain the usage, when should explain the method (i.e AI slop)
| # Read-and-reset the pre-queue overflow counter. This counter fires when | ||
| # the queue is full before the bounded context copy, so a full queue is | ||
| # an O(1) drop, not a copy-then-discard. | ||
| def take_dropped_pre_queue_overflow |
There was a problem hiding this comment.
Ruby-style is to avoid take/get/set prefixes and use human-form
def read_and_reset_dropped_pre_queue_overflow|
|
||
| # Emit the context-truncated count with a reason label so the operator can | ||
| # tell which cap was hit. | ||
| def emit_context_truncated_counts(context_truncated) |
There was a problem hiding this comment.
The method name doesn't correspond the method body
| ?runtime_default: bool? | ||
| ?runtime_default: bool?, | ||
| ?observe_full_evaluation_data: bool, | ||
| ?error_code: untyped |
| it "returns false for a nil configuration" do | ||
| evaluator = described_class.new(nil) | ||
| expect(evaluator.observe_full_evaluation_data).to be(false) | ||
| end |
There was a problem hiding this comment.
| it "returns false for a nil configuration" do | |
| evaluator = described_class.new(nil) | |
| expect(evaluator.observe_full_evaluation_data).to be(false) | |
| end | |
| it { expect(described_class.new(nil).observe_full_evaluation_data).to be(false) } |
| def ufc(observe: :absent) | ||
| base = "{\"format\":\"SERVER\",\"environment\":{\"name\":\"test\"},\"flags\":{}}" | ||
| return base if observe == :absent | ||
|
|
||
| value = (observe == true) ? "true" : "false" | ||
| extra = "\"observeFullEvaluationData\":#{value}" | ||
| base.sub('{"format"', "{#{extra},\"format\"") | ||
| end |
There was a problem hiding this comment.
This could be done via 2 distinct let statements with 0 programming
Summary
This change makes the no-PII path the default for EVP flagevaluation events in the Ruby SDK.
When the top-level UFC boolean
observeFullEvaluationDataisfalse(the default), the SDK:targeting_keywith thesha256_prefix (unsalted SHA-256, 71 chars).context.evaluationkey entirely (absent, notnil, not{}).error.messageto theerror_codeso it cannot carry raw context data.When
observeFullEvaluationDataistrue, the SDK emits the rawtargeting_key, the fullcontext.evaluation, and the rawerror.message.The kill switch
DD_FLAGGING_EVALUATION_COUNTS_ENABLEDstill wins over everything and emits nothing.DoLoghas no effect on this track.Contract
This work does the Ruby fan-out of the PII-hashing contract (FFL-2780). The pilot reference is
dd-trace-java#12042(FFL-2790). The contract of record is FFL-2784.Canonical cross-SDK vector (every SDK must produce this exact digest):
Consent lifecycle
The consent value travels on the event, not in live config. No component downstream of the evaluator reads live config. This prevents a later Remote Config update from applying a different environment consent.
observeFullEvaluationDatafrom the UFC root (a sibling ofenvironment, not a field on it).observe_full_evaluation_data.Ruby has no global config accessor in this path, so the consent-lifecycle race that the Java pilot fixed does not occur here.
Changes
native_evaluator.rbobserveFullEvaluationDatafrom the UFC JSON root. Absent,null, or wrong-typed values returnfalse. Malformed JSON does not raise.evaluation_engine.rbobserve_full_evaluation_data.provider.rbflag_eval_evp_hook.rbaggregator.rbcontext_attrswhen off.writer.rbtargeting_keyand omit the context when consent is off. Redact theerror.messageto theerror_codewhen off. Add a pre-queue capacity check before the bounded copy. Add three telemetry signals.*.rbsTelemetry signals
The writer now emits three telemetry signals:
offer()time (a race; this is rare when the pre-queue check is in place).Tests
All tests assert on raw wire bytes: the hash is present AND the raw subject string appears nowhere in the payload.
false/true/null/ wrong-typed: the consent value isfalsefor all buttrue.targeting_keyis hashed, thecontextis absent, the raw subject string appears nowhere.targeting_keyand the fullcontext.evaluationare emitted.error.messageredaction: the error message carries only the error code when consent is off; the raw context string appears nowhere.DoLognon-impact: the emitted shape is the same for allDoLogvalues.bundle exec rake standard typecheckis clean.bundle exec rake test:open_featureis 256 examples, 0 failures.L3 caveat
system-tests/manifests/ruby.ymlstill gates the whole oftests/ffe/test_flag_eval_evp.pyasmissing_feature (FFL-2446)even though the base track merged on 2026-06-30. The PII system-tests do not run until that file-level gate is addressed. This PR does not change the manifest. The L3 tests must be activated in a separate change (FFL-2783) before this PR can use them as a contract test.Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com