feat(analyzer): add recognizer-level threshold config

CHANGELOG.md

@@ -39,6 +39,7 @@ All notable changes to this project will be documented in this file.
 
 ### Analyzer
 #### Added
+- Recognizer registry YAML entries now accept `score_thresholds` for recognizer-wide and entity-specific score cutoffs, with the analyzer's `default_score_threshold` as the fallback.
 - UK Driving Licence Number (`UK_DRIVING_LICENCE`) recognizer with pattern matching and context support (#1857) (Thanks @tee-jagz)
 - German PII recognizers for `DE_TAX_ID`, `DE_TAX_NUMBER`, `DE_PASSPORT`, `DE_ID_CARD`, `DE_SOCIAL_SECURITY`, `DE_HEALTH_INSURANCE`, `DE_KFZ`, `DE_HANDELSREGISTER`, and `DE_PLZ`; all are disabled by default (#1909) (Thanks @MvdB)
 - `SE_PERSONNUMMER` recognizer for Swedish personal identity and coordination numbers, plus Swedish Organisationsnummer recognition; both are disabled by default (#1912, #1918) (Thanks @goveebee)

docs/analyzer/analyzer_engine_provider.md

@@ -63,17 +63,22 @@ recognizer_registry:
       - en
     supported_entity: IT_FISCAL_CODE
     type: predefined
+    score_thresholds:
+      default: 0.4
+      CREDIT_CARD: 0.7
 
   - name: ItFiscalCodeRecognizer
     type: predefined
 ```
 
 The configuration file contains the following parameters:
 
-  - `supported_languages`: A list of supported languages that the analyzer will support.
-  - `default_score_threshold`: A score that determines the minimal threshold for detection.
-  - `nlp_configuration`: Configuration given to the NLP engine which will detect the PIIs and extract features for the downstream logic.
-  - `recognizer_registry`: All the recognizers that will be used by the analyzer. 
+- `supported_languages`: A list of supported languages that the analyzer will support.
+- `default_score_threshold`: A score that determines the minimal threshold for detection.
+- `nlp_configuration`: Configuration given to the NLP engine which will detect the PIIs and extract features for the downstream logic.
+- `recognizer_registry`: All the recognizers that will be used by the analyzer. Each recognizer entry can define `score_thresholds`, using `default` as its fallback and entity names for overrides.
+
+Supplying `analyze(score_threshold=...)` bypasses recognizer-level thresholds for that request and applies the supplied threshold to every result. When omitted, precedence is an entity-specific recognizer threshold, the recognizer's `default`, then `default_score_threshold`.
 
 !!! note "Note"
 

docs/analyzer/recognizer_registry_provider.md

@@ -56,6 +56,9 @@ The recognizer list comprises of both the predefined and custom recognizers, for
     - language: it
     - language: pl
     type: predefined
+    score_thresholds:
+      default: 0.4
+      CREDIT_CARD: 0.7
 
   - name: UsBankRecognizer
     supported_languages: 
@@ -107,6 +110,7 @@ The recognizer list comprises of both the predefined and custom recognizers, for
   - `supported_entity`: the detected entity associated by the recognizer.
   - `deny_list`: A list of words to detect, in case the recognizer uses a predefined list of words.
   - `deny_list_score`: confidence score for a term identified using a deny-list.
+  - `score_thresholds`: optional score thresholds for this recognizer. Use `default` as the recognizer-wide threshold and entity names for overrides. Supplying `analyze(score_threshold=...)` bypasses recognizer-level thresholds for that request. When omitted, precedence is an entity override, this recognizer default, then the analyzer's `default_score_threshold`.
 
 !!! tip "Configuration Tip: Agglutinative languages (e.g., Korean)"
 

docs/tutorial/08_no_code.md

@@ -63,6 +63,9 @@ recognizer_registry:
     - language: es
       context: [tarjeta, credito, visa, mastercard, cc, amex, discover, jcb, diners, maestro, instapayment]
     type: predefined
+    score_thresholds:
+      default: 0.4
+      CREDIT_CARD: 0.7
 
   - name: DateRecognizer
     supported_languages:
@@ -119,6 +122,8 @@ recognizer_registry:
 """
 ```
 
+Each recognizer can set a `default` threshold and entity-specific overrides in `score_thresholds`. Supplying `analyze(score_threshold=...)` bypasses recognizer-level thresholds for that request. When omitted, precedence is an entity override, the recognizer default, then the analyzer's `default_score_threshold`.
+
 ### NLP Engine parameters
 
 ([default file](https://github.com/data-privacy-stack/presidio/blob/main/presidio-analyzer/presidio_analyzer/conf/default.yaml))

presidio-analyzer/presidio_analyzer/analyzer_engine.py

@@ -2,7 +2,7 @@
 import logging
 import os
 from collections import Counter
-from typing import List, Optional
+from typing import Dict, List, Optional
 
 import regex as re
 
@@ -25,6 +25,7 @@
 
 REGEX_TIMEOUT_SECONDS = int(os.environ.get("REGEX_TIMEOUT_SECONDS", 60))
 
+
 class AnalyzerEngine:
     """
     Entry point for Presidio Analyzer.
@@ -170,8 +171,10 @@ def analyze(
         :param entities: List of PII entities that should be looked for in the text.
         If entities=None then all entities are looked for.
         :param correlation_id: cross call ID for this request
-        :param score_threshold: A minimum value for which
-        to return an identified entity
+        :param score_threshold: An explicit threshold for every result. When
+        supplied, this bypasses recognizer-level score thresholds for this
+        request. When omitted, the analyzer uses an entity-specific recognizer
+        threshold, the recognizer's default threshold, then the engine default.
         :param return_decision_process: Whether the analysis decision process steps
         returned in the response.
         :param ad_hoc_recognizers: List of recognizers which will be used only
@@ -254,9 +257,10 @@ def analyze(
                 json.dumps([str(result.to_dict()) for result in results]),
             )
 
-        # Remove duplicates or low score results
+        # Filter low-score results before deduplication so recognizer-specific
+        # thresholds do not get lost when duplicate spans collapse.
+        results = self.__remove_low_scores(results, score_threshold, recognizers)
         results = EntityRecognizer.remove_duplicates(results)
-        results = self.__remove_low_scores(results, score_threshold)
 
         if allow_list:
             results = self._remove_allow_list(
@@ -330,21 +334,67 @@ def _enhance_using_context(
         return results
 
     def __remove_low_scores(
-        self, results: List[RecognizerResult], score_threshold: float = None
+        self,
+        results: List[RecognizerResult],
+        score_threshold: Optional[float] = None,
+        recognizers: Optional[List[EntityRecognizer]] = None,
     ) -> List[RecognizerResult]:
         """
         Remove results for which the confidence is lower than the threshold.
 
         :param results: List of RecognizerResult
         :param score_threshold: float value for minimum possible confidence
+        :param recognizers: recognizers selected for this request
         :return: List[RecognizerResult]
         """
         if score_threshold is None:
-            score_threshold = self.default_score_threshold
+            recognizers_by_id: Dict[str, Dict[str, float]] = {
+                recognizer.id: recognizer.score_thresholds
+                for recognizer in recognizers or []
+            }
+            return [
+                result
+                for result in results
+                if result.score
+                >= self.__get_result_score_threshold(result, recognizers_by_id)
+            ]
 
         new_results = [result for result in results if result.score >= score_threshold]
         return new_results
 
+    def __get_result_score_threshold(
+        self,
+        result: RecognizerResult,
+        recognizers_by_id: Dict[str, Dict[str, float]],
+    ) -> float:
+        """Resolve the threshold to apply for a single recognizer result."""
+        metadata = result.recognition_metadata or {}
+        recognizer_id = metadata.get(RecognizerResult.RECOGNIZER_IDENTIFIER_KEY)
+        if recognizer_id is None:
+            logger.debug(
+                "Recognizer identifier is missing; using the engine default "
+                "score threshold"
+            )
+            return self.default_score_threshold
+        recognizer_thresholds = recognizers_by_id.get(recognizer_id)
+        if recognizer_thresholds is None:
+            logger.debug(
+                "Recognizer identifier %s did not match a selected recognizer; "
+                "using the engine default score threshold",
+                recognizer_id,
+            )
+            return self.default_score_threshold
+
+        entity_threshold = recognizer_thresholds.get(result.entity_type)
+        if entity_threshold is not None:
+            return entity_threshold
+
+        recognizer_default_threshold = recognizer_thresholds.get("default")
+        if recognizer_default_threshold is not None:
+            return recognizer_default_threshold
+
+        return self.default_score_threshold
+
     @staticmethod
     def _remove_allow_list(
         results: List[RecognizerResult],

presidio-analyzer/presidio_analyzer/conf/default_analyzer_full.yaml

@@ -57,6 +57,9 @@ recognizer_registry:
       - language: en
         context: [credit, card, visa, mastercard, cc, amex, discover, jcb, diners, maestro, instapayment]
       type: predefined
+      # score_thresholds:
+      #   default: 0.4
+      #   CREDIT_CARD: 0.7
 
     - name: UsBankRecognizer
       type: predefined

presidio-analyzer/presidio_analyzer/conf/default_recognizers.yaml

@@ -36,6 +36,9 @@ recognizers:
     - language: it
     - language: pl
     type: predefined
+    # score_thresholds:
+    #   default: 0.4
+    #   CREDIT_CARD: 0.7
 
   - name: UsBankRecognizer
     supported_languages: 

presidio-analyzer/presidio_analyzer/entity_recognizer.py

@@ -3,6 +3,7 @@
 from typing import TYPE_CHECKING, ClassVar, Dict, List, Optional, Tuple
 
 from presidio_analyzer import RecognizerResult
+from presidio_analyzer.score_thresholds import normalize_score_thresholds
 
 if TYPE_CHECKING:
     from presidio_analyzer.nlp_engine import NlpArtifacts
@@ -33,6 +34,7 @@ class EntityRecognizer:
         recognizers may set it per instance; predefined recognizers should
         prefer the class-level :attr:`COUNTRY_CODE`. Values are stripped,
         lower-cased, and must match :attr:`COUNTRY_CODE` when both are set.
+    :param score_thresholds: Optional default and entity-specific score thresholds.
     """
 
     MIN_SCORE = 0
@@ -53,6 +55,7 @@ def __init__(
         version: str = "0.0.1",
         context: Optional[List[str]] = None,
         country_code: Optional[str] = None,
+        score_thresholds: Optional[Dict[str, float]] = None,
     ):
         self.supported_entities = supported_entities
 
@@ -67,13 +70,27 @@ def __init__(
         self.version = version
         self.is_loaded = False
         self.context = context if context else []
+        self.score_thresholds = score_thresholds
 
         self._country_code = self._resolve_country_code(country_code)
 
         self.load()
         logger.info("Loaded recognizer: %s", self.name)
         self.is_loaded = True
 
+    @property
+    def score_thresholds(self) -> Dict[str, float]:
+        """Return a defensive copy of this recognizer's score thresholds."""
+        return self._score_thresholds.copy()
+
+    @score_thresholds.setter
+    def score_thresholds(self, value: Optional[Dict[str, float]]) -> None:
+        """Validate and store this recognizer's score thresholds.
+
+        :param value: The default and entity-specific score thresholds.
+        """
+        self._score_thresholds = normalize_score_thresholds(value)
+
     @classmethod
     def _resolve_country_code(cls, passed: Optional[str]) -> Optional[str]:
         """Reconcile a constructor-passed country code with the class attribute.

presidio-analyzer/presidio_analyzer/input_validation/schemas.py

@@ -3,6 +3,11 @@
 
 from pydantic import ValidationError
 
+from presidio_analyzer.score_thresholds import (
+    normalize_score_thresholds,
+    validate_score_threshold,
+)
+
 from . import validate_language_codes
 from .yaml_recognizer_models import RecognizerRegistryConfig
 
@@ -38,11 +43,7 @@ def validate_score_threshold(threshold: float) -> float:
 
         :param threshold: score threshold to validate.
         """
-        if not 0.0 <= threshold <= 1.0:
-            raise ValueError(
-                f"Score threshold must be between 0.0 and 1.0, got: {threshold}"
-            )
-        return threshold
+        return validate_score_threshold(threshold)
 
     @staticmethod
     def validate_nlp_configuration(config: Dict[str, Any]) -> Dict[str, Any]:
@@ -80,9 +81,15 @@ def validate_recognizer_registry_configuration(
         try:
             # Use Pydantic model for validation
             validated_config = RecognizerRegistryConfig(**config)
-            return ConfigurationValidator._dump_recognizer_registry_configuration(
-                validated_config
+            dumped_config = (
+                ConfigurationValidator._dump_recognizer_registry_configuration(
+                    validated_config
+                )
             )
+            for recognizer in dumped_config["recognizers"]:
+                if isinstance(recognizer, dict):
+                    normalize_score_thresholds(recognizer.get("score_thresholds"))
+            return dumped_config
         except ValidationError as e:
             raise ValueError("Invalid recognizer registry configuration") from e
 

presidio-analyzer/presidio_analyzer/input_validation/yaml_recognizer_models.py

@@ -81,6 +81,10 @@ class BaseRecognizerConfig(BaseModel):
     supported_entities: Optional[List[str]] = Field(
         default=None, description="List of supported entities for this recognizer"
     )
+    score_thresholds: Optional[Any] = Field(
+        default=None,
+        description="Default and entity-specific score thresholds",
+    )
 
     @field_validator("supported_language")
     @classmethod
@@ -463,6 +467,7 @@ def parse_recognizers(
                 continue
 
             if isinstance(recognizer, dict):
+                recognizer = recognizer.copy()
                 recognizer_type = recognizer.get("type")
 
                 # Validate conflicting custom-only fields if explicitly predefined

presidio-analyzer/presidio_analyzer/recognizer_registry/recognizer_registry.py

@@ -22,6 +22,7 @@
     RecognizerConfigurationLoader,
     RecognizerListLoader,
 )
+from presidio_analyzer.score_thresholds import normalize_score_thresholds
 
 logger = logging.getLogger("presidio-analyzer")
 
@@ -314,7 +315,12 @@ def add_pattern_recognizer_from_dict(self, recognizer_dict: Dict) -> None:
         >>> registry.add_pattern_recognizer_from_dict(recognizer)
         """  # noqa: E501
 
-        recognizer = PatternRecognizer.from_dict(recognizer_dict)
+        recognizer_config = recognizer_dict.copy()
+        score_thresholds = normalize_score_thresholds(
+            recognizer_config.pop("score_thresholds", None)
+        )
+        recognizer = PatternRecognizer.from_dict(recognizer_config)
+        recognizer.score_thresholds = score_thresholds
         self.add_recognizer(recognizer)
 
     def add_recognizers_from_yaml(self, yml_path: Union[str, Path]) -> None:

presidio-analyzer/presidio_analyzer/recognizer_registry/recognizers_loader_utils.py

@@ -20,6 +20,7 @@
 import yaml
 
 from presidio_analyzer import EntityRecognizer, PatternRecognizer
+from presidio_analyzer.score_thresholds import normalize_score_thresholds
 
 logger = logging.getLogger("presidio-analyzer")
 
@@ -393,9 +394,13 @@ def get(
             "supported_languages",
             "class_name",
             "country_code",
+            "score_thresholds",
         }
-        custom_to_exclude = {"enabled", "type", "class_name"}
+        custom_to_exclude = {"enabled", "type", "class_name", "score_thresholds"}
         for recognizer_conf in predefined:
+            score_thresholds = normalize_score_thresholds(
+                recognizer_conf.get("score_thresholds")
+            )
             for language_conf in RecognizerListLoader._get_recognizer_languages(
                 recognizer_conf=recognizer_conf, supported_languages=supported_languages
             ):
@@ -421,19 +426,25 @@ def get(
                         new_conf, language_conf, recognizer_cls
                     )
 
-                    recognizer_instances.append(recognizer_cls(**kwargs))
+                    recognizer = recognizer_cls(**kwargs)
+                    recognizer.score_thresholds = score_thresholds
+                    recognizer_instances.append(recognizer)
 
         for recognizer_conf in custom:
             if RecognizerListLoader.is_recognizer_enabled(recognizer_conf):
+                score_thresholds = normalize_score_thresholds(
+                    recognizer_conf.get("score_thresholds")
+                )
                 new_conf = RecognizerListLoader._filter_recognizer_fields(
                     recognizer_conf, to_exclude=custom_to_exclude
                 )
-                recognizer_instances.extend(
-                    RecognizerListLoader._create_custom_recognizers(
-                        recognizer_conf=new_conf,
-                        supported_languages=supported_languages,
-                    )
+                custom_recognizers = RecognizerListLoader._create_custom_recognizers(
+                    recognizer_conf=new_conf,
+                    supported_languages=supported_languages,
                 )
+                for recognizer in custom_recognizers:
+                    recognizer.score_thresholds = score_thresholds
+                recognizer_instances.extend(custom_recognizers)
 
         for recognizer_conf in recognizer_instances:
             if isinstance(recognizer_conf, PatternRecognizer):