Policy Recommendations

Every audit writes a recommendations array into report.json alongside the metrics. Each entry is one suggested change to your Philter policy, carrying both the prose a person reads and the structured fields a tool reads, so an importer such as the Redaction Policy Editor never has to parse the description.

Recommendations are advisory. PhilterScope explains what it measured and what would address it, and stops there. It does not edit a policy, and it does not apply anything on your behalf.

What produces a recommendation

Recall is the objective: missing PII is the expensive error, so a recommendation that asks you to change a policy asks you to find more.

Precision bounds that advice rather than competing with it. Advice driven by recall alone points one way forever, because a policy that redacts every character scores perfect recall, so PhilterScope reports what a recall change is expected to cost and flags a filter that has clearly gone too far. It never proposes redacting less to make precision look better.

Two kinds follow from that.

Kind	Raised when	What it asks for
`recall_below_threshold`	An entity's recall is under its threshold (`--threshold`, or its entry in `--thresholds`)	A policy change that finds more of that entity
`precision_collapsed`	An entity's precision is under `--precision-floor`	A look at the filter, since matching this far past the labels usually means a misconfiguration

An LLM run (--ai) adds entries of kind ai. Those carry no measurement, since a model produced them from the summary rather than from the detections, and their structured fields are left empty rather than filled with invented numbers.

Fields

Field	Type	Description
`id`	string	Stable identity, `<kind>:<entity>`. One entity can carry several recommendations, so this and not `entity` is what identifies one.
`kind`	string	`recall_below_threshold`, `precision_collapsed`, or `ai`.
`entity`	string	The entity type, as the gold standard labels it.
`description`	string	What was measured, for a person to read.
`action`	string	What to do about it, for a person to read.
`snippet`	string	A Philter policy fragment applying the change. Empty where no valid fragment can be written.
`metric`	string	`recall` or `precision`: which measurement raised this.
`value`	number	The measured value, 0.0 to 1.0.
`threshold`	number	What it was compared against.
`current_recall`	number	The entity's recall at audit time. Absent where the gold standard labels no spans for it.
`current_precision`	number	The entity's precision at audit time. Absent where nothing was detected for it.
`change`	object	The proposed edit, structured. See below.
`projection`	object	What the change is expected to do. See below.
`resolved`	boolean	Marked resolved in the dashboard.
`dismissed`	boolean	Marked dismissed in the dashboard.
`is_ai`	boolean	Generated by a language model rather than measured.

`change`

Field	Type	Description
`type`	string	`enable_filter`, `lower_confidence_threshold`, `raise_confidence_threshold`, `review_filter`, or `none`.
`filter`	string	The policy identifier key, as a policy spells it (`phoneNumber`, `creditCard`). Empty when the entity maps to no Philter filter.
`field`	string	The field within the filter, where the change targets one (`thresholds.PERSON`).
`from`	number	The current value, where there is one.
`to`	number	The proposed value, where one is proposed.

review_filter and none are deliberate outcomes, not failures. review_filter means the change needs a human decision: the policy could not be read, or the gold standard uses a label that matches no Philter filter, and naming one would be a guess. none means the problem is real but no policy change addresses it without costing recall.

The gold standard's labels are free text, and filter is PhilterScope's mapping of a label onto a Philter filter name. Common spellings resolve on their own (PHONE_NUMBER, phone-number, and phoneNumber all reach phoneNumber) and a few short labels are mapped by hand (NAME to person, ADDRESS to streetAddress). A label matching nothing leaves filter empty rather than inventing a key Philter would reject.

`projection`

Field	Type	Description
`predicted`	boolean	Whether the numbers were computed, or the effect is simply unknown.
`recall`	number	Recall after the change. Present only when `predicted` is true.
`precision`	number	Precision after the change. Present only when `predicted` is true.
`note`	string	What the projection does and does not establish.

predicted is the field to branch on, and treating an unpredicted projection as a measurement will mislead your users.

Only raising a confidence threshold can be predicted. Every detection it would drop is already in the report, so the arithmetic is exact for that dataset. Widening a filter cannot be: the spans it would newly admit were filtered out before Philter returned anything, so nothing in the report says how many of them are real. Recall gaps therefore carry predicted: false and report the entity's current precision as the standing cost, and you learn the actual effect by re-running the audit.

A predicted projection is also constrained: PhilterScope proposes a cutoff only where recall is left completely unchanged. A cutoff that would buy precision by dropping a real detection is never suggested, even when recall would stay above its threshold.

Example

An entity the audited policy has no filter for:

{
  "id": "recall_below_threshold:PHONE_NUMBER",
  "kind": "recall_below_threshold",
  "entity": "PHONE_NUMBER",
  "description": "Recall for PHONE_NUMBER is 40.0%, which is below the 75% threshold. 3 of 5 labeled spans were missed.",
  "action": "Enable the phoneNumber filter. The audited policy has no filter for it.",
  "snippet": "{\n  \"identifiers\": {\n    \"phoneNumber\": {\n      \"enabled\": true\n    }\n  }\n}",
  "metric": "recall",
  "value": 0.4,
  "threshold": 0.75,
  "current_recall": 0.4,
  "current_precision": 0.667,
  "change": {
    "type": "enable_filter",
    "filter": "phoneNumber"
  },
  "projection": {
    "predicted": false,
    "note": "Precision for PHONE_NUMBER is 66.7% today and this change will not raise it. Widening a filter admits spans this audit never saw, so the effect on precision cannot be computed from this report. Re-run the audit after the change to measure it."
  },
  "resolved": false,
  "dismissed": false,
  "is_ai": false
}

A filter matching far past its labels, where a cutoff exists that costs no recall:

{
  "id": "precision_collapsed:PERSON",
  "kind": "precision_collapsed",
  "entity": "PERSON",
  "description": "Precision for PERSON is 25.0%, below the 25% floor: 6 of 8 detections did not match a labeled span. A filter matching this far past the gold standard is usually misconfigured rather than deliberately wide.",
  "action": "Raise the confidence threshold for the person filter to 0.9. Recall is unchanged at 100.0%: this drops only detections that matched no labeled span.",
  "metric": "precision",
  "value": 0.25,
  "threshold": 0.25,
  "current_recall": 1.0,
  "current_precision": 0.25,
  "change": {
    "type": "raise_confidence_threshold",
    "filter": "person",
    "field": "thresholds.PERSON",
    "to": 0.9
  },
  "projection": {
    "predicted": true,
    "recall": 1.0,
    "precision": 1.0,
    "note": "Computed from this audit's own detections: recall 100.0% (from 100.0%), precision 100.0% (from 25.0%). Raising a cutoff only drops detections already measured, so these are exact for this dataset."
  },
  "resolved": false,
  "dismissed": false,
  "is_ai": false
}

Per-entity statistics

Recommendations are derived from entity_stats, which the report carries for every entity either the gold standard labels or Philter detected:

"entity_stats": {
  "PHONE_NUMBER": {
    "true_positives": 2,
    "false_positives": 1,
    "false_negatives": 3,
    "precision": 0.667,
    "recall": 0.4
  }
}

entity_metrics, which holds recall alone, is still written for compatibility. It covers only entities the gold standard labels, so an entity that was detected but never labeled appears in entity_stats and not in entity_metrics.

Consuming recommendations

A few things worth holding to when you import these.

Address a recommendation by id, never by entity. One entity can raise both a recall gap and a precision warning, and they are separate decisions.

Branch on kind and change.type rather than reading action. The prose is written for people and will change wording.

Check projection.predicted before showing a number as a projection, and present a false as unknown rather than as no change.

Treat change.type of review_filter or none as needing a person, not as an empty result.

Read snippet as a fragment to merge into a policy, not as a complete policy. It carries only the keys the change touches, and it is empty where no valid fragment could be written.

Nothing here is a guarantee that a suggested policy catches all PII. These are calibrated suggestions measured against one gold standard, and how well they hold depends on how well that gold standard represents your data. Validate the result against your own before relying on it.