Running PhilterScope

PhilterScope is a standalone CLI tool for PII redaction auditing and policy optimization. This document explains the available commands and flags.

Note that PhilterScope is intended to be run locally and not over a network. If running over a network, be sure to use SSL/TLS connection to MongoDB and to the PhilterScope UI.

1. Installation

PhilterScope is written in Go. You can build the binary for your platform using the provided Makefile:

make build

This will create philterscope-audit and philterscope-serve binaries in the project root.

2. Commands

PhilterScope provides two primary commands: philterscope-audit for performing audits and philterscope-serve for viewing results.

`philterscope-audit`

The philterscope-audit command compares raw text files against a "golden dataset" to evaluate redaction quality.

Usage:

PHILTERSCOPE_MONGODB_CONNECTION_STRING=mongodb://localhost:27017/philterscope ./philterscope-audit [flags]

Or without MongoDB:

./philterscope-audit [flags]

Commonly Used Flags:

Flag	Default	Description
`--url`	`http://localhost:8080`	The Philter API URL to use for redacting the raw text files.
`--token`	(none)	The Philter API Token, if required by your Philter server.
`--policy`	`default`	The name of the Philter policy to use for redaction.
`--input`	`./raw`	The directory containing the raw text files or Philter `explain` JSON files.
`--golden`	`golden.json`	The path to the golden dataset file or directory.
`--output`	`.`	The directory where the `report.html` and `report.json` will be saved.
`--threshold`	`0.5`	The default recall threshold for policy suggestions (0.0 to 1.0).
`--thresholds`	(none)	Per-entity recall thresholds (e.g., `NAME=0.9,SSN=1.0`).
`--group`	`default`	Assign a group name to the audit for history tracking.
`--ai`	`false`	Enable AI-driven policy recommendations (requires Ollama).
`--best-effort`	`false`	Score the files that can be scored instead of failing the run.
`--version`		Print the version stamped in at build time and exit.

Example:

./philterscope-audit --input ./examples/raw --golden ./examples/golden --output ./examples/ --threshold 0.75 --ai

Thresholds can also be set individually for each entity type:

./philterscope-audit --golden ./examples/golden/ --input ./examples/raw/ --output ./examples/ --threshold 0.75 --thresholds "NAME=0.9,SSN=1.0"

Exit Codes and Skipped Files

Code	Meaning
`0`	The audit ran and a report was written.
`1`	The audit did not produce a usable result.

An audit fails rather than reporting a score it cannot stand behind:

If Philter cannot be reached, the run stops at the first file with one error, instead of warning once per file and then scoring nothing.
If no input file could be scored, the run fails rather than writing a report whose precision and recall are zero because nothing was measured.

A file that cannot be read or redacted, while other files can, is skipped and the run continues. Skipped files are counted in the summary line and recorded in the JSON report under files_skipped and skipped, each with the reason. A skipped file is not the same as a file that scored zero: it contributed nothing to the metrics.

--best-effort opts out of both failures, for callers that would rather have a partial score than an error. The report still records what was skipped.

`philterscope-serve`

The philterscope-serve command launches the Evaluation UI, allowing you to view and interact with the results of a previous audit.

Usage:

./philterscope-serve [flags]

Flags:

Flag	Default	Description
`--report`	`report.json`	The JSON report file generated by the `audit` command.
`--port`	`5000`	The port on which the UI will be served.
`--privacy`	`false`	Enable privacy mode (obfuscates PII in UI).
`--id`	(none)	The ID of a specific audit result to view from history.
`--version`		Print the version stamped in at build time and exit.

Example:

PHILTERSCOPE_MONGODB_CONNECTION_STRING=mongodb://localhost:27017/philterscope ./philterscope-serve --privacy

Or without MongoDB:

./philterscope-serve --report ./examples/report.json --port 5000 --privacy

With no arguments, philterscope-serve picks a source in this order:

MongoDB, if PHILTERSCOPE_MONGODB_CONNECTION_STRING is set and reachable.
The report named by --report, or a report.json in the current directory.
The local .philterscope audit history.

Serving a single report shows that one audit. The other two serve the full history, where you can browse past audits, add notes, and resolve or dismiss recommendations.

Health Endpoint

philterscope-serve exposes an unauthenticated health endpoint at /api/health for container runtimes, load balancers, and uptime checks. It is a liveness probe: it reports that the server is answering and does not check MongoDB.

curl http://localhost:5000/api/health

{"status": "UP", "applicationVersion": "1.2.3"}

A healthy server returns 200 with "status": "UP". The version comes from the VERSION set at build time (make build VERSION=1.2.3, or docker build --build-arg VERSION=1.2.3); builds made without it report dev.

This is a liveness check. It reports that the server is answering and deliberately does not touch storage, so a database outage does not make a working server look dead to a container runtime that would restart it. Use the readiness endpoint for that.

Readiness Endpoint

GET /api/readyz reports whether the storage behind the server can be reached. It is unauthenticated.

curl http://localhost:5000/api/readyz

{"status": "READY", "mode": "mongodb"}

mode says which backend the server came up with: mongodb, file for the local .philterscope history, or report when serving a single report file, which has no backend and is always ready.

When MongoDB is configured but cannot be reached, the endpoint returns 503 and says why:

{"status": "NOT_READY", "mode": "mongodb", "reason": "failed to reach MongoDB: server selection error..."}

Point a load balancer at /api/readyz so it stops sending requests that would fail, and a container runtime's liveness probe at /api/health so it only restarts a server that has actually stopped serving.

Metrics Endpoint

GET /metrics exposes Prometheus text format, unauthenticated:

curl http://localhost:5000/metrics

Alongside the standard Go runtime and process metrics:

Metric	Type	Labels	Description
`philterscope_http_requests_total`	counter	`route`, `method`, `code`	API requests served.
`philterscope_http_request_duration_seconds`	histogram	`route`, `method`	API request duration.

route is the registered route, not the request path, so an audit ID in a query string never becomes a label value. A labelled series appears once the first matching request has been served. The health, readiness and metrics endpoints are not counted: probe and scrape traffic would otherwise swamp the request counts they report.

History and Audit Management

PhilterScope maintains a history of your audit runs. When using philterscope-serve, you can browse previous audits and their results.

Local Storage: By default, audits are stored as one JSON file per audit in a .philterscope directory, created in the directory you run the command from. philterscope-serve reads that history, so audits are browsable without a database. It assumes a single user on one machine: nothing coordinates concurrent writers.
Shared Storage: Set PHILTERSCOPE_MONGODB_CONNECTION_STRING for a centralized audit repository across your team. It takes precedence over local storage.
Privacy Mode: When --privacy is enabled, all PII found in the audit results (both expected and actual) is replaced with a cryptographic hash in the UI.

Running in Docker

The image carries both binaries. philterscope-serve is the default command, the working directory is /data, and the container runs as a non-root user. It is published on Docker Hub for linux/amd64 and linux/arm64:

docker pull philterd/philter-scope:latest

latest tracks the most recent release. Pin a version for anything you deploy:

docker pull philterd/philter-scope:0.1.0

Serve a report:

docker run --rm -p 5000:5000 \
  -v "$PWD/examples/report.json:/data/report.json:ro" \
  philterd/philter-scope:latest

Setting PHILTERSCOPE_MONGODB_CONNECTION_STRING takes precedence over the report file.

Run an audit by overriding the command. --user is needed for the non-root container to write the reports back to a bind-mounted directory:

docker run --rm --user "$(id -u):$(id -g)" \
  -v "$PWD/examples:/data" \
  philterd/philter-scope:latest \
  philterscope-audit --golden /data/golden/ --input /data/raw/ --output /data/ --threshold 0.75

The running container reports its version at /api/health and from philterscope-serve --version. Building the image yourself is covered in the development guide.

Docker Compose

The repository ships a docker-compose.yaml that runs both steps in order. The audit runs once and writes its reports into ./data, then the dashboard starts and serves them on port 5000:

curl -O https://raw.githubusercontent.com/philterd/philter-scope/main/docker-compose.yaml

mkdir -p data/golden data/raw   # your golden dataset, and the text to score

export PHILTER_URL=http://philter.internal:8080
docker compose up

The file reads its settings from the environment or from a .env file next to it.

Variable	Default	Description
`PHILTER_URL`	`http://localhost:8080`	Philter API used to redact the raw text.
`PHILTER_TOKEN`	(none)	Philter API token, if your Philter requires one.
`PHILTERSCOPE_VERSION`	`0.1.0`	Image tag to run.
`PHILTERSCOPE_THRESHOLD`	`0.75`	Recall threshold, applied to both the audit and the dashboard.
`PHILTERSCOPE_UID`	`1000`	Your `id -u`, so the audit can write into `./data`.
`PHILTERSCOPE_GID`	`1000`	Your `id -g`.

3. Data Formats

PhilterScope is designed to be flexible with your data. It supports multiple formats for both input files and golden datasets.

Input Files

The input directory (--input) can contain:

Raw Text Files: Simple .txt files that will be sent to the Philter API for redaction.
Philter Explain JSON: If you have already redacted text using Philter's explain API, you can provide the JSON response directly. This allows you to audit pre-redacted data without calling the Philter API again.

Golden Datasets

The golden dataset (--golden) defines the expected redactions. PhilterScope looks for a match in several ways:

Tagged Text: Wrap PII in your raw text files with tags like <NAME>John Doe</NAME>. PhilterScope can parse these directly.
JSON Spans: A JSON file that defines the text and the character offsets for each PII entity. This is the recommended format for large datasets.

Example JSON Span format:

{
  "text": "My name is John Doe and I live at 123 Main St.",
  "labels": [
    {
      "text": "John Doe",
      "start": 11,
      "end": 19,
      "label": "NAME"
    }
  ]
}

Matching Logic

When you run philterscope-audit, PhilterScope searches for the golden data in this order:

The path provided by the --golden flag (if it's a file).
Matching filenames in the directory provided by --golden (if it's a directory).
<filename>.golden in the input directory.
A golden/ subdirectory within or next to your input directory.
Inline tags within the input file itself.

4. Understanding the Report

After an audit completes, PhilterScope generates an HTML report and a JSON report containing overall metrics (precision, recall, F1-score), per-entity recall, a confusion matrix, per-document results, and recommended policy changes.

Confusion Matrix

The confusion matrix shows how each expected entity type was classified by Philter. It is displayed immediately below the PII Recall Performance table in the report.

Each row represents an expected entity type from the golden dataset, and each column represents what Philter detected it as. The cells contain the count of occurrences.

There are two special labels in the matrix:

(missed) (column): The entity was present in the golden dataset but Philter did not detect it at all. These are false negatives.
(none) (row): Philter detected an entity that was not present in the golden dataset. These are false positives (spurious detections).

Cells are color-coded:

Green: Correct classifications (expected and detected types match).
Orange: Misclassifications (Philter detected the entity but assigned the wrong type, e.g., a NAME detected as an ADDRESS).
Red: Missed entities or spurious detections.

Using the Confusion Matrix for Policy Tuning

The confusion matrix helps identify specific weaknesses in your Philter policy:

High counts in the (missed) column for a given entity type indicate that Philter is failing to detect that type. Consider adjusting the policy to add or tune the relevant filter.
Off-diagonal orange cells reveal type confusion. For example, if LOCATION entities are frequently detected as ADDRESS, the policy may need more specific patterns to distinguish the two.
High counts in the (none) row indicate false positives. Philter is flagging text that is not PII, which may require tightening filter rules or adjusting confidence thresholds.

The confusion matrix data is also included in the JSON report under the confusion_matrix field for programmatic analysis.