Cities

Filter

This filter identifies common US cities as determined by the US census in text.

Required Parameters

This filter has no required parameters.

Optional Parameters

Parameter	Description	Default Value
cityFilterStrategies	A list of filter strategies.	None
terms	A list of terms to use instead of the bundled word list. When given, only these terms are matched. When omitted, the bundled list is used. Terms are plain text, not regular expressions, and the words of a multi-word term may be separated in the text by any whitespace, such as a line break or tab.	None
fuzzy	When true, a word that is close to a term, such as a misspelling, is also reported, as set by sensitivity and capitalized. When false, only exact matches are reported, ignoring case, and sensitivity and capitalized have no effect.	false
sensitivity	How close a word must be to a term to be reported when fuzzy is true, measured in edits (a character inserted, removed, or changed). high reports only exact matches, medium allows one edit, and low allows up to two. off reports only exact matches. auto allows no edits for a word under five characters, one edit for a word under nine, and two for a longer word, counts two swapped neighboring letters as one edit (so Smiht is one edit from Smith), and reports a fuzzy match only when it begins with a capital letter, which helps keep common short words such as and and the from matching names and places. Words shorter than three characters, not counting punctuation around them, are not compared. Any other value refuses the policy when it is loaded, even when fuzzy is false.	auto
capitalized	When true, a match is reported only when it begins with a capital letter. Only applies when fuzzy is true.	false
windowSize	Sets the size of the window (in terms) surrounding a span to look for contextual terms. If set, this value overrides the value of span.window.size in the configuration.	The value of span.window.size which is by default 5.
priority	The priority (integer) of this filter. Valid values are any positive integer, where a higher value indicates a higher priority. Priority is used for tie-breaking when two spans may be otherwise identical.	0

Cities That Are Common Words

About 1,300 names in the bundled list are also common English words, such as Call, Driver, Hope, Center, Phoenix, and Mobile. One of these is reported as a city only when it begins with a capital letter and either:

• follows in, from, to, near, outside, toward, towards, or via, as in Born in Phoenix., or
• is followed by a comma and a state name or two-letter state abbreviation, as in Mobile, Alabama or Reading, PA.

So Call the clinic with questions. and She lives on Oak Street in Austin. report only Austin. Other names in the list, such as Austin, are reported wherever they appear, in any case. This applies to exact and fuzzy matches of the bundled list, not to terms a policy gives.

Filter Strategies

The filter may have zero or more filter strategies. When no filter strategy is given the default strategy of REDACT is used. When multiple filter strategies are given the filter strategies will be applied in as they are listed. See Filter Strategies for details.

Strategy	Description
REDACT	Replace the sensitive text with a placeholder.
MASK	Replace each character of the sensitive text with a mask character (* by default).
TRUNCATE	Replace all but a few characters of the sensitive text with a truncation character (* by default).
RANDOM_REPLACE	Replace the sensitive text with a similar, random value.
STATIC_REPLACE	Replace the sensitive text with a given value.
CRYPTO_REPLACE	Replace the sensitive text with its encrypted value.
HASH_SHA256_REPLACE	Replace the sensitive text with its SHA256 hash value.
FPE_ENCRYPT_REPLACE	Replace the sensitive text with a value generated by format-preserving encryption (FPE)
LAST_4	Replace the sensitive text with just the last four characters of the text.

Conditions

Each filter strategy may have one condition. See Conditions for details.

Conditional	Description	Operators
TOKEN	Compares the value of the sensitive text.	==, !=, startswith
CONTEXT	Compares the filtering context.	== , !=
CONFIDENCE	Compares the confidence in the sensitive text against a threshold value.	< , <=, > , >=, ==, !=

Limitations

• A city that is also a common word is missed without a location context. Phoenix is hot this summer., in the Phoenix area, Hope AR (no comma), and lives in phoenix (lowercase) are not reported. To find such a city wherever it appears, give the cities you need in terms, which are matched without this rule.
• A capitalized common word after a location word is reported, as in a title such as Welcome to Paradise. List such words in ignored.
• With fuzzy, a capitalized word close to a city name can be reported, such as Please at the start of a sentence.

Example Policy

{
   "identifiers": {
      "city": {
         "sensitivity": "medium",
         "cityFilterStrategies": [
            {
               "strategy": "REDACT",
               "redactionFormat": "{{{REDACTED-%t}}}"
            }
         ]
      }
   }
}