Simulating Auxiliary Data Joins Against a Masked Release

A re-identification risk figure derived from the release alone answers the wrong question. The right question is what happens when someone joins the release to the address register, the company filings, the electoral roll, or a scraped listings site — and the only honest way to answer it is to build that auxiliary dataset and run the join.

Core Calculation # permalink

An adversary holds an auxiliary table $A$ with identified records and some spatial attribute, and the masked release $R$. A join succeeds for target $a \in A$ when exactly one released record falls in its candidate set:

Compatibility for a masked point is a disc: the released point could have come from anywhere within the mask radius, so the adversary inverts it.

Three risk figures come out, and they answer different questions:

$\pi_{\text{unique}}$ is what the adversary believes they achieved; $\pi_{\text{correct}}$ is what they actually achieved; $\pi_{\text{marketer}}$ is the expected success of guessing at random within each candidate set. The gap between the first two is the mask’s most useful property — a mask that produces many confident wrong answers is doing real work — and reporting only $\pi_{\text{correct}}$ understates the harm, because a confident wrong match damages the person it lands on.

Worked numeric example # permalink

A masked mobility release, 62 000 subjects, 300 m fuzz radius, quasi-identifiers age band and trip purpose. Auxiliary table: 8 400 identified individuals assembled from a commercial address file joined to an employer directory.

Location alone is nearly harmless. Adding two ordinary demographic attributes takes correct re-identification from 0.4 % to 14.8 % — a 37-fold increase from information that appears on a marketing list. Adding a single known trip time takes it to 49 %.

That last row is the one the assessment exists to produce. It says the release is safe against an adversary with a phone book and unsafe against a colleague, a neighbour, or an ex-partner — and those adversaries are far more common and far more dangerous than the commercial data broker the threat model was probably written around.

Python Implementation # permalink

from __future__ import annotations

from dataclasses import dataclass

import numpy as np
import pandas as pd
from scipy.spatial import cKDTree


@dataclass(frozen=True)
class JoinSpec:
    mask_radius: float               # the inversion radius the adversary assumes
    qid_columns: tuple[str, ...]     # attributes present in BOTH tables
    truth_column: str = "subject_id" # held out of the adversary's view


def simulate_join(auxiliary: pd.DataFrame, release: pd.DataFrame,
                  spec: JoinSpec) -> dict:
    """Run the adversary's join and report what they get, not what we hope.

    The adversary is assumed to know the mask radius — it is usually in the
    methodology note, and assuming otherwise makes the assessment optimistic
    in exactly the way that gets releases withdrawn.
    """
    rel_xy = release[["x", "y"]].to_numpy()
    tree = cKDTree(rel_xy)

    unique_hits = 0
    correct_hits = 0
    marketer_sum = 0.0
    candidate_sizes = []

    for _, target in auxiliary.iterrows():
        # Spatial candidates: any released point whose mask disc covers the
        # target's true position.
        idx = tree.query_ball_point([target["x"], target["y"]], spec.mask_radius)
        if not idx:
            candidate_sizes.append(0)
            continue

        cand = release.iloc[idx]
        for col in spec.qid_columns:
            cand = cand[cand[col] == target[col]]

        n = len(cand)
        candidate_sizes.append(n)
        if n == 0:
            continue
        marketer_sum += 1.0 / n
        if n == 1:
            unique_hits += 1
            if cand.iloc[0][spec.truth_column] == target[spec.truth_column]:
                correct_hits += 1

    n_targets = len(auxiliary)
    pi_unique = unique_hits / n_targets
    pi_correct = correct_hits / n_targets
    return {
        "pi_unique": pi_unique,
        "pi_correct": pi_correct,
        "pi_marketer": marketer_sum / n_targets,
        # Precision below ~0.35 means most confident matches are wrong, which
        # is the mask working. It is not a reason to relax pi_unique.
        "precision": (pi_correct / pi_unique) if pi_unique else 0.0,
        "median_candidates": float(np.median(candidate_sizes)),
        "targets_with_no_candidate": int(sum(1 for c in candidate_sizes if c == 0)),
    }


def escalation_curve(auxiliary: pd.DataFrame, release: pd.DataFrame,
                     radius: float, attribute_order: list[str]) -> pd.DataFrame:
    """Add one auxiliary attribute at a time and record where risk crosses."""
    rows = []
    for i in range(len(attribute_order) + 1):
        spec = JoinSpec(radius, tuple(attribute_order[:i]))
        result = simulate_join(auxiliary, release, spec)
        result["attributes"] = ["location"] + attribute_order[:i]
        rows.append(result)
    return pd.DataFrame(rows)

Verification # permalink

The auxiliary table must be obtainable, not hypothetical. Assemble it from sources you can name and, ideally, actually acquired: an address file you licensed, a public register you downloaded, a directory you scraped. Record the provenance and the cost. An assessment against an auxiliary table nobody could build is a lower bound presented as an estimate.

The simulation must find something on unmasked data. A join that reports low risk against the masked release and also reports low risk against the raw data is broken, not reassuring:

def test_join_finds_raw_data():
    """The rejection test: against unmasked data the join must succeed."""
    aux, raw = load_fixtures()
    result = simulate_join(aux, raw, JoinSpec(mask_radius=1.0, qid_columns=()))
    # Every target has an exact-position match in its own source data.
    assert result["pi_correct"] > 0.95, (
        "the join cannot re-identify raw data, so a low score on the masked "
        "release says nothing about the mask")

Report the escalation curve, not a single number. One figure invites the reading “we measured risk and it was 4 %”. The curve says what an additional attribute costs, which is the thing that decides whether the release survives contact with a second dataset.

Sample the auxiliary table to match the adversary’s coverage, not the population. A broker file covers 60 % of households and skews toward homeowners. Simulating against a uniform 60 % sample understates risk for the covered group; simulate against the actual coverage pattern where you can characterise it, and say so where you cannot.

Re-run after every change to the release schema. A new column is a new quasi-identifier. The escalation curve makes this cheap: add the column to attribute_order and read the new row.

Edge Cases & Adjustments # permalink

• The adversary knows the mask radius. Publish it, and assume it is known — obscuring the parameter is not a control, and an adversary who guesses too large a radius simply gets more candidates and lower precision, which does not help them but does not help you either.
• Auxiliary tables with errors. Real address files are 3–8 % stale. Model that: inject an error rate into the auxiliary positions and re-run. Risk falls, but less than people expect, because errors mostly turn correct matches into no-matches rather than into wrong ones.
• Sequential releases. Two releases of the same population let the adversary intersect candidate sets. Simulate the join against the union of all releases, not the newest one, and re-run the whole history whenever a new product launches.
• Targets not in the release. An auxiliary record with no compatible released point tells the adversary the subject is probably not in the dataset — a membership inference, and sensitive when the dataset is a clinic’s or a support service’s. Report targets_with_no_candidate and treat a low number as evidence of high coverage, which is its own disclosure.
• Legal constraints on holding the auxiliary data. Assembling a real auxiliary table may itself need a lawful basis. Do it under the same controls as the source data, keep it inside the analysis environment, and delete it on a stated schedule — the assessment must not create the risk it measures.

FAQ # permalink

Isn’t this just an attack on my own data?

Yes, conducted under your own controls, with your own data, to establish a fact you would otherwise be guessing at. It is the spatial analogue of a penetration test and it needs the same authorisation and the same handling discipline.

What if I cannot obtain a realistic auxiliary table?

Then simulate one from published marginals — census age and tenure distributions joined to address points — and label the result a lower bound. Say clearly which attributes you could not model.

Which figure goes in the release note?

The curve, plus $\pi_{\text{unique}}$ and $\pi_{\text{correct}}$ at the auxiliary strength you consider realistic, plus a sentence naming what an adversary would need in order to cross the threshold.

Does a low precision mean the release is safe?

No. It means confident matches are often wrong, which protects the true subject and harms the misidentified one. Both are outcomes of the release, and $\pi_{\text{unique}}$ is the figure that counts them together.

Related # permalink

• Re-identification Risk Assessment for Geospatial Datasets — the assessment this is the empirical half of
• Spatial Linkage Attack Vectors & Mitigation — the attack surface being simulated
• Gating Releases on a Re-identification Risk Budget — turning the figure into a blocking check
• Handling Outliers That Break Spatial K-Anonymity — the records that dominate the match rate
• Mitigating Co-location Inference Across Joined Datasets — the inference that survives a failed identity join

← Back to Re-identification Risk Assessment for Geospatial Datasets