Home
Geospatial Masking & Perturbation Techniques
Coordinate Jittering & Noise Injection Methods
Testing Jitter Quality with Displacement Distribution Checks

Testing Jitter Quality with Displacement Distribution Checks

Q: Which single test would I keep if I could keep one?

Test 4, per-subject stability. The others degrade the guarantee; independent re-jitter destroys it, and it is invisible in every single-record inspection.

Q: Is a KS threshold of 0.01 too strict?

For 40 000 points it is about right — the critical value at \alpha = 0.05 is roughly 0.0068. For 1 000 points use 0.043. Scale the threshold with 1.36/\sqrt{n} rather than fixing it.

Q: Why test independence separately when both marginals pass?

Because uniform-on-a-square jitter has correct-looking marginals and correlated joint structure: the maximum displacement available at 45° is \sqrt{2} times that at 0°. Only the joint test sees it.

Q: Should these run on every release or once at implementation?

Every release. The lattice failure in the worked example was introduced by a change to the export query months after the jitter code was verified.

A jitter implementation that runs without error, keeps every point inside its radius, and produces a plausible-looking scatter can still deliver a fraction of the protection it claims. The failures are all in the distribution, not the code path, so they need distributional tests — six of them, each catching something the others do not.

Core Calculation permalink

For coordinate jittering with an area-uniform disc of radius $R$ , the displacement magnitude $D = \lVert y - x \rVert$ has CDF

F_D(r) = \left(\frac{r}{R}\right)^2, \qquad 0 \le r \le R

with median $R/\sqrt{2} = 0.707R$ and mean $2R/3$ . For an isotropic Gaussian with per-axis standard deviation $\sigma$ , $D$ is Rayleigh:

F_D(r) = 1 - e^{-r^2 / (2\sigma^2)}, \qquad \mathbb{E}[D] = \sigma\sqrt{\pi/2}, \qquad \text{median} = \sigma\sqrt{2\ln 2}

The bearing $\Theta$ must be uniform on $[0, 2\pi)$ under both, and — this is the test people skip — independent of $D$ .

The six tests:

#	Test	Catches
1	KS against the radial CDF	Uniform-radius sampling; wrong $R$ or $\sigma$
2	Rayleigh test on bearings	Axis-aligned noise; degrees-vs-metres bugs
3	Independence of $D$ and $\Theta$	Rectangular jitter dressed as radial
4	Per-subject displacement variance	Re-jittering that averages away
5	Quantisation / lattice detection	Rounding after jitter
6	Correlation of $D$ with local density	Constraint-induced bias

Test 2 catches the single most common production bug. Adding $\mathcal{N}(0, \sigma)$ independently to latitude and longitude in degrees produces displacements that are stretched east–west by $1/\cos(\phi)$ : at latitude 55° a nominal 100 m jitter is 100 m north–south and 174 m east–west, and the bearing distribution has two clear lobes.

Worked numeric example permalink

A release of 40 000 jittered points, claimed radius $R = 300$ m, area-uniform disc.

Test	Statistic	Expected	Observed	Verdict
1. Radial KS	$D_{KS}$	< 0.010	0.144	fail
2. Bearing Rayleigh	$p$	> 0.01	0.31	pass
3. $D \perp \Theta$	$\chi^2$ $p$	> 0.01	0.62	pass
4. Per-subject var	ratio	≈ 1.0	0.08	fail
5. Lattice	distinct residues	> 1000	4	fail
6. Density corr	Spearman $\rho$	\|ρ\| < 0.05	0.02	pass

Three failures, three distinct bugs. Test 1’s KS of 0.144 with an observed median of 148 m against an expected 212 m is the missing square root. Test 4’s ratio of 0.08 means the same subject was jittered independently 12 times, so averaging their releases recovers the true point to within 300/√12 ≈ 87 m. Test 5’s four distinct residues means coordinates were rounded to 5 decimal places after jittering — quantising the output to a 1.1 m lattice, which is harmless, and to 4 residues, which means the rounding was to about 150 m and half the jitter was thrown away.

Any one of these ships silently. All three together mean the released points sit within roughly 90 m of the truth while the documentation claims 300.

A "max displacement ≤ R" assertion passes on this data. The CDF is the assertion that does not.

Python Implementation permalink

from __future__ import annotations

import numpy as np
from scipy import stats


def radial_cdf_test(orig: np.ndarray, rel: np.ndarray, radius: float) -> dict:
    """Test 1 — displacement magnitude against the area-uniform disc law."""
    d = np.linalg.norm(rel - orig, axis=1)
    stat, p = stats.kstest(d / radius, lambda r: np.clip(r, 0, 1) ** 2)
    return {"test": "radial_cdf", "ks": float(stat), "p": float(p),
            "median_m": float(np.median(d)),
            "expected_median_m": radius * np.sqrt(0.5),
            "passes": bool(stat < 0.01)}


def bearing_uniformity_test(orig: np.ndarray, rel: np.ndarray) -> dict:
    """Test 2 — bearings must be uniform, not stretched east-west.

    A failure here almost always means the noise was added in degrees:
    a degree of longitude is cos(lat) times shorter than one of latitude,
    so the displacement ellipse is elongated by 1/cos(lat).
    """
    delta = rel - orig
    theta = np.arctan2(delta[:, 1], delta[:, 0])
    # Rayleigh test: the mean resultant length of unit vectors is ~0 if uniform.
    r_bar = np.abs(np.exp(1j * theta).mean())
    n = len(theta)
    p = np.exp(np.sqrt(1 + 4 * n + 4 * (n ** 2 - (n * r_bar) ** 2)) - (1 + 2 * n))
    return {"test": "bearing_uniformity", "resultant": float(r_bar),
            "p": float(p), "passes": bool(p > 0.01)}


def independence_test(orig: np.ndarray, rel: np.ndarray, bins: int = 8) -> dict:
    """Test 3 — magnitude and bearing must be independent.

    Rectangular jitter (uniform on a square) passes tests 1 and 2 loosely but
    has larger displacements available on the diagonals, so D and Theta are
    coupled and this contingency table detects it.
    """
    delta = rel - orig
    d = np.linalg.norm(delta, axis=1)
    theta = np.arctan2(delta[:, 1], delta[:, 0])
    table, _, _ = np.histogram2d(
        d, theta, bins=[np.quantile(d, np.linspace(0, 1, bins + 1)),
                        np.linspace(-np.pi, np.pi, bins + 1)])
    chi2, p, _, _ = stats.chi2_contingency(table + 1e-9)
    return {"test": "independence", "chi2": float(chi2), "p": float(p),
            "passes": bool(p > 0.01)}


def per_subject_stability_test(subject_ids: np.ndarray, orig: np.ndarray,
                               rel: np.ndarray) -> dict:
    """Test 4 — one displacement per subject, not one per record.

    Independent draws for repeated records let an adversary average them:
    n records collapse the effective radius by a factor of sqrt(n).
    """
    delta = rel - orig
    within, counts = [], []
    for sid in np.unique(subject_ids):
        m = subject_ids == sid
        if m.sum() < 2:
            continue
        within.append(delta[m].var(axis=0).mean())
        counts.append(int(m.sum()))
    if not within:
        return {"test": "per_subject_stability", "passes": True, "note": "no repeats"}
    ratio = float(np.mean(within) / delta.var(axis=0).mean())
    return {"test": "per_subject_stability", "within_between_ratio": ratio,
            "max_records_per_subject": max(counts),
            # Near zero is what we want: the same subject always moves the
            # same way, so averaging their records recovers nothing.
            "passes": bool(ratio < 0.05)}


def lattice_test(rel: np.ndarray, min_residues: int = 1000) -> dict:
    """Test 5 — rounding after jitter quantises the output onto a grid."""
    residues = np.unique(np.round(np.modf(rel[:, 0])[0], 9))
    return {"test": "lattice", "distinct_residues": int(len(residues)),
            "passes": bool(len(residues) >= min_residues)}


def density_correlation_test(orig: np.ndarray, rel: np.ndarray,
                             local_density: np.ndarray) -> dict:
    """Test 6 — displacement must not depend on where the point started."""
    d = np.linalg.norm(rel - orig, axis=1)
    rho, p = stats.spearmanr(d, local_density)
    return {"test": "density_correlation", "spearman_rho": float(rho),
            "p": float(p), "passes": bool(abs(rho) < 0.05)}

The dashed circle is the claimed radius in both panels. The right-hand ellipse is what the code actually produced.

Verification permalink

Wire all six into one gate and fail the build on any of them:

def jitter_gate(subject_ids, orig, rel, radius, local_density) -> dict:
    """All six, with the failure names in the output so CI reports the cause."""
    results = [
        radial_cdf_test(orig, rel, radius),
        bearing_uniformity_test(orig, rel),
        independence_test(orig, rel),
        per_subject_stability_test(subject_ids, orig, rel),
        lattice_test(rel),
        density_correlation_test(orig, rel, local_density),
    ]
    failed = [r["test"] for r in results if not r["passes"]]
    return {"results": results, "failed": failed, "passes": not failed}

The gate must reject something. A check that has never failed proves nothing, so feed it input it must refuse — this is the part most teams omit:

def test_gate_rejects_uniform_radius():
    """Deliberately broken jitter: r ~ U(0,R) instead of R*sqrt(U)."""
    rng = np.random.default_rng(0)
    n, R = 20_000, 300.0
    orig = rng.uniform(0, 10_000, size=(n, 2))
    theta = rng.uniform(0, 2 * np.pi, n)
    r = rng.uniform(0, R, n)                      # the bug
    rel = orig + np.c_[r * np.cos(theta), r * np.sin(theta)]

    out = jitter_gate(np.arange(n), orig, rel, R, np.ones(n))
    assert "radial_cdf" in out["failed"]

def test_gate_rejects_degree_space_noise():
    rng = np.random.default_rng(1)
    n = 20_000
    orig = np.c_[rng.uniform(-1, 1, n), rng.full(n, 55.0)]
    sigma_deg = 100 / 111_320
    rel = orig + rng.normal(0, sigma_deg, size=(n, 2))   # isotropic in degrees
    metres = np.c_[rel[:, 0] * 111_320 * np.cos(np.radians(55)),
                   rel[:, 1] * 111_320]
    orig_m = np.c_[orig[:, 0] * 111_320 * np.cos(np.radians(55)),
                   orig[:, 1] * 111_320]
    out = bearing_uniformity_test(orig_m, metres)
    assert not out["passes"]

Both tests should be red before the fix and green after. If a rejection test passes on unbroken input as well, the threshold is too loose and the gate is decorative.

Run the suite on the released artefact, not on the function’s return value. Serialisation is where quantisation happens — a ROUND(lon, 4) in the export SQL undoes the jitter after every in-process test has passed.

The third defect is the one that motivates running the suite against the exported file rather than the function output.

Edge Cases & Adjustments permalink

Small samples. Below a few thousand points the KS test lacks power to detect a 20 % median error. Report the observed median against the expected one alongside the p-value, because the point estimate is informative when the test is not.
Deliberately anisotropic jitter. Some releases jitter more along a road than across it, on purpose. Then test 2 should fail, and the gate needs the expected anisotropy as a parameter rather than a pass/fail on uniformity.
Constrained jitter. Points near boundaries deviate from the radial law by design. Run tests 1–3 on interior points only, and cover the constrained ones with the acceptance-mass check instead.
Test 6 on genuinely non-uniform designs. Density-dependent jitter — larger displacements in sparse areas — is a legitimate design, and it fails test 6 correctly. Parameterise the expected relationship rather than removing the test.
You do not always have the originals. Test 1, 4 and 6 need the true positions. Run the gate inside the pipeline where both are available, and export only the summary statistics — the test outputs are safe to publish and the inputs are not.

FAQ permalink

Which single test would I keep if I could keep one?

Test 4, per-subject stability. The others degrade the guarantee; independent re-jitter destroys it, and it is invisible in every single-record inspection.

Is a KS threshold of 0.01 too strict?

For 40 000 points it is about right — the critical value at $\alpha = 0.05$ is roughly 0.0068. For 1 000 points use 0.043. Scale the threshold with $1.36/\sqrt{n}$ rather than fixing it.

Why test independence separately when both marginals pass?

Because uniform-on-a-square jitter has correct-looking marginals and correlated joint structure: the maximum displacement available at 45° is $\sqrt{2}$ times that at 0°. Only the joint test sees it.

Should these run on every release or once at implementation?

Every release. The lattice failure in the worked example was introduced by a change to the export query months after the jitter code was verified.

Coordinate Jittering & Noise Injection Methods — the mechanisms under test
Preserving Topology When Fuzzing Points Near Boundaries — where deviation from the law is intended
Automating Spatial Privacy Checks in CI — making the gate blocking
Implementing Planar Laplace Noise in Python — the Rayleigh-law variant of test 1
Writing Pytest Assertions for K-Anonymity Guarantees — the same rejection-test discipline

← Back to Coordinate Jittering & Noise Injection Methods

Testing Jitter Quality with Displacement Distribution Checks

Core Calculation # permalink

Worked numeric example # permalink

Python Implementation # permalink

Verification # permalink

Edge Cases & Adjustments # permalink

FAQ # permalink

Related # permalink

Related topics