Privacy-Preserving Personalisation: Technologies and Trade-Offs

Privacy-preserving personalisation is not one technology, and it is not a claim that identifiable customer data has gone away.

It is an architectural approach. A team defines the decision it has to make, removes the data and outputs that decision does not require, identifies which parties must not see what remains, and then selects technical controls that address those specific disclosure risks.

In practice that might mean producing campaign statistics without exposing individual activity, training a recommendation model without centralising raw device histories, comparing two organisations’ customer lists without exchanging them, or proving that a customer meets an eligibility condition without collecting their date of birth.

The right method depends on what must stay private, from whom, and at which point in the system. Aggregation, differential privacy, federated learning, secure aggregation, clean rooms, synthetic data and confidential computing solve different problems. Some provide formal mathematical or cryptographic guarantees under stated assumptions. Others simply reduce how much data exists in a given place.

That distinction is the whole subject. Federated learning is not automatically private. A clean room is not automatically anonymous. Pseudonymisation does not take data outside data-protection law. Synthetic data can reproduce information about its training records. Ordinary noise is not differential privacy.

The ICO describes privacy-enhancing technologies as tools that can support data minimisation and information security, while stressing that their risks and benefits depend on how they are implemented. That guidance was published in June 2023 and is currently under review following the Data (Use and Access) Act 2025. (1)

What privacy-preserving personalisation means

A personalisation system usually contains several stages: collecting or observing signals, joining those signals to an identity or device, deriving features, training or updating a model, selecting an experience, delivering it, measuring the response, and retaining the data, logs and model artefacts that result.

A privacy-preserving design interrogates each stage. Does this step need personal data? Must the data leave its current environment? Does an exact value need to be retained? Does the recipient need a row-level result? Could the decision be made locally? Could the output be a count, a range, a score or a yes or no? Who could infer something from model updates, logs or repeated queries? What happens when the output is combined with information the recipient already holds?

A system can reduce one exposure while leaving another untouched. Training on a phone stops raw interaction histories being copied to a central server, but an unprotected model update may still reveal what was on the phone. Encrypting a joint computation protects the inputs from the party computing them, while the final output may still describe an audience of three people. Generating synthetic records separates later development work from the original database, but a high-fidelity generator can memorise unusual rows.

Privacy-preserving personalisation therefore has two parts. The first is data-minimising architecture: changing what is collected, centralised, retained or disclosed. The second is demonstrable protection: showing that the chosen statistical, cryptographic or system mechanism actually delivers the claimed property under a defined threat model.

Neither part establishes a lawful, fair and transparent purpose. A PET does not supply a lawful basis. (2) It does not make an incompatible purpose compatible, and it does not make an unfair targeting practice fair. (3)

Legal implementation, including lawful basis, notices, consent records, cookies, direct-marketing rules and data-subject rights, belongs in the GDPR and CCPA compliance guide. Whether a purpose is justified at all belongs in ethical consumer-data use.

Start with the data and the decision you do not need

Technology selection should not begin with a vendor category. It should begin with the smallest decision and the least detailed output that can support the objective.

Suppose a campaign team asks for a table of every customer who saw an advert and later purchased. That single request contains at least four different decisions: estimate aggregate reach, estimate incremental purchases, cap exposures per customer, or suppress existing purchasers from the campaign.

Each needs different information. Aggregate reach may need only protected counts. Incrementality may need random assignment and aggregate outcomes rather than a reusable customer-level file. Frequency management may need a local or pseudonymous counter with short retention. Suppression may need a private set-intersection protocol that reveals matches without disclosing either party’s full list.

So the sequence is: state the decision, define the least detailed output that supports it, identify the data strictly needed to produce that output, delete the unnecessary fields and intermediate artefacts, and only then ask whether a PET is required. No PET compensates for collecting data the decision never needed.

I sat in a great many first meetings where the budget for advanced personalisation had already been approved. The stated reasons were always user experience and revenue. The tell came when I asked what was wrong with the current setup, and the room went quiet. They knew something was broken, because growth and performance told them so, but they could not name it. What they were mostly buying was the thing an advanced digital marketing team is supposed to own. That is a commercial observation rather than a technical one, and it now applies to privacy-enhancing technology exactly as it applied to personalisation platforms.

Define the threat model

A privacy claim means very little until the protected data, the adversary and the assumptions are named.

Question	Examples
What requires protection?	Raw events, identity links, preference vectors, model gradients, audience membership, query results, credentials, keys
From whom?	Brand, publisher, clean-room operator, cloud provider, model coordinator, another participant, external attacker, insider, output recipient
At which stage?	Collection, transport, storage, matching, training, inference, release, deletion
What does the adversary already know?	Public records, its own customer database, previous query answers, model versions, timestamps, auxiliary identifiers
Can parties collude?	Two clean-room participants, several query users, clients and server, operator and recipient
Passive or active?	Merely observing, or manipulating inputs, clients, models, code or query sequences
What output is allowed?	Exact count, noisy count, aggregate model, yes or no, cohort, score, personalised item
What failure is unacceptable?	Singling out, membership inference, reconstruction, attribute inference, unauthorised linking, discriminatory use, group harm

Data is encrypted is not a threat model. It does not say who holds the keys, whether data is decrypted during processing, whether the operator can alter the code, or what the output discloses. Data never leaves the device is not one either. A server may still receive a stable identifier, a model update, the selected content, a diagnostic event or a detailed response.

I spent three seasons refereeing, where every decision is made in public and none can be withdrawn. What that taught me is that a wrong call stays survivable if the reasoning was stated at the time, and becomes indefensible if it was not. The decisions I made on instinct were the ones I could not account for afterwards. A written threat model does the same work. It will sometimes turn out to be wrong, and then the assumption that failed is visible and fixable. An undocumented judgement that turns out wrong just looks like carelessness, because nobody can reconstruct what was assumed.

What each technology actually protects

Technique	Suitable problem	Formal guarantee	Main trade-off	Common misuse
Aggregation	Group-level reporting	Usually none by itself	Loss of granularity	Assuming a small group cannot identify its members
Differential privacy	Limiting one record’s influence on an output or model	Formal, if mechanism and parameters are correctly specified	Privacy budget against utility	Calling ordinary noise or suppression differential privacy
Federated learning	Training without centralising raw records	None by itself	Systems complexity and update leakage	Assuming model updates reveal nothing
Secure aggregation	Stopping a coordinator reading individual updates	Protocol-specific cryptographic guarantee	Dropout handling and operational complexity	Confusing update privacy with model privacy
Data clean room	Controlled cross-party measurement or matching	Usually contractual, access and technical controls combined	Restricted queries, dependence on operator design	Treating controlled access as anonymity
Synthetic data	Development and testing without exposing source records	None automatically	Fidelity, bias and memorisation risk	Assuming synthetic means non-personal
Secure computation	Joint computation without exposing raw inputs	Protocol-specific	Cost, latency, restricted operations	Ignoring what the output reveals
On-device processing	Local ranking, features, eligibility	Architectural reduction only	Device resources and observability	Returning detailed local results as telemetry
Trusted execution environment	Plaintext processing isolated from the operator	Hardware and attestation specific	Trust in hardware, side channels, memory limits	Treating an enclave as protection against output leakage
Homomorphic encryption	Computation on encrypted values	Scheme-specific cryptographic security	Substantial cost, restricted operations	Assuming encryption constrains the released result
Pseudonymisation	Separating identity from operational records	No anonymity guarantee	Key and mapping management	Treating tokens as anonymous data
Selective-disclosure credentials	Proving an attribute without a full identity record	Protocol and suite specific	Issuer ecosystem, revocation, correlation risk	Reusing stable identifiers that allow tracking

Aggregation and minimum group sizes

Aggregation replaces individual observations with totals, rates, averages or distributions. It is often the right first move for campaign reporting, because most reporting decisions do not need customer-level output.

Aggregation alone is not a formal guarantee. A group of five customers stays identifiable when the recipient knows four of them. A total can reveal an individual through differencing between two overlapping reports. Repeated filters narrow an audience until one person remains. A rare combination of geography, product and behaviour singles someone out even with direct identifiers removed.

There is no universal minimum cohort size that makes every output safe. Appropriate controls depend on the recipient’s auxiliary information, the number of available dimensions, query history, sensitivity and release context.

Official statistical practice layers several measures rather than relying on a threshold. For Census 2021 the Office for National Statistics used record swapping, cell-key perturbation and disclosure rules, with stronger protection for sparse outputs. (4) Its wider statistical disclosure control policy treats disclosure risk as a case-by-case balance rather than something a single minimum count can eliminate. (5)

For personalisation reporting, pair aggregation with limits on dimensions and arbitrary filters, suppression or perturbation of sparse outputs, controls against differencing across reports, a retained query history, a check on whether recipients hold matching customer-level data, export restrictions, and simulated attacks using realistic auxiliary data.

Aggregation protects the released view. It does not protect the source database, which still needs access control, retention rules and the security measures covered in secure personalisation under GDPR.

Pseudonymisation and tokenisation

Pseudonymisation replaces or transforms direct identifiers and keeps the information needed to restore the association separately. Tokenisation usually replaces an identifier with a random or cryptographically derived token.

These reduce exposure inside feature stores, analytical environments and activation workflows. They can stop routine analysts and suppliers seeing names, addresses or account numbers. They do not make data anonymous.

A stable token still supports a detailed behavioural history. A recipient may recognise someone from a pattern of transactions, locations or rare interests. The token service, mapping table, key or matching process can restore the link. Even without reversing the token, the system still makes decisions about the same person over time.

The ICO states that pseudonymised personal data remains within data-protection law, that the additional attribution information must be kept separately and protected, and that pseudonymisation does not by itself permit a new purpose. (6)

A sound implementation uses different tokens for different domains where cross-context linking is unnecessary, keeps the mapping or key in a separately administered system, applies strict access and rotation controls, prevents raw identifiers leaking through logs and exports, assesses whether behavioural attributes allow recognition without the mapping, and sets deletion rules for both tokens and re-identification material.

Pseudonymisation protects identity association from specified parties. It does not prevent profiling, attribute inference, unfair decisions or group harm.

Differential privacy

Differential privacy limits how much the output of a computation can change when one protected unit is added to or removed from the input.

The protected unit has to be specified. It might be one event, one customer, one household, one device, or every contribution a customer made in a defined period. Protecting a single event is far weaker than protecting the whole history of someone who generated hundreds.

A credible claim identifies the unit of privacy, the relationship between neighbouring datasets, the mechanism, epsilon and where applicable delta, how contributions are bounded or clipped, the sensitivity calculation, whether the model is central or local, the privacy accountant and composition method, every release charged to the budget, the treatment of repeated training or queries, and which parties are trusted.

NIST describes differential privacy as a mathematical framework for quantifying privacy loss, and sets out the hazards that arise as that framework is translated into software and deployment. Its central point is that the guarantee can be undermined between the definition and the running system. (7)

Central and local differential privacy

Under central differential privacy a trusted curator sees the unmodified records and applies a private mechanism to the output or the model. This generally achieves more utility for a given level of protection, but the curator and the source system remain high-value targets.

Under local differential privacy each device randomises its contribution before sending it. The collector never receives the exact input, but substantially more data or noise is needed to obtain a useful population estimate. The ICO notes that the local model generally produces less accurate information than the central model for comparable protection, and that any original information retained by a party remains personal data in that party’s hands. (8)

Differential privacy in recommendation systems

Differential privacy can be applied during aggregate learning or model training, so that the learned correlations depend less on any one customer. It can also protect reporting outputs.

It does not usually mean the final recommendation is independent of the person receiving it. A system may train a population model with differential privacy and then combine that model locally with the recipient’s own history.

McSherry and Mironov applied exactly that separation to selected Netflix Prize algorithms. Their adaptations required sensitivity analysis, bounded contributions and post-processing, and they reported useful empirical performance in that setting. They also stated that differentially private algorithms necessarily introduce uncertainty and trade accuracy for privacy. (9)

That result supports the proposition that useful recommendation models can sometimes be built with formal guarantees. It does not support a promise of negligible cost across every model, dataset, privacy budget or objective.

Governing a privacy budget

A production deployment needs an accountable owner for the definition and the budget, a registry of every release, model run and repeated query, technical enforcement that blocks unbudgeted queries, independent validation of the implementation and its randomness, documented clipping and contribution limits, utility tests across important customer groups, controls against analysts reconstructing information through multiple interfaces, and a plan for what happens when the cumulative budget is exhausted.

Random noise, rounding or suppression should not be described as differential privacy unless the complete mechanism implements the stated definition.

Local and on-device processing

Local processing keeps a decision, feature calculation or ranking on the customer’s device. It suits ranking a downloaded catalogue against recent local behaviour, calculating a short-lived affinity vector without uploading the underlying events, maintaining a frequency counter, checking a contextual condition, choosing among several already-authorised messages, or adapting an interface using local settings.

The value is architectural: information that is never transmitted cannot be exposed through the central store in raw form. That is a reduction, not a guarantee. A design still has to inspect what model, content or rules are sent to the device, what result is returned, whether a stable device identifier accompanies it, whether diagnostics carry local attributes, whether the server can extract information indirectly through repeated queries, how local storage is protected, how deletion and reset work, and whether compromised devices can poison shared systems.

On-device ranking suits cases where the server needs to know only which item was delivered, or nothing at all. It is pointless when the chosen item, the full score list and every contributing feature are uploaded immediately afterwards.

Federated learning

Federated learning trains a shared model by sending model code or parameters to distributed data holders, calculating updates locally and combining them.

The foundational paper evaluated iterative model averaging across five model architectures and four datasets, leaving raw training data on devices while communicating updates, with communication cost as the principal constraint. (10)

Federated learning addresses data location. By itself it provides no formal privacy guarantee for the training records. Model updates may expose whether a record participated, labels or sensitive attributes, representative features, rare text or images, properties of a client’s local dataset, or enough information to reconstruct training examples. Gradient-reconstruction research has demonstrated recovery of training data from shared gradients in experimental settings. (11)

The ICO therefore recommends considering federated learning alongside other PETs such as secure aggregation, secure multiparty computation, homomorphic encryption or differential privacy. (12)

A federated system needs to state whether the coordinator is trusted, whether participants can see one another’s updates, the minimum aggregation population, whether clients can collude, how updates are authenticated, how poisoned updates are detected, whether individual updates are retained, whether the global model is tested for memorisation and membership inference, whether participation is protected with differential privacy, and whether device, timing and network metadata are themselves revealing.

Federated learning suits cases where raw data cannot or should not be centralised and where the coordination cost is justified. It is not automatically better than training a smaller model on properly minimised central data.

Secure aggregation

Secure aggregation is a cryptographic protocol that lets a coordinator learn the aggregate of many client updates without learning any individual update. The 2017 protocol by Bonawitz and colleagues was designed for high-dimensional federated updates including client dropout, and states its security properties for honest-but-curious and malicious-server settings under the protocol’s assumptions. (13)

It protects an update from the aggregation server when enough participants take part and the assumptions hold. It does not necessarily protect the final aggregate, the trained model, a very small aggregation group, information revealed across repeated aggregates, client metadata, a participant whose update can be isolated through manipulation or dropout, or data already exposed on the client.

Deployment requires a protocol and implementation review, explicit collusion and dropout assumptions, minimum participation thresholds, authenticated clients, key establishment and recovery procedures, resistance to malicious input, controls preventing the coordinator constructing isolating cohorts, monitoring of failed rounds, and leakage testing of the final model.

Secure aggregation is not full federated-learning privacy. It solves the narrower problem of concealing individual contributions during aggregation.

Trusted execution environments

A trusted execution environment is a hardware-backed isolated area where code can operate on plaintext while the operating system, hypervisor or cloud operator is restricted from reading or altering the protected memory and code. Remote attestation lets a participant verify which code is running before releasing a key or a dataset.

The ICO describes confidentiality, data integrity and code integrity as the central properties, while noting scalability and implementation risks. (14)

The guarantee depends on the hardware and firmware, the attestation chain, the exact measured code, key provisioning, patch management, side-channel resistance, memory and input-output boundaries, who can alter the surrounding orchestration, and what leaves the enclave.

A TEE processes plaintext internally. It does not reduce the detail of an authorised output. A query returning a one-person audience is still identifying even when the computation ran inside a correctly attested enclave.

Secure multiparty computation

Secure multiparty computation lets two or more parties calculate an agreed function over their combined inputs without revealing all inputs to one another. A publisher and an advertiser can compute an audience overlap, a conversion count or an aggregate value without exchanging their customer tables.

The ICO explains that this can limit what parties learn beyond the configured output, while warning that the output itself may remain personal information and that some operations are computationally expensive or hard to deploy at scale. (15)

An assessment should document the exact function, the parties and permitted coalitions, the adversary assumptions, input validation, the protocol and its security proof, failure and abort behaviour, the output recipients, output disclosure controls, computational and communication benchmarks, and whether repeated computations enable inference.

Secure computation protects inputs under protocol assumptions. It does not make an unrestricted output safe.

Homomorphic encryption

Homomorphic encryption allows selected computations while data remains encrypted, and the party performing the computation may not hold the decryption key. Partially homomorphic schemes support restricted operations; fully homomorphic encryption supports more general computation at much greater cost.

The ICO notes that homomorphic encryption can preserve exact computational results while protecting inputs during processing, and that fully homomorphic encryption may be orders of magnitude slower than plaintext computation, which keeps it impractical for many workloads. (16)

It may fit limited aggregate calculations, selected scoring functions, encrypted matching components, outsourced analysis where the processor must not see inputs, and building blocks inside private set-intersection or federated systems. It is usually unsuitable for complex low-latency recommendation serving unless the operation and scale have been benchmarked realistically.

Key ownership is decisive. A system does not protect data from a party that also controls every decryption key. And as with secure computation and enclaves, encrypting the calculation places no constraint at all on how revealing the result is.

Data clean rooms

A data clean room is a controlled environment for cross-party matching, measurement or analysis. It is not one architecture. A given clean room may combine isolated storage, pseudonymous matching, private set intersection, secure computation, trusted execution environments, homomorphic encryption, approved query templates, minimum audience sizes, differential privacy, output review, contractual restrictions and audit logs.

The question is never whether the vendor calls it a clean room. It is what each participant, the operator and the output recipient can learn.

The ICO’s guidance on private set intersection states the underlying principle. Two parties can establish the intersection of their datasets without revealing the datasets themselves, each participant should receive only the minimum information the purpose requires, and some implementations return aggregate statistics rather than the matched records. It also warns that small intersections, intersection sums and repeated analysis can still permit singling out or inference, so thresholds and further safeguards may be needed. (17)

An ICO case study on trusted third parties for market insights applies the same logic to marketing data held by different controllers. A retailer combining a loyalty dataset with a separate market-view dataset keeps the datasets apart, prevents row-level joining, and receives aggregated group-level information rather than customer-level detail. (18) That example deliberately avoids person-level matching, so it is a model for insight work rather than for clean rooms built to establish matches.

IAB Tech Lab’s clean-room guidance tells buyers to examine a clean room’s actual privacy-preservation capabilities and limitations. Its own specification portfolio shows how fast this area moves: an earlier join-and-activation protocol was deprecated in July 2025 and superseded, while other specifications were finalised during 2025. It is industry material produced with substantial vendor participation, not an independent standard for privacy guarantees. (19)

Useful procurement questions: are raw inputs visible to the operator; how are identities matched; are hashes salted or keyed, or vulnerable to dictionary matching; can participants submit arbitrary rows or queries; what minimum audience rules apply; can one party run a differencing attack; are outputs exact, suppressed or differentially private; can participants export row-level results; is code attested or only contractually controlled; who manages keys; how are collusion and operator access handled; are logs independently reviewable; what happens to source, intermediate and output data after the job; and can the vendor demonstrate any of this independently.

Controlled access is valuable. It is not anonymity.

Synthetic data

Synthetic data is generated to reproduce selected statistical properties of source data rather than copying the source rows. It can reduce routine access to personal data during development, pipeline testing, demonstrations, analyst training, early prototyping, quality assurance and some external collaboration.

The privacy question is whether information about real people can be inferred from the generated records or from the generator. A high-capacity model may memorise rare or outlying examples. A synthetic dataset may preserve a unique combination of attributes. Membership-inference or model-inversion attacks may reveal whether someone contributed to training. Bias and under-representation in the source data are reproduced too.

The ICO states that higher fidelity can increase disclosure risk, that synthetic data is not automatically anonymous, and that model inversion, membership inference, attribute disclosure and outlier risks should be assessed. (20)

Evaluation should cover fidelity, including distributions, correlations, rare categories, temporal structure, downstream task performance, calibration across subgroups and whether the intended analysis reaches the same decision. It should cover privacy, including exact and near-duplicate detection, distance to source records, membership and attribute inference tests, model extraction, outlier exposure, tests with realistic auxiliary information, and whether differential privacy was applied during training. And it should cover governance: who can access the source data and the trained generator, who receives the release, permitted uses, retention, documented limitations, repeat-release risk, and re-evaluation after retraining.

Synthetic data can be an effective separation layer. It is not anonymous merely because its rows were generated.

Privacy-preserving identity and credentials

Many customer experiences do not need identity. They need proof of an attribute: that a customer is over a required age, belongs to a loyalty tier, lives in an eligible area, holds a valid entitlement, has not already redeemed a discount, or falls into a permitted account class. Collecting a full identity record for those decisions creates exposure that the decision never required.

Verifiable credentials and selective-disclosure protocols let an issuer attest an attribute and a holder present only the required claim. The W3C Verifiable Credentials Data Model recommends abstract claims such as an age-over assertion instead of a complete date of birth, and supports selective-disclosure designs. (21) At the protocol level, RFC 9901 defines an IETF Standards Track mechanism for selective disclosure of individual JSON Web Token claims, with optional holder key binding to prevent a presented credential being replayed by someone else. (22)

Credential systems still need a threat model. Privacy can be undermined by stable subject or credential identifiers, issuer visibility at presentation time, reusable signatures, verifier collusion, status or revocation checks, wallet telemetry, network metadata, overbroad verifier requests, and credentials carrying more attributes than the transaction needs.

Verification establishes that a signed claim came from an accepted issuer and satisfies the protocol. It does not prove the claim was true when issued, that the verifier’s purpose is fair, or that the resulting decision is appropriate.

Matching the technology to the problem

Use case	Minimum output	Appropriate approaches	Usually inappropriate
Aggregate campaign measurement	Counts, rates, uncertainty, aggregate incremental effect	Aggregation, disclosure control, differential privacy; clean room or secure computation for cross-party joins	Exporting joined customer-level exposure and purchase histories to every participant
Frequency management	A bounded counter or a cap-reached result	On-device counters, short-lived pseudonymous tokens, purpose-specific identifiers, private matching	Centralising full browsing and purchase histories in order to count impressions
Recommendation training	A model meeting utility and leakage requirements	Feature minimisation, central differential privacy, federated learning with secure aggregation and possibly differential privacy	Treating federated learning alone as a guarantee; releasing individual gradients
On-device ranking	The selected item, or nothing returned	Local feature calculation and ranking, downloaded population model, short-lived context	Returning the full local feature vector, score list or detailed telemetry
Cross-party collaboration	Overlap count, aggregate conversions or an authorised cohort	Private set intersection, secure computation, enclave-based clean room, query controls, possibly differential privacy	Plain unsalted hashes, unrestricted queries, row-level exports
Development and testing	Representative test records	Synthetic data, masked templates, tightly controlled samples where necessary	Copying production databases into development environments
Eligibility verification	A yes or no, or one limited attribute	Selective-disclosure credentials, abstract claims, zero-knowledge or blinded proofs	Collecting a passport, date of birth and address for a simple age check

The correct design often uses several techniques together. A federated recommendation system may combine local training, secure aggregation, user-level differential privacy and a centrally distributed model. A clean-room measurement workflow may combine private matching, an enclave, minimum cohort sizes, query logging and differential privacy on exported statistics.

Combining technologies does not combine their guarantees. The end-to-end claim is limited by the weakest stage.

Measure privacy and utility together

A privacy-preserving system needs separate, measurable requirements in three dimensions.

Privacy requirements may include differential-privacy parameters and composition limits, protocol security assumptions, minimum aggregation populations, a maximum acceptable probability of successful membership inference, re-identification test results, maximum output granularity, which parties cannot access plaintext, the permitted collusion threshold, attested code identity, a credential unlinkability target, and deletion of source, intermediate and derived artefacts.

Utility requirements may include model accuracy, ranking or calibration, performance by customer group, absolute and relative error in reported counts, bias in small segments, stability over time, decision agreement with an unprotected reference analysis, statistical power, error rates, and the business decision the output actually supports.

Systems requirements may include training and query latency, communication volume, device memory and battery use, cryptographic computation cost, failure and dropout rates, job completion, key-recovery performance, model-update cadence, throughput, supplier cost and operational staffing.

That last item is the one that gets left out, and I have watched it decide things. The most persistent complaint I heard in quarterly reviews across ten years was that the client’s own team spent too much time operating the tool. I never fully solved it, partly because the complaint is often about internal positioning as much as the software. It is still information. Privacy-enhancing technology sharpens it, because these systems add cryptographic review, budget accounting, attestation checks and failed rounds to somebody’s week. Commercially, staff time is the cost most often missing from the business case, and it tends to arrive as a surprise.

An assessment should never conclude only that a protected model has similar accuracy. It should show the baseline, the exact metric, the test population, the uncertainty, the subgroup results, the privacy parameters, every modelling change, the computational cost, and whether the difference affects the intended decision.

For causal questions, such as whether personalisation increased sales at all, use a proper experiment. The guide to measuring personalisation effectiveness covers assignment, exposure, incrementality and instrumentation. PET evaluation answers whether the protected system works as designed. It says nothing about the commercial effect of personalisation.

Test leakage and re-identification

Testing should mirror the threat model rather than produce a generic privacy score.

On data and outputs, search for exact and approximate source-record matches, attempt linkage using information a realistic recipient holds, test differencing between overlapping reports, probe sparse cohorts and rare attribute combinations, submit repeated and adaptive queries, check whether errors, previews, logs or exports bypass output controls, and test whether a participant can insert known records and spot them in a result.

On models, run membership-inference attacks, test reconstruction and gradient inversion where updates are exposed, search outputs for memorised rare sequences, evaluate attribute inference, compare leakage before and after quantisation, clipping, aggregation or noise, test malicious clients and poisoning, and assess whether successive model versions leak through differencing.

On identity, determine whether tokens are stable across contexts that do not require it, test dictionary attacks against hashed identifiers, inspect whether timestamps and network metadata restore identity, check whether credential presentations can be correlated, and examine whether status checks tell issuers where credentials are being used.

On infrastructure, verify enclave attestation and code identity, test key custody, rotation, recovery and revocation, review operator and administrator access, examine backups, queues, temporary files and observability systems, trigger participant dropout and protocol failure, and test whether fail-open behaviour exposes unprotected data.

Passing a test does not prove anonymity. It provides evidence against specified attacks under specified conditions, and the results expire when the data, model, query interface, recipients or surrounding information change.

A twelve-step evaluation framework

1. Define the decision and the minimum output that supports it.
2. Define the threat model: protected data, adversaries, capabilities, auxiliary information, collusion and unacceptable outcomes.
3. Identify which parties should not see raw data, including the operator, cloud provider, coordinator, analyst and other participants.
4. Determine whether centralisation is necessary at all.
5. Select a technique that addresses the actual threat. Do not choose federated learning for an output-disclosure problem, differential privacy for key theft, or a clean room merely because several organisations are involved.
6. Define measurable privacy, utility and systems requirements.
7. Test leakage and re-identification using realistic attacks and side information.
8. Validate analytical performance against an appropriate baseline, with uncertainty and subgroup effects reported.
9. Document the legal and ethical assumptions separately: purpose, necessity, lawful basis, reasonable expectations, fairness, transparency and potential group harms.
10. Establish monitoring and incident handling, including containment, key revocation and model withdrawal.
11. Reassess when data, models, vendors or query interfaces change.
12. Stop if the claimed guarantee cannot be demonstrated.

Something I learned selling to large enterprises transfers directly to step twelve. What kills a deal at legal, procurement or security review is rarely disagreement. It is a spot left even slightly ambiguous. Those organisations sense the gap immediately and read it as risk that could rebound on them, and vendors consistently misdiagnose it because their own companies have no comparable bureaucracy to empathise with. Turn that instinct on your own privacy claim. If a step above cannot be answered precisely, that is not a documentation gap to tidy up later. It is the part a reviewer will find first.

Evaluate vendor privacy claims

The ICO’s work on PET adoption identifies inconsistent terminology, limited evidence, uncertain maturity, unclear costs and information asymmetry between suppliers and buyers as barriers to responsible adoption. (23)

A procurement process should require evidence rather than accept phrases such as data never moves, military-grade encryption, fully anonymised or GDPR compliant.

Make the vendor define the claim

What exact privacy property is claimed? Which data and intermediate artefacts receive it? From which parties? Under what adversary and collusion assumptions? Is it mathematical, cryptographic, hardware-based, architectural, contractual or organisational? Which source, standard or proof defines it? Which components fall outside the claim?

Make them describe the architecture

Where is plaintext present? Who controls the keys? What can the operator see? What identifiers and metadata are retained? How are inputs matched? What leaves the protected environment? What ends up in logs, support tools, backups and observability systems? Which subprocessors are involved?

For differential privacy, require the unit of privacy, epsilon, delta, clipping, sampling assumptions, accountant and composition policy. For secure computation, require the protocol, adversary model, collusion threshold, input validation and abort behaviour. For enclaves, require the hardware, attestation, measured code, patching and side-channel position. For clean rooms, require the matching method, query language, minimums, output controls and operator access. For synthetic data, require the generator, source-data controls, privacy testing and evidence against memorisation.

Ask what happens when it fails

What happens if participation falls below the secure threshold? Can the system fail open? What happens when the privacy budget is exhausted? Can an administrator export raw rows? How are compromised keys revoked? Can the operator change code without new attestation? How are models withdrawn? Can data and derived artefacts be deleted on exit? Which guarantees disappear if participants collude?

I sold and serviced a platform marketed heavily on deep artificial intelligence and machine learning. What actually moved client results was predictive segmentation: clients chose a segment, the platform did the work behind it, and the measurable value appeared when those segments were pushed into the ad platforms. That is a genuinely useful capability. It is not what the word deep was doing in the pitch. I offer this as a decoder rather than a complaint. Ask what specific mechanism produces the outcome, then see whether the label survives the answer. Privacy-enhancing is now being used the way artificial intelligence was then, and the same question works on it.

A contract can preserve the stated configuration, audit rights, incident obligations, deletion, subprocessor control and change notification. It cannot turn a technically unsupported claim into a guarantee.

Monitor changes and incidents

Privacy protection degrades as the environment changes. Monitoring should cover cumulative privacy-budget consumption, query volume and repeated segmentation, cohort sizes, unusual export patterns, model-update and training populations, secure-aggregation failures, client dropout, attestation changes, key rotation and failed verification, synthetic-data drift, model memorisation indicators, utility and bias by group, new auxiliary datasets available to recipients, vendor releases and subprocessor changes, and unauthorised configuration changes.

Incident playbooks should distinguish source-data exposure, key compromise, operator access, query or output leakage, model extraction or inversion, malicious clients, re-identification, incorrect privacy accounting, invalid attestation, synthetic-data memorisation, and a use that has drifted outside the approved purpose. Containment may mean suspending queries, revoking keys, withdrawing a model, disabling an export, invalidating credentials or stopping the use case.

There is a behavioural pattern worth naming here, because I see it constantly in a neighbouring area. No brand I speak to tracks its own visibility inside AI assistants on its own initiative. The trigger is always external: a competitor’s result, or a report someone puts in front of them. The underlying belief is that being large, plus an investment made a couple of years ago, will carry them. Privacy monitoring fails in exactly the same shape. Budgets get consumed, cohorts shrink, vendors change subprocessors, and nobody looks until an outside event forces it. Scheduled review exists precisely because no internal signal will announce that a guarantee quietly stopped holding.

Limits of two frequently cited research models

Two pieces of research circulate in marketing writing about this subject in a stronger form than their authors claimed, and both were load-bearing in earlier versions of this article.

The first is a theoretical result on privacy-preserving recommendation. It analyses recommendation probabilities derived from preference rankings under a differential-privacy constraint, with interception of transmitted recommendations as the motivating threat, and proves that a coarse-grained threshold policy is optimal within that model in an asymptotic large-product setting. (24) It is a result about a specified mathematical model. It does not establish that interception during transmission is the main way personalisation systems leak, and it does not make a threshold policy a general recommendation architecture.

The second is a proposal for privacy-preserving training-data release, combining mixture-model approximation, kernel-inducing adaptation and Gaussian differential privacy, evaluated on selected classification tasks against selected synthetic-data baselines. (25) It is a preprint describing an experimental data-release workflow. Reported gains against those baselines are not a general personalisation benchmark, and the workflow is not an industry implementation methodology.

Both are worth reading. Neither should be used as the frame for a production decision, and any vendor or article presenting either as settled practice has over-read its source.

Implementation checklist

Before approving a privacy-preserving personalisation system, confirm that the decision and minimum output are documented; unnecessary raw fields and intermediate data have been removed; the protected unit and parties are defined; the threat model includes insiders, collusion, repeated outputs and auxiliary data; centralisation has been justified; the technique addresses the stated threat; formal claims include complete parameters and assumptions; architectural claims account for every outbound data flow; source data, keys, mappings and models have named owners; leakage tests reflect realistic attackers; utility is compared against a stated baseline with uncertainty; subgroup performance and group harms are assessed; outputs have minimum, query and export controls; lawful basis, fairness, purpose and transparency are documented separately; monitoring covers budgets, cohorts, models, keys and vendor changes; failure modes do not expose unprotected data; incident procedures include model and output leakage; supplier claims are supported by technical evidence; review repeats after material changes; and the system will be stopped if its claimed protection cannot be demonstrated.

Frequently Asked Questions

Conclusion

Privacy-preserving personalisation begins by refusing unnecessary data, not by buying a technology.

Define the smallest decision. Identify who must not see which data. Select a technique whose protection matches that specific threat. Then keep the distinctions straight: a formal guarantee is not a controlled environment, an encrypted calculation is not a safe output, and distributed data is not private model updates.

The most defensible systems usually combine methods. Local processing to avoid central collection, secure aggregation to hide client updates, differential privacy to limit individual influence, pseudonymisation to separate identity, and output controls to prevent small-group disclosure.

Those combinations cost something. They may reduce granularity, accuracy or analytical flexibility, and they may increase computation, latency, specialist staffing and supplier dependency. Evaluation does not make those costs vanish. It establishes whether the minimum useful decision can be supported at a demonstrable and proportionate level of exposure.

Technical controls also do not remove the need for customers to understand how personalisation affects them and to exercise real control, which is covered in the privacy-personalisation paradox. For the wider direction of data access and what may become available to analyse at all, see the future consumer-data landscape.

References

1. Information Commissioner’s Office. Privacy-enhancing technologies (PETs). Regulatory guidance. No reference number. New guidance announced 19 June 2023. Under review as at 28 July 2026 following the Data (Use and Access) Act 2025. Official regulatory guidance, not a technical standard or certification scheme. United Kingdom. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/data-sharing/privacy-enhancing-technologies/

2. Information Commissioner’s Office. A guide to lawful basis. Regulatory guidance. No reference number; relates to UK GDPR Article 6. No publication or update date shown on the page; accessed 28 July 2026. Guidance in this area is under review following the Data (Use and Access) Act 2025, which added a new recognised legitimate interests basis. United Kingdom. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/lawful-basis/a-guide-to-lawful-basis/

3. Information Commissioner’s Office. Principle (b): Purpose limitation. Regulatory guidance. No reference number; relates to UK GDPR Article 5(1)(b). No publication date shown; last updated 23 March 2026. No review banner shown. United Kingdom. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/data-protection-principles/a-guide-to-the-data-protection-principles/purpose-limitation/

4. Office for National Statistics. Protecting personal data in Census 2021 results. Official statistical methodology. No reference number. Last revised 9 March 2023. Written for census outputs; applied here by analogy to marketing reporting rather than as a rule for commercial data. United Kingdom. https://www.ons.gov.uk/peoplepopulationandcommunity/populationandmigration/populationestimates/methodologies/protectingpersonaldataincensus2021results

5. Office for National Statistics. Statistical disclosure control. Official data policy. No reference number. No publication or update date shown on the page; accessed 28 July 2026. Describes national-statistics practice; not a data-protection compliance standard. United Kingdom. https://www.ons.gov.uk/aboutus/transparencyandgovernance/datastrategy/datapolicies/statisticaldisclosurecontrol

6. Information Commissioner’s Office. Pseudonymisation. Regulatory guidance within the anonymisation guidance. No reference number. No publication or update date shown on the page; accessed 28 July 2026. Under review following the Data (Use and Access) Act 2025. United Kingdom. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/data-sharing/anonymisation/pseudonymisation/

7. Joseph P. Near and David Darais, edited by Naomi Lefkovitz and Gary Howarth. Guidelines for Evaluating Differential Privacy Guarantees. NIST Special Publication 800-226. DOI 10.6028/NIST.SP.800-226. Initial public draft December 2023; final publication 6 March 2025. Final, not draft. Not UK law; a United States federal technical publication. https://www.nist.gov/publications/guidelines-evaluating-differential-privacy-guarantees

8. Information Commissioner’s Office. Differential privacy. Regulatory technical guidance within the PET guidance. No reference number. Part of guidance created June 2023; under review as at 28 July 2026. United Kingdom. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/data-sharing/privacy-enhancing-technologies/what-pets-are-there/differential-privacy/

9. Frank McSherry and Ilya Mironov. Differentially Private Recommender Systems: Building Privacy into the Netflix Prize Contenders. Peer-reviewed conference paper, Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. DOI 10.1145/1557019.1557090. Published June 2009. Not UK law. Both authors were affiliated with Microsoft Research. Results are specific to the Netflix Prize dataset and to selected algorithms with non-trivial adaptations. https://www.microsoft.com/en-us/research/publication/differentially-private-recommender-systems-building-privacy-into-the-netflix-prize-contenders/

10. H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson and Blaise Agüera y Arcas. Communication-Efficient Learning of Deep Networks from Decentralized Data. Peer-reviewed conference paper, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, PMLR volume 54, pages 1273 to 1282. No DOI shown on the canonical page. Published April 2017. Not UK law; Google-affiliated research directly relevant to distributed learning products. https://proceedings.mlr.press/v54/mcmahan17a.html

11. Ligeng Zhu, Zhijian Liu and Song Han. Deep Leakage from Gradients. Peer-reviewed conference paper, Advances in Neural Information Processing Systems 32. No DOI shown on the canonical abstract. Published 2019. Not UK law. Reports experimental reconstruction attacks; attack performance varies with model, batching and defences. https://papers.nips.cc/paper_files/paper/2019/hash/60a6c4002cc7b29142def8871531281a-Abstract.html

12. Information Commissioner’s Office. Federated learning. Regulatory technical guidance within the PET guidance. No reference number. Part of guidance created June 2023; under review as at 28 July 2026. United Kingdom. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/data-sharing/privacy-enhancing-technologies/what-pets-are-there/federated-learning/

13. Kallista A. Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal and Karn Seth. Practical Secure Aggregation for Privacy-Preserving Machine Learning. Peer-reviewed conference paper, Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pages 1175 to 1191. DOI 10.1145/3133956.3133982. Published 2017. Not UK law; Google-affiliated research relevant to federated-learning infrastructure. Security properties hold only under the protocol’s stated assumptions. https://research.google/pubs/practical-secure-aggregation-for-privacy-preserving-machine-learning/

14. Information Commissioner’s Office. Trusted execution environments. Regulatory technical guidance within the PET guidance. No reference number. Part of guidance created June 2023; under review as at 28 July 2026. Official guidance, not a hardware certification. United Kingdom. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/data-sharing/privacy-enhancing-technologies/what-pets-are-there/trusted-execution-environments/

15. Information Commissioner’s Office. Secure multiparty computation (SMPC). Regulatory technical guidance within the PET guidance. No reference number. Part of guidance created June 2023; under review as at 28 July 2026. Official guidance, not a protocol standard. United Kingdom. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/data-sharing/privacy-enhancing-technologies/what-pets-are-there/secure-multiparty-computation-smpc/

16. Information Commissioner’s Office. Homomorphic encryption (HE). Regulatory technical guidance within the PET guidance. No reference number. Part of guidance created June 2023; under review as at 28 July 2026. Official guidance, not a cryptographic scheme specification. United Kingdom. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/data-sharing/privacy-enhancing-technologies/what-pets-are-there/homomorphic-encryption-he/

17. Information Commissioner’s Office. Private set intersection (PSI). Regulatory technical guidance within the privacy-enhancing technologies guidance. No reference number. No date shown on the individual page; the parent PET guidance collection was created on 19 June 2023. Carries the banner stating the guidance is under review following the Data (Use and Access) Act. Does not use data clean room as its organising concept and prescribes no universal disclosure level; the appropriate output depends on purpose, architecture, intersection size and re-identification risk. United Kingdom. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/data-sharing/privacy-enhancing-technologies/what-pets-are-there/private-set-intersection-psi/

18. Information Commissioner’s Office. Case study: trusted third parties for market insights. Illustrative case study within the anonymisation guidance. No reference number. No date shown on the individual case study; the parent anonymisation guidance was published on 28 March 2025. Carries the under-review banner, and states that it was developed in collaboration with Truata, a commercial data-privacy company. Non-statutory good-practice advice rather than a clean-room rule or a private set intersection specification. United Kingdom. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/data-sharing/anonymisation/case-studies-on-pseudonymisation-and-anonymisation-techniques/case-study-trusted-third-parties-for-market-insights/

19. IAB Technology Laboratory. Data Clean Rooms (browser title Data Clean Rooms Guidance). Standards and guidance landing page. No version number for the page itself. No publication date shown; last updated 22 July 2025. Statuses shown on the page: Data Clean Rooms Guidance and Recommended Practices v1.0 dated July 2024, although the PDF it links to states it was released on 5 July 2023 and its title carries a post public comment draft label; ADMaP v1.0 finalised February 2025; PAIR v1.1 finalised July 2025; an earlier join-and-activation protocol deprecated as of July 2025 and superseded by PAIR. IAB Tech Lab’s separate standards index still records PAIR at v1.0 in public comment, so the portfolio record is internally inconsistent. Not UK law and not an independent privacy standard; a member-backed advertising standards body whose members have a commercial interest in clean-room adoption. https://iabtechlab.com/datacleanrooms/

20. Information Commissioner’s Office. Synthetic data. Regulatory technical guidance within the PET guidance. No reference number. Part of guidance created June 2023; under review as at 28 July 2026. Official guidance, not a synthetic-data certification standard. United Kingdom. https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/data-sharing/privacy-enhancing-technologies/what-pets-are-there/synthetic-data/

21. World Wide Web Consortium. Verifiable Credentials Data Model v2.0. W3C Recommendation. Published 15 May 2025 and still the current Recommendation as at 28 July 2026; a version 2.1 Working Draft dated 11 May 2026 exists but does not supersede it. Not UK law; an international open standard. Specifies a data model and does not certify products or establish legal compliance. https://www.w3.org/TR/vc-data-model-2.0/

22. Daniel Fett, Kristina Yasuda and Brian Campbell. Selective Disclosure for JSON Web Tokens. RFC 9901, IETF Standards Track, Proposed Standard. DOI 10.17487/RFC9901; ISSN 2070-1721. Published November 2025. Not UK law; an international protocol specification. Two authors are affiliated with commercial identity providers, as disclosed in the RFC. The specification does not by itself close every correlation channel. https://www.rfc-editor.org/rfc/rfc9901.html

23. Information Commissioner’s Office. Tackling barriers to privacy-enhancing technologies adoption: a PETs project report. Regulator research and workshop report. No reference number. No publication or update date is shown on the report page; the workshop it draws on was held on 20 February 2024, which is not a publication date. Findings reflect workshop participants and a pre-event survey rather than a representative market census, and participant recommendations are expressly not ICO policy. United Kingdom. https://ico.org.uk/about-the-ico/research-reports-impact-and-evaluation/research-and-reports/technology-and-innovation/tackling-barriers-to-privacy-enhancing-technologies-adoption/

24. Xingyu Fu, Ningyuan Chen, Pin Gao and Yang Li. Privacy-Preserving Personalized Recommender Systems. Peer-reviewed journal article, Manufacturing and Service Operations Management, volume 28, issue 1. DOI 10.1287/msom.2023.0271. Published online 18 August 2025; issue dated January to February 2026. Not UK law. A theoretical result within a specified model; the article discloses university and national research grant support. https://pubsonline.informs.org/doi/10.1287/msom.2023.0271

25. Tamas Madl, Weijie Xu, Olivia Choudhury and Matthew Howard. Approximate, Adapt, Anonymize (3A): a Framework for Privacy Preserving Training Data Release for Machine Learning. Preprint, arXiv:2307.01875. Submitted 4 July 2023. Peer-review status not established. Not UK law. A research proposal for training-data release evaluated on selected classification tasks, not an industry implementation methodology. https://arxiv.org/abs/2307.01875

Position as at 28 July 2026. The ICO’s privacy-enhancing technologies guidance, including every linked technology page cited above, is under review following the Data (Use and Access) Act 2025, and the wider lawful-basis guidance has changed since these pages were written. The regulator is also mid-transition: the Information Commission has been established and is expected to take over the ICO’s functions later in 2026, so the issuing body named in these references may change. Standards and research move as well, and a technique that is impractical at the time of writing may become deployable. Verify the current status of any source, and re-run any evaluation, before relying on it for a technical or compliance decision.