Wiki Home

No Privacy Left Outside: On the (In-)Security of TEE-Shielded DNN Partition for On-Device ML

Paper: IEEE S&P 2024

High-level overview

On-device machine learning ships a trained model to a phone or IoT device, where the party holding the device can read its weights directly. That white-box access makes two attacks cheap: model stealing (MS), which reproduces a model’s weights or functionality, and the membership inference attack (MIA), which recovers whether a record was in the training set. Hosting the whole model in a trusted execution environment (TEE) downgrades both to the harder black-box setting, but a TEE runs a deep network up to about 50 times slower than a GPU. TEE-Shielded DNN Partition (TSDP) is the class of defenses that splits the difference: a privacy-sensitive subset of the network runs inside the TEE while the rest is offloaded to the device’s untrusted GPU for speed (see model partitioning). Every TSDP scheme assumes the offloaded part exposes no more than a black-box interface.

This paper is the first systematic security evaluation of that assumption. It surveys TSDP schemes published from 2018 to 2023, sorts them into five partition strategies by which part of the network the TEE shields, and benchmarks a representative of each with both MS and MIA under one practical adversary. The measurements run against the schemes: model stealing reaches several times the accuracy of the shielding-whole-model baseline, and membership inference stays consistently above it, close to fully white-box quality. The deployed partitions do not deliver the black-box-level protection they claim. The “sweet spot” partition, the one giving the most security at the least utility cost, shifts across models and datasets, so it cannot be set in advance. The paper closes with TeeSlice, a partition-before-training design that isolates privacy-bearing weights into small slices and reaches shielding-whole-model security at over ten times less TEE overhead with no accuracy loss.

Threat Model: A model owner (the defender) ships a model to a device whose rich execution environment, the ordinary operating system, applications, and the device user, is adversarial. The adversary reads, and in principle can tamper with, the offloaded part of the model running outside the TEE, and observes the TEE’s inputs and outputs, which return predicted labels only (label-only outputs). It cannot break the TEE, whose internal data and computation are assumed secure, and side channels against the TEE are out of scope. Beyond the device the adversary holds public pretrained models and datasets, used to infer the protected model’s architecture and to initialize a surrogate, plus a query budget against the deployed label-only model under one percent of the training-set size. The protected assets are the model, against extraction, and the training-data privacy that model leakage would expose, against membership inference. The defender’s claim, which the paper measures, is that shielding a privacy-sensitive subset downgrades white-box MS and MIA to the black-box label-only setting, matching the protection of shielding the entire model.

Basic Background

On-device ML and trusted execution environments

On-device ML runs a model on the end user’s hardware for latency and privacy, which also hands the device owner the model file. A trusted execution environment is a hardware-isolated region, an enclave, that holds code and data so the rest of the system, including a malicious operating system, cannot read or alter them. Commodity examples are Intel SGX (McKeen et al., 2013), AMD SEV (Kaplan et al., 2016), and Arm TrustZone (Alves, 2004). The hardware outside the enclave, the normal OS and applications and the accelerators it drives, is the rich execution environment, and on a user device it is what the adversary controls.

Partitioning a model across a TEE and a GPU

A whole model inside a TEE is secure but slow, so TSDP runs only part of the network in the enclave and offloads the rest to the GPU (see model partitioning). The schemes differ in which part they shield: a block of layers, the largest-magnitude weights, or the non-linear layers with the linear ones obfuscated before offloading. More shielded computation buys more protection and costs more latency, the trade-off every partition must resolve.

Model stealing and membership inference

Model stealing, or model extraction, reproduces a deployed model’s parameters or functionality from external access; early attacks solved simple model families from prediction-API outputs (Tramèr et al., 2016), and later ones trained accurate or high-fidelity copies across architectures and data distributions (Orekondy et al., 2019; Jagielski et al., 2020). The membership inference attack predicts whether a specific example was in the training set, the standard privacy audit; a common construction trains shadow models that imitate the target to calibrate the decision (Shokri et al., 2017), later recast as a calibrated likelihood-ratio test (Carlini et al., 2022). The paper’s central framing is the white-box versus black-box gap: shielding is meant to deny the adversary the weights and force the harder query-only attack.

Public models and transfer learning

The benchmarked models are standard convolutional networks, including ResNet, VGG, and AlexNet, built by transfer learning: a public backbone pretrained on a large dataset, then fine-tuned on the task. The same public backbones are an adversary resource, supplying a near-equivalent model from which to read architecture and initialize a surrogate.

Differential privacy as the formal alternative

Differential privacy (DP) bounds any single training example’s influence on a model and so caps membership-inference success with a provable guarantee (Dwork and Roth, 2014). It addresses data privacy rather than model stealing, and its accuracy cost is the reason the on-device line looks to hardware instead.

Reading guidance

• Section 2.2: what the threat model grants the adversary, namely a public model taken from the internet, the offloaded weights, and fewer queries to the victim than one percent of its training data. Section 2.1 places TEE side channels out of scope.
• Sections 3.3 and 3.4, Figure 2: the three-phase attack pipeline (surrogate initialization, model stealing, membership inference) and the No-Shield and Black-box baselines every scheme is read against. Note which one the paper treats as the ceiling on protection.
• Section 3.5, Table 2: the relative-accuracy row, which divides each scheme’s attack accuracy by the black-box baseline’s. A value near 1.00x is protection at the black-box level, the goal the paper sets; note how far the schemes sit from it.