Wiki Home

ASGARD: Protecting On-Device Deep Neural Networks with Virtualization-Based Trusted Execution Environments

Paper: NDSS 2025

High-level overview

On-device deep learning runs DNN inference on a user’s phone or IoT device, keeping sensitive inputs such as faces and fingerprints off remote servers. The same shift hands the device the model itself, where an adversary who controls the rich execution environment (REE), the device’s untrusted software stack, can read or dump the model’s weights during inference. Prior protections host inference inside a trusted execution environment (TEE) built on Arm TrustZone, the hardware-isolated secure world of a mobile SoC, but they hit two walls. Keeping the whole model in the TEE either runs on the CPU only, losing the device’s accelerator, or must port proprietary accelerator software into TrustZone’s specialized secure-world OS. Offloading part of inference to the REE to use its accelerator either leaves that part unprotected or pays heavy run-time cost obfuscating it across the secure boundary (see model partitioning).

ASGARD is the first virtualization-based TEE for on-device DNN protection on legacy Armv8-A SoCs. A virtualization-based TEE realizes an enclave as a virtual machine isolated by a small trusted hypervisor (see hardware virtualization), in place of TrustZone’s two-world split. ASGARD extends such an enclave to hold the entire model together with an SoC-integrated accelerator, a neural processing unit (NPU), through secure I/O passthrough, so inference runs fully accelerated inside the enclave while keeping the trusted computing base (TCB) small: the isolation monitor sits at the hypervisor exception level (EL2) and leaves the proprietary EL3 secure monitor and the vendor accelerator driver stack unmodified. On a commodity Android board (Rockchip RK3588S) with no changes to Rockchip- or Arm-proprietary software, ASGARD adds roughly two thousand lines to the hypervisor, ships an enclave image of about 17 MB, and reaches DNN inference latency on par with or below unprotected accelerated inference, in one case slightly faster, by coalescing accelerator interrupts that would otherwise force costly exits out of the enclave.

Threat Model: Three parties take part. A remote model provider ships a model it wants to protect; an REE-side privileged adversary on the device aims to steal it; and a trusted DNN-serving enclave runs on ASGARD-extended platform software. The adversary fully controls the REE, down to the unprivileged hypervisor that manages resources, and seeks to read or dump the model’s weights and architecture while it is in use for inference. Trusted are the TEEvisor, the small hypervisor at EL2 that enforces isolation between virtualization domains, and the secure world, which on Armv8-A includes the EL3 secure monitor and can reach the entire normal world regardless. The protected asset is the model, kept confidential in transit, at rest, and in use; the accelerator sits behind an IOMMU and exposes a reset interface. The defender’s claim is software-secure enclaves against REE-side software adversaries on commodity Armv8-A hardware. Physical attacks and side channels are out of scope. The paper notes the guarantee can be strengthened to hardware-secure enclaves where additional hardware support, such as Arm’s Confidential Compute Architecture, is available.

Basic Background

On-device deep learning and model confidentiality

On-device DNN inference runs the model on the end user’s hardware, which exposes the model file to the device. The on-device models are typically compact convolutional networks and small transformers, the object-detection and text-understanding backbones used on mobile NPUs. The direct threat is model extraction: reproducing a model’s weights, architecture, or functionality from external access, which on a device can be as crude as dumping the model from memory during inference. A stolen model is also a privacy exposure, since its behavior reveals which records were in its training set (membership inference) (Shokri et al., 2017) and large models can emit training examples verbatim (Carlini et al., 2021).

Trusted execution environments and Arm TrustZone

A trusted execution environment (TEE) isolates code and data from the rest of the system, including a malicious OS, and attests what code loaded. Arm TrustZone, the commodity TEE on mobile SoCs, splits the processor into a normal world and a secure world with physical resources statically partitioned between them, and places security-critical services in the secure world (Arm Limited, 2021). Everything outside the secure world, the ordinary OS, applications, and accelerators, is the rich execution environment (REE), and on a user’s device it is what the adversary controls. The trusted computing base (TCB) is the set of hardware and software whose correctness the protection assumes; a smaller TCB is a smaller attack surface.

Hardware virtualization and VM-based enclaves

Hardware virtualization lets a hypervisor give each virtual machine its own view of physical memory through a second stage of address translation, and isolates DMA-capable peripherals behind an IOMMU. A virtualization-based TEE uses a small, trusted hypervisor to turn one virtual machine into an enclave, isolated from a larger untrusted hypervisor and the host OS. On Arm this isolation is enforced at the hypervisor exception level (EL2), beneath the EL3 secure monitor, so an enclave can be built without modifying the most privileged firmware and can run a stock OS with stock drivers.

Partitioning a model across a TEE and an accelerator

A TEE runs a DNN far slower than a dedicated accelerator, so one line of on-device protection partitions the network (see model partitioning): a privacy-sensitive subset runs in the enclave while the rest is offloaded to the device’s untrusted accelerator, in the clear or under obfuscation, for speed. The split trades protection for latency, and the offloaded part is the side an adversary can read.

Reading guidance

• Section III-B with Figure 2: the threat model. The guarantee is stated as software-secure; physical attacks and side channels are out of scope, and the privileged hypervisor, the EL3 secure monitor, and the secure world are trusted.
• Section IV-A: secure I/O passthrough, and what has to move into the privileged hypervisor to support it, namely the IOMMU driver and accelerator reset control. Note what that adds to the platform-level trusted computing base.
• Table III with Figure 5: enclave and TCB size, then end-to-end latency against the REE-only and partition-and-offload baselines. Note what ASGARD costs relative to running with no protection at all.