Four ML Frameworks, Four Attack Surfaces, One Outcome
Introduction
Encryption is widely considered the industry standard for protecting on-device and on-premises ML models.
In July, we demonstrated that an encrypted TensorRT model running on a Jetson Orin can be extracted through a runtime hook. The extraction occurs when TensorRT loads the decrypted model into GPU memory to prepare for inference: after the application decrypts the model buffer in host RAM, passes it to TensorRT’s deserialization API, and the runtime loads it into GPU memory. This behavior is independent of the encryption method used and does not require breaking the encryption algorithm itself.
We have since extended this approach to OpenVINO, ONNX Runtime, and TensorFlow Lite (see our previous TensorRT demonstration).
The result is consistent across all four frameworks: encryption protects the model at rest, but all four inference libraries require the model to be available in a usable form to run inference, which makes it vulnerable once decrypted on the device.
What changes between frameworks is not the outcome, but where and how the model crosses that runtime boundary.
Four frameworks, four boundaries
Each framework exposes a different runtime boundary where the decrypted model (or compiled representation) is handed to the execution engine. We therefore used a different interception point for each one.
Figure 1: Runtime interception points where encrypted models become accessible in a usable form across four ML frameworks.
How to intercept an AI model for each framework?
Each framework exposes a different entry point where the decrypted model is handed to the runtime. We targeted each one:
TensorRT uses an internal vtable to deserialize the engine binary. We patched this vtable at runtime so our hook intercepted the model before it was uploaded to GPU.
OpenVINO requires both the model IR and weights tensor in a usable form at the same moment. We hooked the read_model API to capture both simultaneously.
ONNX Runtime exposes a C API with a function-pointer table. The C++ wrapper is just an abstraction layer that calls into this C API. We patched the C API table once, and the same hook worked for both C and C++ applications. This proves the real boundary is the C-level API, not the C++ wrapper.
TensorFlow Lite is the simplest: it loads models via a plain C function (TfLiteModelCreate). We shadowed this function directly, with no indirection or name mangling, just a straightforward function interception.
These four frameworks illustrate a fundamental principle: the model must exist in memory in a usable form to run inference. That boundary is predictable and hookable, regardless of the API design.
In all cases, the model was captured and the victim process continued inference normally, with no indication of compromise.
What this means for model protection
Encryption protects models at rest (on disk) and in transit (over the network). But encryption has a fundamental limitation:
Once the application decrypts the model for an inference, that decrypted model becomes part of the attack surface on the device.
Practical implications:
-
Extraction is straightforward and fast. With access to the execution environment (in on-device or on-premises deployments), an attacker can inject a hook and extract any model within minutes.
-
One extraction yields all models. If the attacker is inside a process that runs inference on N different models (even sequentially), a single hook can capture all N models with equal ease.
-
No visible indicators. The victim process continues inference with no errors, no crashes, no warnings.
Across TensorRT, OpenVINO, ONNX Runtime, and TensorFlow Lite, we observed the same fundamental behavior:
Once the runtime needs the model in a usable form for execution, it becomes observable by any code with sufficient privilege on that device.
So the key security question for teams deploying proprietary models is not only whether a model is encrypted at rest, which framework is used, or which API is exposed. The key question is whether an attacker can extract the model once it becomes usable on the device.
Based on our testing, encryption alone does not answer that question.
Mitigations
Since encryption is not a complete answer, practical mitigations include:
Hardware-based protection can be effective but is difficult to apply in practice. Many models are too large to run inside CPU-based secure enclaves, and most inference workloads run on GPUs, NPUs or TPUs rather than environments designed for CPU confidential execution. Only a small set of platforms support confidential GPU execution (for example NVIDIA H100 Confidential Computing), and developers often lack control over the hardware on which their models run.
Obfuscation can help at rest by making the on-disk model file opaque to an attacker, but it has important limitations: the inference library and runtime are typically not obfuscated, and an attacker who observes the execution path can often recover model structure or parameters.
Runtime protections (hardware-agnostic): approaches that protect parameters during execution can bridge the gap when confidential hardware is unavailable. For example, Skyld applies mathematical transformations to keep model weights confidential throughout the execution path, protecting parameters before and during inference. These methods are complementary to hardware-based solutions and may be the most practical option on commodity accelerators.
The takeaway
- Encryption at rest and in transit is necessary, but runtime protection requires more. Once a model is decrypted for inference, new defenses kick in.
The attack surface is not about API design - it’s about architecture. Every framework that decrypts and runs inference has a boundary where model data becomes usable. Change your API, and attackers just find the new boundary.
-
Hardware protection (enclaves, secure execution) is the strongest defense when available; otherwise, hardware-agnostic runtime protections such as Skyld offer a practical alternative. If model data never leaves a protected zone, extraction becomes orders-of-magnitude harder.
-
Speed and ease matter. A single
LD_PRELOADhook can steal any model in a process in minutes. The barrier to extraction is low on untrusted infrastructure. -
This isn’t framework-specific. We tested TensorRT, OpenVINO, ONNX Runtime, and TensorFlow Lite. The pattern holds across all of them.
