Secure Systems Group
Menu×

CS858 Paper List

Reading list for CS858 (Trustworthy Machine Learning), Fall 2026

Part 1: Risks to trustworthiness in ML

#PaperThemeTopicEssential readings
1Towards Deep Learning Models Resistant to Adversarial AttacksInference-Time Integrity of Model BehaviorAdversarial Robustness of Classification ModelsExplaining and Harnessing Adversarial Examples; Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
2Jailbroken: How Does LLM Safety Training Fail?Inference-Time Integrity of Model BehaviorJailbreaking LLMsUniversal and Transferable Adversarial Attacks on Aligned Language Models; Jailbreaking Black Box Large Language Models in Twenty Queries
3Not What You’ve Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt InjectionInference-Time Integrity of Model BehaviorIndirect Prompt Injection in AI AgentsInjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated LLM Agents; Formalizing and Benchmarking Prompt Injection Attacks and Defenses
4Safety Alignment Should Be Made More Than Just a Few Tokens DeepInference-Time Integrity of Model BehaviorSafety Alignment and GuardrailsLlama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations; SecAlign: Defending Against Prompt Injection with Preference Optimization
5Membership Inference Attacks From First PrinciplesTraining-Data Memorization and LeakageMembership Inference AttacksMembership Inference Attacks Against Machine Learning Models; Detecting Pretraining Data from Large Language Models
6Extracting Training Data from Large Language ModelsTraining-Data Memorization and LeakageTraining-data Extraction from LLMsScalable Extraction of Training Data from (Production) Language Models; Extracting Training Data from Diffusion Models
7Deep Learning with Differential PrivacyTraining-Data Memorization and LeakageDifferential Privacy in MLEvaluating Differentially Private Machine Learning in Practice; Large Language Models Can Be Strong Differentially Private Learners
8Knowledge Unlearning for Mitigating Privacy Risks in Language ModelsTraining-Data Memorization and LeakageUnlearning for Generative AIErasing Concepts from Diffusion Models; TOFU: A Task of Fictitious Unlearning for LLMs
9Knockoff Nets: Stealing Functionality of Black-Box ModelsModel Extraction and DistillationModel Extraction / StealingHigh Accuracy and High Fidelity Extraction of Neural Networks; Stealing Part of a Production Language Model
10DAWN: Dynamic Adversarial Watermarking of Neural NetworksModel Extraction and DistillationModel Watermarking / FingerprintingFalse Claims against Model Ownership Resolution; Can LLM Watermarks Robustly Prevent Unauthorized Knowledge Distillation?
11Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural NetworksData Integrity and Supply-Chain SecurityTraining-data PoisoningBadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain; Poisoning Language Models During Instruction Tuning
12PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language ModelsData Integrity and Supply-Chain SecurityPreference Manipulation / RAG PoisoningAdversarial Search Engine Optimization for Large Language Models; Certifiably Robust RAG against Retrieval Corruption
13What Makes and Breaks Safety Fine-tuning? A Mechanistic StudyCross-Cutting TopicsMechanistic Interpretability for AI safetyRefusal in Language Models Is Mediated by a Single Direction; Improving Alignment and Robustness with Circuit Breakers
14SoK: Unintended Interactions among Machine Learning Defenses and RisksCross-Cutting TopicsUnintended Interactions among ML Defenses and RisksConflicting Interactions Among Protection Mechanisms for Machine Learning Models; Combining Machine Learning Defenses without Conflicts
15A Watermark for Large Language ModelsCross-Cutting TopicsMedia Forensics and Proactive ProvenanceTree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust; Media Forensics and DeepFakes: An Overview
16Examining Zero-Shot Vulnerability Repair with Large Language ModelsCross-Cutting TopicsAI for CybersecurityLLM Agents can Autonomously Exploit One-day Vulnerabilities; AI Agents Enable Adaptive Computer Worms

Part 2: Software vs. Hardware defenses

#PaperThemeTopicEssential readings
17Secure Transformer Inference Made Non-interactiveLeakage Resistance (Client Data)SoftwareBumbleBee: Secure Two-party Inference Framework for Large Transformers
18BliMe: Verifiably Secure Outsourced Computation with Hardware-Enforced Taint TrackingLeakage Resistance (Client Data)HardwareOrigami Inference: Private Inference Using Hardware Enclaves
19Unlocking the Power of Differentially Private Zeroth-order Optimization for Fine-tuning LLMsLeakage Resistance (Training Data)SoftwarePrivateFL: Accurate, Differentially Private Federated Learning via Personalized Data Transformation; Local and Central Differential Privacy for Robustness and Privacy in Federated Learning
20No Privacy Left Outside: On the (In-)Security of TEE-Shielded DNN Partition for On-Device MLLeakage Resistance (Training Data)HardwareSlalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware
21ModelGuard: Information-Theoretic Defense Against Model Extraction AttacksModel Theft Resistance (+ On-Device Private Inference)SoftwareBeowulf: Mitigating Model Extraction Attacks Via Reshaping Decision Regions
22ASGARD: Protecting On-Device Deep Neural Networks with Virtualization-Based Trusted Execution EnvironmentsModel Theft Resistance (+ On-Device Private Inference)HardwareShadowNet: A Secure and Efficient On-device Model Inference System for Convolutional Neural Networks
23zkGPT: An Efficient Non-interactive Zero-knowledge Proof Framework for LLM InferenceRegulatory ComplianceSoftwareExperimenting with Zero-Knowledge Proofs of Training; Zero-Knowledge Proofs of Training for Deep Neural Networks
24PAL*M: Property Attestation for Large Generative ModelsRegulatory ComplianceHardwareLaminator: Verifiable ML Property Cards using Hardware-assisted Attestations; Sentry: Authenticating Machine Learning Artifacts on the Fly