CS858 Wiki

Here you find the list of papers relevant for this course. Every student is expected to at least read the assigned readings.

Section	Description
Papers	The paper pages, each a short guide to one reading.
Concepts	Background concepts the papers rely on.

Reading list

The course runs in two parts, grouped by theme. The assigned reading in each row is required; expand the additional readings beneath it to see what goes alongside.

Part 1: Risks to trustworthiness in ML

#	Topic	Reading
Inference-Time Integrity of Model Behavior
1	Adversarial Robustness of Classification Models	Assigned reading Towards Deep Learning Models Resistant to Adversarial Attacks Additional readings (2) Explaining and Harnessing Adversarial Examples Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
2	Jailbreaking LLMs	Assigned reading Jailbroken: How Does LLM Safety Training Fail? Additional readings (2) Universal and Transferable Adversarial Attacks on Aligned Language Models Jailbreaking Black Box Large Language Models in Twenty Queries
3	Indirect Prompt Injection in AI Agents	Assigned reading Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection Additional readings (2) InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated LLM Agents Formalizing and Benchmarking Prompt Injection Attacks and Defenses
4	Safety Alignment and Guardrails	Assigned reading Safety Alignment Should Be Made More Than Just a Few Tokens Deep Additional readings (2) Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations SecAlign: Defending Against Prompt Injection with Preference Optimization
Training-Data Memorization and Leakage
5	Membership Inference Attacks	Assigned reading Membership Inference Attacks From First Principles Additional readings (2) Membership Inference Attacks Against Machine Learning Models Detecting Pretraining Data from Large Language Models
6	Training-data Extraction from LLMs	Assigned reading Extracting Training Data from Large Language Models Additional readings (2) Scalable Extraction of Training Data from (Production) Language Models Extracting Training Data from Diffusion Models
7	Differential Privacy in ML	Assigned reading Deep Learning with Differential Privacy Additional readings (2) Evaluating Differentially Private Machine Learning in Practice Large Language Models Can Be Strong Differentially Private Learners
8	Unlearning for Generative AI	Assigned reading Knowledge Unlearning for Mitigating Privacy Risks in Language Models Additional readings (2) Erasing Concepts from Diffusion Models TOFU: A Task of Fictitious Unlearning for LLMs
Model Extraction and Distillation
9	Model Extraction / Stealing	Assigned reading Knockoff Nets: Stealing Functionality of Black-Box Models Additional readings (2) High Accuracy and High Fidelity Extraction of Neural Networks Stealing Part of a Production Language Model
10	Model Watermarking / Fingerprinting	Assigned reading DAWN: Dynamic Adversarial Watermarking of Neural Networks Additional readings (2) False Claims against Model Ownership Resolution Can LLM Watermarks Robustly Prevent Unauthorized Knowledge Distillation?
Data Integrity and Supply-Chain Security
11	Training-data Poisoning	Assigned reading Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks Additional readings (2) BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain Poisoning Language Models During Instruction Tuning
12	Preference Manipulation / RAG Poisoning	Assigned reading PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models Additional readings (2) Adversarial Search Engine Optimization for Large Language Models Certifiably Robust RAG against Retrieval Corruption
Cross-Cutting Topics
13	Mechanistic Interpretability for AI safety	Assigned reading What Makes and Breaks Safety Fine-tuning? A Mechanistic Study Additional readings (2) The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence Improving Alignment and Robustness with Circuit Breakers
14	Unintended Interactions among ML Defenses and Risks	Assigned reading SoK: Unintended Interactions among Machine Learning Defenses and Risks Additional readings (2) Conflicting Interactions Among Protection Mechanisms for Machine Learning Models Combining Machine Learning Defenses without Conflicts
15	Media Forensics and Proactive Provenance	Assigned reading A Watermark for Large Language Models Additional readings (2) Tree-Ring Watermarks: Fingerprints for Diffusion Images that are Invisible and Robust Media Forensics and DeepFakes: An Overview
16	AI for Cybersecurity	Assigned reading Examining Zero-Shot Vulnerability Repair with Large Language Models Additional readings (2) LLM Agents can Autonomously Exploit One-day Vulnerabilities AI Agents Enable Adaptive Computer Worms

Part 2: Software vs. Hardware defenses

#	Topic	Reading
Leakage Resistance (Client Data)
17	Software	Assigned reading Secure Transformer Inference Made Non-interactive Additional readings (1) BumbleBee: Secure Two-party Inference Framework for Large Transformers
18	Hardware	Assigned reading BliMe: Verifiably Secure Outsourced Computation with Hardware-Enforced Taint Tracking Additional readings (1) Origami Inference: Private Inference Using Hardware Enclaves
Leakage Resistance (Training Data)
19	Software	Assigned reading Unlocking the Power of Differentially Private Zeroth-order Optimization for Fine-tuning LLMs Additional readings (2) PrivateFL: Accurate, Differentially Private Federated Learning via Personalized Data Transformation Local and Central Differential Privacy for Robustness and Privacy in Federated Learning
20	Hardware	Assigned reading No Privacy Left Outside: On the (In-)Security of TEE-Shielded DNN Partition for On-Device ML Additional readings (1) Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware
Model Theft Resistance (+ On-Device Private Inference)
21	Software	Assigned reading ModelGuard: Information-Theoretic Defense Against Model Extraction Attacks Additional readings (1) Beowulf: Mitigating Model Extraction Attacks Via Reshaping Decision Regions
22	Hardware	Assigned reading ASGARD: Protecting On-Device Deep Neural Networks with Virtualization-Based Trusted Execution Environments Additional readings (1) ShadowNet: A Secure and Efficient On-device Model Inference System for Convolutional Neural Networks
Regulatory Compliance
23	Software	Assigned reading zkGPT: An Efficient Non-interactive Zero-knowledge Proof Framework for LLM Inference Additional readings (2) Experimenting with Zero-Knowledge Proofs of Training Zero-Knowledge Proofs of Training for Deep Neural Networks
24	Hardware	Assigned reading PAL*M: Property Attestation for Large Generative Models Additional readings (2) Laminator: Verifiable ML Property Cards using Hardware-assisted Attestations Sentry: Authenticating Machine Learning Artifacts on the Fly

Last compiled: 2026-08-31. Papers: 24. Concepts: 82.

CS858 Wiki

Contents

Reading list

Part 1: Risks to trustworthiness in ML

Part 2: Software vs. Hardware defenses