Secure Systems Group
Menu×

Wiki Home

Examining Zero-Shot Vulnerability Repair with Large Language Models

Paper: IEEE S&P 2023

High-level overview

Code-completion models trained on public source code can write functional code from a prompt of comments and signatures. This paper asks whether the same off-the-shelf models can fix security bugs. The task: given a program with a known, localized vulnerability, a security-relevant defect named by its CWE class, produce a replacement that removes the weakness and preserves intended behavior, through prompting alone with no fine-tuning on bug-fix data. The models are large language models for code used as black boxes.

The work is an empirical measurement study. The authors wrap a vulnerable function and its bug report in prompt templates that vary how much context they supply, then query seven models: commercial black-box APIs (OpenAI’s Codex family and AI21’s Jurassic-1) alongside the open PolyCoder and a 774M-parameter C/C++ model the authors trained. A patch counts as a repair only when an external security check confirms the weakness is gone and the project’s tests still pass.

The headline finding splits by difficulty. Across synthetic and hand-crafted scenarios, the ensemble produced at least one patch passing both checks, so simple, self-contained bugs are within reach of prompting. On twelve historical real-world CVEs, the models repaired 8 of 12 but often broke functional correctness, and the paper concludes the approach is not yet reliable enough to anchor a real program-repair pipeline. More prompt context generally helped, and the models did best on short, local fixes.

Threat Model: No adversary acts here. The actor is a developer or an automated repair pipeline that wants to fix a known vulnerability, with black-box access to a pretrained code model and the prompt and decoding parameters (temperature, top-p) as its only levers. The inputs are the vulnerable source and a bug report that localizes the defect; the paper assumes localization is already done, using the developer’s own patch site as the oracle, and treats root-cause finding as out of scope. “Success” is a patch that is both functionally correct, passing the project’s tests, and security-correct, no longer flagged by the security tool or crashing on the proof-of-concept input. The authors bound that notion: passing tests does not prove a bug absent, repairs are confined to one location in one file, and the real-world result rests on a small sample.

Basic Background

Software vulnerabilities and CWE

A software vulnerability is a security-relevant defect an attacker can exploit, such as an out-of-bounds write, a use-after-free, or an SQL injection. MITRE’s Common Weakness Enumeration (CWE) gives each weakness type a stable identifier and ranks the most dangerous in a “Top 25” list. Two kinds of tool detect such defects here: static analysis, which inspects code without running it (the paper uses CodeQL), and runtime sanitizers such as AddressSanitizer and UndefinedBehaviorSanitizer, which instrument a compiled program to abort on a memory or undefined-behavior error when given an input that triggers it (Serebryany et al., 2012).

Large language models for code

Large language models for code are transformers pretrained by next-token prediction over public source code; given a prompt of code and comments they emit a likely continuation. OpenAI’s Codex, a GPT-3 model adapted to code and the engine behind GitHub Copilot, is the prominent example (Chen et al., 2021); this work also evaluates AI21’s Jurassic-1 (Lieber et al., 2021) and the open PolyCoder (Xu et al., 2022). The caller shapes the output through the prompt and through decoding parameters like temperature and top-p, which trade determinism for diversity. Trained on unaudited code, these models reproduce insecure patterns as readily as secure ones.

Zero-shot prompting

Zero-shot prompting elicits a task from a pretrained model through the prompt alone, with no worked examples in the context and no task-specific fine-tuning (Brown et al., 2020). The lever is prompt design: which comments, surrounding code, and hints to include. Because the model continues the most likely text, phrasing strongly affects whether the continuation is the intended fix.

Automated program repair

Automated program repair turns a faulty program into one that passes a correctness oracle, with no human writing the patch. The oracle is usually a regression test suite, which raises the plausible-versus-correct problem: a patch that passes weak tests need not be correct (Qi et al., 2015). Repairing a vulnerability adds a second oracle, a security check that must witness the weakness is gone.

Reading guidance

  • Section IV-D, Table IV: the five repair-prompt templates and how much context each wraps around the bug. Section III gives the design rationale, vendor user guides and informal GitHub searches; note what that rationale rests on.
  • Section V-B, the Bug Localizing paragraph: the developer-provided patches are used as an oracle to localize each vulnerability, so every real-world CVE result presupposes a known bug location.
  • Section V-D, the sentence defining success: the functional test suite passes and the program no longer crashes under ASAN or UBSAN on the original problematic input. Section VII states what such a pass does and does not guarantee about the vulnerability.

Paper Context

Automatic program repair had a decade of work behind it. Generate-and-validate systems search a space of edits and keep those that pass a test suite; GenProg, using genetic search, is the canonical example (Le Goues et al., 2012), and the broader area is surveyed by Monperrus (2018). A known weakness of the paradigm is that a patch can be plausible, passing the supplied tests, without being correct; under stricter testing many reported GenProg fixes turned out wrong (Qi et al., 2015). Repairing security bugs is a recognized specialization, and security patches were found to be smaller and more localized than ordinary bug fixes (Li and Paxson, 2017); one line of work extracts a crash constraint from a sanitizer to drive the repair (Gao et al., 2021).

A second thread replaced hand-built search with learned models. Neural machine translation systems train sequence-to-sequence models on bug-fix pairs mined from version history (Tufano et al., 2019), with later systems narrowing to one-line fixes (SequenceR; Chen et al., 2021) or adding code-aware search (CURE; Jiang et al., 2021). These models are trained for the repair task and aim to reproduce the developer’s exact patch, and they tend to carry restrictions: a single language, a single edited line, or a curated bug-fix corpus.

In parallel, general code models arrived. GPT-3 showed that a large model prompted with no task-specific training could perform many tasks (Brown et al., 2020); Codex adapted this to code and powers GitHub Copilot (Chen et al., 2021), alongside contemporaries for program synthesis (Austin et al., 2021) and open code models (Xu et al., 2022). These are general-purpose, prompt-driven, and black-box, a different posture from a repair model trained on bug-fix pairs.

What such models do to security was just starting to be measured. A prior study by the same group applied a security lens to Copilot’s completions and found that roughly 40% contained weaknesses, and that the rate shifted with how the prompt was phrased (Pearce et al., 2021). That study measured the security of code the model writes from scratch and did not weigh functional correctness.

Additional Readings

References
  • Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., and Sutton, C. “Program Synthesis with Large Language Models.” arXiv:2108.07732, 2021.
  • Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. “Language Models are Few-Shot Learners.” arXiv:2005.14165, 2020.
  • Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H.P.d.O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. “Evaluating Large Language Models Trained on Code.” arXiv:2107.03374, 2021.
  • Chen, Z., Kommrusch, S., Tufano, M., Pouchet, L.-N., Poshyvanyk, D., and Monperrus, M. “SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repair.” IEEE Transactions on Software Engineering, 47(9):1943-1959, 2021.
  • Gao, X., Wang, B., Duck, G.J., Ji, R., Xiong, Y., and Roychoudhury, A. “Beyond Tests: Program Vulnerability Repair via Crash Constraint Extraction.” ACM Transactions on Software Engineering and Methodology, 30(2):1-27, 2021.
  • Jiang, N., Lutellier, T., and Tan, L. “CURE: Code-Aware Neural Machine Translation for Automatic Program Repair.” IEEE/ACM International Conference on Software Engineering (ICSE), 2021.
  • Le Goues, C., Nguyen, T., Forrest, S., and Weimer, W. “GenProg: A Generic Method for Automatic Software Repair.” IEEE Transactions on Software Engineering, 38(1):54-72, 2012.
  • Li, F. and Paxson, V. “A Large-Scale Empirical Study of Security Patches.” ACM SIGSAC Conference on Computer and Communications Security (CCS), 2017.
  • Lieber, O., Sharir, O., Lentz, B., and Shoham, Y. “Jurassic-1: Technical Details and Evaluation.” AI21 Labs Technical Report, 2021.
  • Monperrus, M. “Automatic Software Repair: A Bibliography.” ACM Computing Surveys, 51(1):17:1-17:24, 2018.
  • Pearce, H., Ahmad, B., Tan, B., Dolan-Gavitt, B., and Karri, R. “An Empirical Cybersecurity Evaluation of GitHub Copilot’s Code Contributions.” arXiv:2108.09293, 2021.
  • Qi, Z., Long, F., Achour, S., and Rinard, M. “An Analysis of Patch Plausibility and Correctness for Generate-and-Validate Patch Generation Systems.” International Symposium on Software Testing and Analysis (ISSTA), 2015.
  • Serebryany, K., Bruening, D., Potapenko, A., and Vyukov, D. “AddressSanitizer: A Fast Address Sanity Checker.” USENIX Annual Technical Conference (USENIX ATC), 2012.
  • Tufano, M., Watson, C., Bavota, G., Di Penta, M., White, M., and Poshyvanyk, D. “An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation.” ACM Transactions on Software Engineering and Methodology, 28(4):19:1-19:29, 2019.
  • Xu, F.F., Alon, U., Neubig, G., and Hellendoorn, V.J. “A Systematic Evaluation of Large Language Models of Code.” arXiv:2202.13169, 2022.