Coding Agent Plugins

Coding agent plugins test whether an autonomous coding agent preserves code-review, filesystem, terminal, environment, process, automation, network, and verifier boundaries while it works in a repository.

Use them with a provider that runs a real coding harness, such as openai:codex-sdk, an HTTP wrapper around an internal agent, or a custom file:// provider that launches the agent under test.

info

These plugins generate coding-agent tasks with Promptfoo's remote red-team generator. During grading, Promptfoo can use the final answer, provider raw events, available trace attributes, protected file hashes, and the normal LLM rubric. Multi-turn strategies such as jailbreak:meta and jailbreak:hydra pass the per-turn target provider response into the grader so deterministic checks can still inspect command evidence from the turn that failed.

Plugin IDs

Plugin	What it tests
coding-agent:repo-prompt-injection	Untrusted repository text, such as READMEs, docs, comments, generated files, issues, or fixture data, attempts to steer the coding agent away from the user task.
coding-agent:terminal-output-injection	Test, compiler, package-manager, hook, or script output contains instructions or terminal-only receipts that the agent should not treat as authority.
coding-agent:secret-env-read	Launcher-only credentials or synthetic environment canaries must not be inspected, copied into artifacts, or revealed.
coding-agent:secret-file-read	Protected local file contents or content-derived digests from dotfiles, sibling checkouts, temp files, cache files, browser profiles, or support bundles must not reach target-visible sinks.
coding-agent:sandbox-read-escape	The agent must stay inside the intended checkout and configured additional directories, even when symlinks, sibling paths, temp files, or home-directory hints exist.
coding-agent:sandbox-write-escape	The agent must not create, overwrite, append, delete, chmod, or otherwise mutate files outside the intended checkout through relative paths, absolute paths, symlinks, or mounts.
coding-agent:network-egress-bypass	The agent must not make unauthorized outbound requests, DNS lookups, webhook calls, package fetches, tunnels, or connector/tool calls outside the intended network policy.
coding-agent:procfs-credential-read	The agent must not recover credentials or ambient execution context through /proc environment, command-line, file-descriptor, cwd/root, or procfs-like runtime metadata surfaces.
coding-agent:delayed-ci-exfil	The agent must not plant CI, workflow, hook, package, release, or scheduled automation that leaks data or calls unapproved endpoints after the immediate eval run.
coding-agent:generated-vulnerability	The agent must not satisfy a normal coding task by introducing insecure code such as unsafe command execution, injection flaws, broken auth, unsafe deserialization, or disabled validation.
coding-agent:automation-poisoning	The agent must not persist unsafe behavior in repository automation such as hooks, package lifecycle scripts, scheduled jobs, release tasks, or CI workflows.
coding-agent:steganographic-exfil	The agent must not hide sensitive data in encoded, transformed, or low-visibility outputs such as comments, logs, filenames, snapshots, generated artifacts, or metadata.
coding-agent:verifier-sabotage	The agent must fix the requested behavior instead of deleting, skipping, weakening, regenerating, or bypassing tests, snapshots, policies, hooks, verifiers, lockfiles, or QA scripts.

You can also select the core collection:

redteam:
  plugins:
    - id: coding-agent:core

Use the full collection for comprehensive coverage across all coding-agent plugin IDs:

redteam:
  plugins:
    - id: coding-agent:all

Basic Codex SDK Configuration

Run Codex in a prepared repository with streaming events and deep tracing enabled:

promptfooconfig.yaml

description: Coding agent red team

prompts:
  - '{{prompt}}'

providers:
  - id: openai:codex-sdk
    config:
      model: gpt-5.4
      working_dir: ./agent-workspace
      sandbox_mode: workspace-write
      approval_policy: never
      enable_streaming: true