CyberSecEval Dataset for LLM Security Testing

Overview

The CyberSecEval plugin uses the textual prompt injection datasets from Meta's Purple Llama CyberSecEval benchmark.

The datasets include multilingual prompt injection attacks, testing LLM systems in multiple languages.

The dataset includes test cases that evaluate:

• Direct prompt injection attacks
• Indirect prompt injection attempts
• Security-violating content
• Logic-violating content
• Cross-language attack vectors
• System prompt leakage attempts

tip

This plugin requires a target provider that accepts both a system prompt and user input in OpenAI-compatible chat format.

How it works

The plugin:

• Downloads test cases from the CyberSecEval dataset
• Formats each test case as a system/user message pair
• Randomly selects prompts from the dataset
• Tests if the model appropriately handles the injection attempts
• Uses language-specific LLM-as-a-judge criteria to evaluate responses

Configuration

To include the CyberSecEval plugin in your LLM red teaming setup, add the following configuration to your YAML file:

redteam:
  plugins:
    - cyberseceval

You can control the number of test cases using the config.numTests parameter (defaults to 5):

redteam:
  plugins:
    - id: cyberseceval
      numTests: 25

You can also enable multilingual prompt injection by setting the config.multilingual parameter to true:

redteam:
  plugins:
    - id: cyberseceval
      config:
        multilingual: true

Full example

Create a configuration file promptfooconfig.yaml:

targets:
  # You can target a model
  - openai:chat:gpt-5
  # Or an API endpoint
  - id: https
    config: