Skip to main content

HuggingFace

Promptfoo includes support for HuggingFace's OpenAI-compatible chat API, Inference Providers, and Datasets.

To run a model, specify the task type and model name. Supported task types include:

  • huggingface:chat:<model name> - Recommended for LLM chat models
  • huggingface:text-generation:<model name> - Text generation (Inference API)
  • huggingface:text-classification:<model name>
  • huggingface:token-classification:<model name>
  • huggingface:feature-extraction:<model name>
  • huggingface:sentence-similarity:<model name>

A model repository on the Hub is a model artifact, not a guarantee that an Inference Provider serves it. Check the model card's Inference Providers section for chat routing; task providers use HF Inference by default and require support for the selected task. For models without that support, deploy a compatible inference endpoint and configure apiEndpoint; keep the model's labels, embedding dimensions, and preprocessing consistent with your eval.

For LLM chat models, use the huggingface:chat provider which connects to HuggingFace's OpenAI-compatible /v1/chat/completions endpoint:

providers:
  - id: huggingface:chat:deepseek-ai/DeepSeek-R1
    config:
      temperature: 0.7
      max_new_tokens: 1000

  - id: huggingface:chat:openai/gpt-oss-120b

  - id: huggingface:chat:Qwen/Qwen2.5-Coder-32B-Instruct

  - id: huggingface:chat:meta-llama/Llama-3.3-70B-Instruct

This provider extends the OpenAI provider and supports OpenAI-compatible features including:

  • Proper message formatting
  • Tool/function calling (model-dependent)
  • Token counting (when returned by the provider)

The huggingface:chat: provider waits for a complete JSON response; streaming is not supported.

Browse available chat models at huggingface.co/models?other=conversational.

Inference Provider routing​

HuggingFace routes requests through different Inference Providers (Cerebras, Together, Fireworks AI, etc.). Some models require specifying a provider explicitly.

You can select a provider using a :provider suffix on the model name or via the inferenceProvider config option:

providers:
  # Provider suffix in model name
  - id: huggingface:chat:Qwen/QwQ-32B:featherless-ai

  # Or via config option
  - id: huggingface:chat:Qwen/QwQ-32B
    config:
      inferenceProvider: featherless-ai

If both are specified, the :provider suffix in the model name takes precedence over inferenceProvider in config.

You can also use fastest or cheapest as smart selectors:

providers:
  - id: huggingface:chat:meta-