One Hot Encoding in Python

Using category labels as features in Python starts with encoding them into a form a model can use. A one-hot encoding gives each category its own binary feature column.

I’ll walk through encoding columns and checking how the fitted encoder handles new category values.

TL;DR

I use OneHotEncoder when later rows must keep the training feature layout, and the fitted category order stays fixed across training and new rows.

• Fit on training features, then call transform() on validation and incoming rows.
• Use Pandas get_dummies() for direct conversion of a DataFrame.
• Choose an unknown-category policy and check the columns added by high-cardinality fields.

What Is One-Hot Encoding?

A one-hot encoding gives each category its own binary feature column. In the run, the red row was [0 0 1] because the fitted category order placed red last.

I chose colors because their labels have no rank. A numeric-looking field such as a postal code can be categorical when its digits identify a place rather than measure an amount.

Each vector position maps to one category in the displayed order.

How to Encode Categorical Columns in Python

The later rows include yellow, which was absent during fitting. I use the fitted encoder to keep the training feature layout.

Step 1: Fit OneHotEncoder and transform later rows

Put the category in a DataFrame column, then pass a two-dimensional selection such as train[[“Color”]] to OneHotEncoder. A single-bracket selection returns one dimension instead.

import pandas as pd
from sklearn.preprocessing import OneHotEncoder

train = pd.DataFrame({"Color": ["red", "blue", "red", "green"]})
test = pd.DataFrame({"Color": ["green", "yellow"]})
encoder = OneHotEncoder(sparse_output=False, handle_unknown="ignore", dtype=int)
encoded_train = encoder.fit_transform(train[["Color"]])
encoded_test = encoder.transform(test[["Color"]])
print("Categories:", encoder.categories_[0].tolist())
print("Training rows:")
print(encoded_train)
print("New rows:")
print(encoded_test)
dummies = pd.get_dummies(train["Color"], dtype=int)
print("Pandas columns:", dummies.columns.tolist())
print("Pandas rows:")
print(dummies.to_numpy())
new_dummies = pd.get_dummies(test["Color"], dtype=int)
print("Pandas new columns:", new_dummies.columns.tolist())
aligned_new = new_dummies.reindex(columns=dummies.columns, fill_value=0)
print("Aligned Pandas rows:")
print(aligned_new.to_numpy())
print("Feature names:", encoder.get_feature_names_out(["Color"]).tolist())

try:
    strict = OneHotEncoder(sparse_output=False)
    strict.fit(train[["Color"]])
    strict.transform(test[["Color"]])
except ValueError as error:
    print("Default unknown-category error:", str(error).split(". ")[0] + ".")

Run the named file with ./.venv/bin/python one_hot_demo.py. The output shows the fitted order and transformed rows for training and later input.

Categories: ['blue', 'green', 'red']
Training rows:
[[0 0 1]
 [1 0 0]
 [0 0 1]
 [0 1 0]]
New rows:
[[0 1 0]
 [0 0 0]]
Pandas columns: ['blue', 'green', 'red']
Pandas rows:
[[0 0 1]
 [1 0 0]
 [0 0 1]
 [0 1 0]]
Pandas new columns: ['green', 'yellow']
Aligned Pandas rows:
[[0 1 0]
 [0 0 0]]
Feature names: ['Color_blue', 'Color_green', 'Color_red']
Empty batch error: Found array with 0 sample(s) (shape=(0,)) while a minimum of 1 is required.
Default unknown-category error: Found unknown categories ['yellow'] in column 0 during transform.

The first training row is red, so its final position is 1.

Step 2: Inspect Pandas dummy columns

The Pandas call returns a DataFrame and keeps its source index. NumPy conversion or resetting the index drops those labels. Keep a unique record key through filtering or sorting, then join transformed rows by that key rather than trusting equal counts.

Pass a Series to convert one field. For a mixed DataFrame, select categorical inputs so numeric fields stay outside the conversion.

Reindexing aligns the test columns with training. Keep the raw label separately when the application must flag new values.

Pandas offers two options that change the output shape:

• dummy_na adds a column for missing values.
• drop_first omits a reference category.

Choose these settings to match the model and the missing-data policy.

For other options, see AskPython’s Pandas get_dummies guide.

Step 3: Use OneHotEncoder for training and later rows