Encoding categorical data: one-hot and ordinal encoding

A Titanic passenger record can contain a numerical age and a letter such as S for the embarkation port. The letter identifies a category; it is not an amount. Many learning algorithms expect numerical inputs, so we need a representation that preserves what the category actually means.

Categorical encoding represents category values in a form an algorithm can use. One-hot encoding creates membership indicators for named categories. Ordinal encoding assigns codes in an explicitly chosen order when that order makes sense. Converting words into numbers does not by itself establish meaningful distances or quantities.

We will continue with Seaborn’s Titanic teaching data. In its embarked field, C means Cherbourg, Q means Queenstown and S means Southampton. These ports have no natural rank for our prediction task. Ticket classes First, Second and Third do have an order, although their differences are not equal measured amounts. Review missing values and training-only preparation first if those ideas are unfamiliar.

A nominal category is a name without a natural order: C, Q and S identify ports. An ordinal category has an intended order: First, Second and Third identify ticket-class ranks. A numerical quantity, such as age in years, describes an amount where differences have a measurement interpretation.

The source already stores pclass as 1, 2 and 3. That does not make Third “three times First,” or guarantee that moving between adjacent classes has the same predictive effect. Meaning comes from the source and task, not merely the column’s data type.

Suppose we assign C=0, Q=1 and S=2 simply because the names sort that way. A distance-based algorithm would see a numerical gap of one between C and Q, but two between C and S. We have supplied a relationship the port codes themselves do not establish.

A linear model using one such numerical input also imposes a particular numerical relationship through its coefficient. These codes do not measure geographical distance or a passenger’s circumstances. Changing the arbitrary mapping can change what the algorithm learns. This matters when you later use K-nearest neighbors and feature distances.

One-hot encoding avoids that ordered scalar representation by giving each category its own membership column. It still makes a modelling choice: distinct known categories are different indicator patterns, not a learned map of how similar the ports are. Some algorithms support categorical inputs directly, but require their own documented preparation.

Prepare and run

Run this complete Python program in the course environment with scikit-learn and NumPy. All input values are included; no dataset download is required.

Python · complete runnable example

from sklearn.preprocessing import OneHotEncoder

# Embarkation codes from Titanic source rows 0 to 4.
train = [["S"], ["C"], ["S"], ["S"], ["S"]]
encoder = OneHotEncoder(handle_unknown="ignore", sparse_output=False)
encoder.fit(train)
print("Categories:", encoder.categories_[0].tolist())
print("Columns:", encoder.get_feature_names_out(["embarked"]).tolist())
print("Known C:", encoder.transform([["C"]]).tolist())
print("Known S:", encoder.transform([["S"]]).tolist())
# Q occurs in the source dataset, but not in this training subset.
print("Unseen Q:", encoder.transform([["Q"]]).tolist())

Expected output

Categories: ['C', 'S']
Columns: ['embarked_C', 'embarked_S']
Known C: [[1.0, 0.0]]
Known S: [[0.0, 1.0]]
Unseen Q: [[0.0, 0.0]]

Follow the representation

The training values repeat the embarked entries from source rows 0–4. They contain S and C; Q is present elsewhere in Titanic but absent from this deliberately small subset. The nested lists have five rows and one input column, the shape the encoder expects.

Fit discovers the category vocabulary. For these strings, the fitted order is C then S, regardless of which appeared first in the input. Categories_ shows that order, and get_feature_names_out attaches the field name to each output category. Transform then activates the matching column: C becomes [1, 0] and S becomes [0, 1].

Sparse_output=False gives a dense array so we can read the values easily. The .0 in the output is the encoder’s numerical representation, not a probability. Tolist converts the array to nested Python lists for display. We selected handle_unknown="ignore" explicitly; the next section explains that decision. See the OneHotEncoder API for these settings.

In this fitted example, the columns are embarked_C and embarked_S. Incoming Q therefore becomes [0, 0] under the ignore policy. Neither known indicator is active. This does not mean that Q has no passengers, is unimportant or has zero survival probability; no predictive relationship for Q was learned here.

The default unknown-category policy raises an error rather than silently accepting an unseen category. Ignoring can keep a request running, but several different unseen values can then share the same all-zero representation. A deliberate “other” group, an approved fixed vocabulary or rare-category grouping may suit another task. Choose and evaluate the policy instead of assuming one behavior is universally safe.

Do not add a new Q column during prediction while leaving the trained model unchanged. Its input shape and meanings were established during fitting. An updated vocabulary generally belongs in a coordinated preparation/model update. Track unexpected categories so a typo such as “s” does not quietly become a new port.

Prepare and run

Run this complete Python program in the course environment with scikit-learn and NumPy. All input values are included; no dataset download is required.

Python · complete runnable example

import numpy as np
from sklearn.impute import SimpleImputer
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import OneHotEncoder

# Embarkation entries from source rows 0, 1 and 61.
train = np.array([["S"], ["C"], [np.nan]], dtype=object)
prepare = make_pipeline(
    SimpleImputer(strategy="constant", fill_value="Unknown"),
    OneHotEncoder(handle_unknown="ignore", sparse_output=False),
)
prepare.fit(train)
encoder = prepare.named_steps["onehotencoder"]
print("Categories:", encoder.categories_[0].tolist())
print("Missing port:", prepare.transform(np.array([[np.nan]], dtype=object)).tolist())
print("Unseen Q:", prepare.transform(np.array([["Q"]], dtype=object)).tolist())

Expected output

Categories: ['C', 'S', 'Unknown']
Missing port: [[0.0, 0.0, 1.0]]
Unseen Q: [[0.0, 0.0, 0.0]]

This program repeats the selected input entries independently. S, C and the absent embarked value come from source positions 0, 1 and 61. NumPy’s object array holds category strings and a numerical missing marker together. The input is a tiny mechanism demonstration, not a representative training set.

The pipeline first replaces the missing entry with the explicit text Unknown, then learns one-hot columns from the prepared training inputs. Because Unknown occurs during this fit, it gets a column. A later missing value activates that column; Q remains unseen and activates none of the known columns.

Make_pipeline connects the steps. Fit learns their preparation in sequence, and transform runs later inputs through the already-fitted steps. Named_steps lets us inspect the fitted onehotencoder. We have not added a survival model, and the Unknown text is a chosen marker rather than a historical port.

This policy needs care: a marker must not collide with a real category, and a missing marker not seen during fitting would not automatically gain its own column. Other policies include a most-frequent replacement or explicit missingness indicators, with different assumptions. Keep the distinction from the previous missing-value lesson visible.

Ticket classes First, Second and Third have a rank. An explicit ordinal mapping can assign First=0, Second=1 and Third=2. The code preserves the intended ordering; its origin at zero is a representation choice, not a statement that First has zero value.

Those codes also have equal numerical spacing. A linear model using one code column assumes a particular relationship across successive steps. A distance-based model sees equal gaps between neighboring classes. The class names alone do not justify equal differences in fare, comfort or survival probability.

You can instead one-hot encode ticket class when you want separate category effects without that scalar-spacing assumption. An ordinal representation can suit other models and tasks. Decide using the category meaning, the algorithm’s interpretation and validation results, rather than a rule that every ordered field must use a single numerical column.

Prepare and run

Run this complete Python program in the course environment with scikit-learn and NumPy. All input values are included; no dataset download is required.

Python · complete runnable example

from sklearn.preprocessing import OrdinalEncoder

order = [["First", "Second", "Third"]]
train = [["Third"], ["First"], ["Second"]]
encoder = OrdinalEncoder(categories=order)
encoder.fit(train)
print("Order:", encoder.categories_[0].tolist())
print("First, Third, Second:", encoder.transform([["First"], ["Third"], ["Second"]]).tolist())

Expected output

Order: ['First', 'Second', 'Third']
First, Third, Second: [[0.0], [2.0], [1.0]]

The program uses the ticket-class names from the source vocabulary, arranged into a small teaching input. Order is a list containing the allowed order for our single input column. Supplying categories=order prevents us from relying on alphabetical discovery or the order in which labels happen to arrive.

OrdinalEncoder returns one code per input column rather than one column per category. First maps to 0, Third to 2 and Second to 1. Transform preserves the input row order; it does not sort the passengers. The default encoder rejects unknown category names, so validate inputs or configure a reviewed policy.

These class codes are model inputs, not a survival prediction. If using a reserved code such as -1 for unknown values, remember that it is a marker: a numerical model might interpret it as below the first rank unless your chosen representation and algorithm account for that.

The OrdinalEncoder reference documents explicit categories and unknown handling. The source’s pclass values 1–3 express a similar rank with different code labels; neither numbering establishes equal real-world differences.

A fitted model sees numbers in specific positions. In the port example, [1, 0] means C because the columns are C then S. Swapping them would silently turn the same vector into the representation for S. Two arrays can have identical sizes and still disagree about their meanings.

Fit category discovery on the training portion, then reuse the fitted encoder for validation and prediction. Keep the column order, transformations and model together. A Pipeline and, for tables with different kinds of columns, a ColumnTransformer help apply the intended preparation consistently. During cross-validation, learned preprocessing belongs inside each training fold.

ColumnTransformer can apply one preparation route to numerical ages and another to categorical ports, then combine the outputs. See its official API and the later Titanic project’s mixed-column pipeline. A documented fixed vocabulary is different from learning the categories by inspecting a reserved test set.

Pandas get_dummies is convenient for exploring a table or producing indicator columns. This complete Python snippet shows the same two known ports, with integer indicators requested explicitly.

import pandas as pd

ports = pd.DataFrame({"embarked": ["S", "C", "S"]})
encoded = pd.get_dummies(ports, columns=["embarked"], dtype=int)
print("Columns:", encoded.columns.tolist())
print("Rows:", encoded.to_numpy().tolist())

Expected output

Columns: ['embarked_C', 'embarked_S']
Rows: [[0, 1], [1, 0], [0, 1]]

Separate calls on training and prediction files can discover different columns. If a later file contains only Q and S, its independently generated output does not automatically match the model’s C/S input contract. Reindexing can align columns to a stored training schema, but your unknown and missing-value policies still need an explicit decision.

For a fitted prediction workflow, OneHotEncoder gives you a reusable learned vocabulary and unknown policy. The get_dummies reference explains missing-value indicators, column selection and output types. Choose a tool according to the workflow rather than treating a convenient conversion as a complete preparation plan.

You fitted the encoder below on the two-category vocabulary C and S. Before running it, predict the output for S and Q. Then explain what would change if Q were genuinely included in a new training dataset.

from sklearn.preprocessing import OneHotEncoder

train = [["S"], ["C"], ["S"]]
encoder = OneHotEncoder(handle_unknown="ignore", sparse_output=False).fit(train)
print("Columns:", encoder.categories_[0].tolist())
print("S and Q:", encoder.transform([["S"], ["Q"]]).tolist())

Worked answer

Columns: ['C', 'S']
S and Q: [[0.0, 1.0], [0.0, 0.0]]

S activates the second column. Q is not in the fitted vocabulary, so the chosen policy produces zeros for both known columns. Reading the fitted column names is essential; the positions are not universal codes for ports.

If you fit a fresh encoder on training inputs containing C, Q and S, its category order is C, Q, S and Q becomes [0, 1, 0]. That is a new three-column representation. A model trained on the previous two-column layout cannot simply receive it unchanged; preparation and the model must be updated together. Including a reserved test category merely to discover this vocabulary would violate the intended separation.

One-hot encoding can create many columns when a feature has many distinct values. Sparse storage records mainly the non-zero entries instead of storing every zero, which can reduce memory use. Our tiny examples use dense output for visibility; this is not a recommendation to densify large matrices. Rare-category grouping or other representations can help, but must be learned and evaluated within the training workflow.

Dropping one category produces a reference-category representation. This can help avoid redundant columns in some linear-model settings with an intercept, but it is not required for every estimator. It also changes what an all-zero row means: it can now represent the dropped known category, which complicates an ignored unknown. Our main example keeps all fitted category columns.

LabelEncoder is intended for target labels, not as a general encoder for unordered input features. A classifier can interpret target codes as distinct class identities; that differs from giving an input algorithm an artificial numerical distance between port names. See the LabelEncoder API.

For predicting survival, the survived field is the target. Putting it into the feature table gives the model the answer, rather than teaching it to predict from available inputs. Encoding a leaked field does not make it a valid feature.

Encoding makes category information usable without changing its underlying meaning. Our Titanic ports needed membership indicators; ticket classes had a meaningful rank whose spacing still needed careful interpretation.

• Identify the field: distinguish names, ordered categories and measured quantities.
• Trace the columns: in the fitted C/S example, S activates the second indicator. A one is membership, not importance or probability.
• Choose explicit policies: a missing port and an unseen port are different cases; an all-zero row depends on the encoder settings.
• Reuse preparation: preserve the fitted vocabulary, column order and model together. Do not independently rebuild prediction columns.
• Check assumptions: ordinal spacing, sparse output, dropped reference categories and target labels each have different implications.

Before the next lesson: explain why encoding port names is different from adjusting the numerical scale of age or fare. Next, we will compare feature scaling and normalization without changing what the measurements mean.