Two ways to turn three colours into numbers. Only one of them avoids making a claim nobody meant.
categories = ['red', 'green', 'blue']
# Label encoding: compact, and it invents an ordering
# red 0, green 1, blue 2
# the model can now learn that blue > green > red, and that the
# average of red and blue is green
# One-hot: three independent columns, no ordering implied
# red -> [1, 0, 0]
# green -> [0, 1, 0]
# blue -> [0, 0, 1]
# The vocabulary comes first, and it fixes the width forever
position = {name: index for index, name in enumerate(categories)}
- Numbering categories smuggles in an ordering
Anything linear will treat the label 2 as twice the label 1, because that is what numbers mean — so a model can conclude that blue is more than green, and that a value halfway between them is meaningful. One-hot removes the possibility by giving each category a dimension of its own. The exception is a genuinely ordered category — small, medium, large — where the numbering is the information and one-hot throws it away.
- The vocabulary is part of the model, not part of the data
The width of every row is len(categories), so the vocabulary has to be decided once, from the training data, and then used unchanged forever. Re-deriving it from whatever values you happen to be encoding gives rows of different widths on different batches, and a model that has been trained on five columns cannot be fed four.
- The unseen category is the decision that matters
A colour that was not in the training data has no column. You can refuse, which stops a bad row entering the system; emit all zeros, which quietly says 'none of the above'; or keep a spare column for unknowns, which is the honest version if you expect new categories. sklearn’s default is to refuse, and that is the right default — silently encoding an unknown product as zeros is how a model starts making confident predictions about nothing.
Build a lookup from category to position once, then every row is a list of zeros with one of them set. Validate the vocabulary before you use it.
enumerate(categories) gives the positions. Check for duplicates by comparing the length of the lookup against the length of the categories — a repeated name collapses into one key.
[0] * width then set one index is clearer here than a comprehension over positions, and it is what makes the single 1 obvious. Look the value up with `in` before indexing, so the error can name the value rather than being a KeyError about a lookup table the caller has never seen.
def encode(values, categories): if not categories: raise ____('no categories to encode into') position = {name: index for index, name in ____(categories)} if len(position) != len(____): raise ValueError('categories contains a repeat') width = len(categories) rows = [] for value in values: if value not in position: raise ____(f'{value!r} is not one of the categories') row = [____] * width row[position[value]] = ____ rows.append(row) return rows
def encode(values, categories):
if not categories:
raise ValueError("no categories to encode into")
position = {name: index for index, name in enumerate(categories)}
# A repeat collapses into one key, so the lengths disagree.
if len(position) != len(categories):
raise ValueError(
f"categories contains a repeat: {categories}"
)
width = len(categories)
rows = []
for value in values:
if value not in position:
raise ValueError(
f"{value!r} is not one of the categories {categories}"
)
row = [0] * width
row[position[value]] = 1
rows.append(row)
return rows
Building the lookup collapses repeats, so its length is the number of distinct categories — the same trick as invert-mapping, and it borrows the dictionary’s own idea of equality rather than reimplementing one. It also means the check costs nothing beyond the lookup you were building anyway.
position[value] on a missing key raises KeyError with just the key in it, and KeyError is the wrong type for a caller catching ValueError. Checking first lets the message say which value and what the alternatives were — which for a typo’d category is usually the whole diagnosis.
Multiplying a list of immutable values copies the value, so this gives genuinely independent zeros. [[0] * width] * rows would *not* work: it repeats a reference to one row, and setting a 1 in one of them sets it in all. That is the aliasing bug this exercise is one line away from, and building each row inside the loop avoids it entirely.
sklearn.preprocessing.OneHotEncoder is the real tool: it learns the vocabulary in fit(), applies it in transform(), and has handle_unknown for the decision this exercise refuses. pandas.get_dummies is the quick version and has a trap — it derives the columns from the data it is given, so training and inference silently produce different widths, which is the bug the 'never appears' case here is about. Beyond a few dozen categories, one-hot stops being sensible: a hundred thousand product ids means a hundred thousand columns, and target encoding or a learned embedding is what gradient-boosted trees and neural networks actually use. And for genuinely ordered categories, keep the numbering — the ordering is information, not an artefact.
- Add an unknown='zeros' option and decide from the arguments in the notes which you would make the default.
- Add a spare final column for unknown categories, and work out what that does to the width and to every previously encoded row.
- Write the decoder — rows back to values — and check the round trip, then see what it does to a row of all zeros.