01Brief08 / 14

Two numbers that disagree, and the one that combines them

Write scores(actual, predicted, positive=1) returning precision, recall and F1.

Function	scores()
Track	Data without the library
Difficulty
Time	~24 min
Concepts	precision versus recall undefined metrics the harmonic mean
Cases	11

02The contract

Precision is the share of positive predictions that were right.
Recall is the share of actual positives that were found.
F1 is the harmonic mean of the two: 2 · precision · recall / (precision + recall).
When nothing was predicted positive, precision is undefined and reported as 0.0.
When there were no actual positives, recall is undefined and reported as 0.0.
When precision and recall are both 0, F1 is 0.0 rather than a division by zero.
All three are floats, and all three keys are always present.
The two lists must be the same length: raise ValueError otherwise.

03Why this one

Precision and recall pull in opposite directions and that is the whole point of having both: you can always improve one by sacrificing the other, and where you sit on that trade is a product decision rather than a modelling one. The reason to write them by hand once is the arithmetic nobody mentions — all three are undefined for a model that predicts nothing, and what your code does in that case decides whether a broken model reports 0.0 or crashes your evaluation run.

—The concept behind it

Lists on pythonmastery.io →

That is the reading half. Read it there if the idea is new, then come back here — explanation is what makes something make sense, and practice is what makes it stay.

—Worth doing first

Four numbers that describe a classifier

Not a gate — start this one whenever you like. These cover ideas it leans on, and a tick means you have already done them.

—What this leads to

Ninety-nine per cent accurate and completely useless

These build on what you are about to write here.

04Study first: the same denominator question, twice

Both metrics count the true positives. They differ only in what they divide them by — and each denominator can be zero.

#             predicted positive        actually positive
# precision = tp / (tp + fp)    recall = tp / (tp + fn)
#
#   precision: of the alarms we raised, how many were real?
#   recall:    of the real ones, how many did we catch?

# A model that predicts positive exactly once and is right:
#   precision 1.0, recall maybe 0.01 — technically perfect, useless

# A model that predicts positive for everything:
#   recall 1.0, precision = the base rate — also useless

# 2 * p * r / (p + r)      the harmonic mean: 0.0 when either is 0

Both are true positives over a denominator, and the denominators differ
Precision divides by everything you predicted positive; recall divides by everything that actually was. That is the only difference, and it is why they move in opposite directions: predicting positive more often raises the recall denominator not at all and the precision one a lot.
Each denominator can be zero, for different reasons
No positive predictions means precision has nothing to be a share of — a property of the model. No actual positives means recall has nothing to find — a property of the data, usually a test set too small or too imbalanced. Reporting 0.0 for both is a convention, not a truth; it is what sklearn does with zero_division=0, and it is defensible because the alternative crashes an evaluation loop halfway through.
The harmonic mean punishes imbalance, which is why it is not the average
Precision 1.0 and recall 0.02 average to 0.51 and look respectable; their harmonic mean is 0.039, which is the honest number. F1 is close to the smaller of the two by design, so a model can only score well on it by being reasonable at both — and its own denominator is zero exactly when both are, which needs its own guard.

your_code.py

05One way to write it

def scores(actual, predicted, positive=1):
    if len(actual) != len(predicted):
        raise ValueError(
            f"actual has {len(actual)} labels, predicted has {len(predicted)}"
        )

    true_positive = false_positive = false_negative = 0

    for truth, guess in zip(actual, predicted):
        was = truth == positive
        said = guess == positive

        if said and was:
            true_positive += 1
        elif said:
            false_positive += 1
        elif was:
            false_negative += 1

    # Each metric's denominator can legitimately be zero, and each zero
    # means something different. The convention is to report 0.0.
    predicted_positive = true_positive + false_positive
    actual_positive = true_positive + false_negative

    precision = true_positive / predicted_positive if predicted_positive else 0.0
    recall = true_positive / actual_positive if actual_positive else 0.0

    total = precision + recall
    f1 = 2 * precision * recall / total if total else 0.0

    return {
        "precision": precision,
        "recall": recall,
        "f1": f1,
    }

Three guards, because there are three denominators

It is tempting to write one check — 'if there were no positives, return all zeros' — and it is wrong, because the no-actual-positives case can still have a real precision, as one of the cases above shows. Each division gets the guard that belongs to it. The F1 guard is the one most often forgotten, because it only fires when both the others already did.

True negatives are never counted, and that is the point

Neither metric mentions them, which is exactly why they are the right metrics for rare events: a test set of 99% negatives cannot inflate them the way it inflates accuracy. If you find yourself wanting the true negatives, the metric you are reaching for is specificity or the false positive rate, and neither belongs in F1.

F1 built from precision and recall, not from the counts

The algebraically equivalent form — 2·tp / (2·tp + fp + fn) — is shorter and skips the zero-division question entirely, which sounds like an advantage and means the guard that reflects the real structure is gone. Deriving F1 from the two numbers it is defined in terms of keeps the relationship visible and makes the third guard obviously necessary.

06When not to write this at all

sklearn.metrics.precision_recall_fscore_support gives all of this plus the support counts, handles multiple classes with micro, macro and weighted averaging, and takes an explicit zero_division argument so the convention is a decision rather than an accident. Use it. What it cannot do is choose your operating point: F1 weights precision and recall equally, and almost no real problem does. F-beta lets you say which one matters more, and for anything where the threshold is tunable the honest report is a precision-recall curve and the average precision under it, not one number at one arbitrary cut-off.

07Take it further

Add F-beta with a beta argument, and check that beta of 1 reproduces your F1 exactly.
Take probabilities rather than labels plus a threshold, and plot precision and recall against the threshold — the trade becomes a shape rather than two numbers.
Add specificity, then work out which of the four confusion counts it needs and why none of the three metrics here wanted it.