The obvious one-liner, and what it does with a duplicate. Count the records going in and the entries coming out.
records = [
{'id': 1, 'name': 'ada'},
{'id': 2, 'name': 'grace'},
{'id': 1, 'name': 'linus'}, # same id as the first
]
index = {record['id']: record for record in records}
print(len(index)) # -> 2 three records went in
print(index[1]['name']) # -> 'linus' ada is gone, silently
- Assignment overwrites, and that is usually what you want
A dict cannot hold two entries under one key, so the later record wins. For counting or last-wins configuration that is the correct behaviour and the reason the comprehension is so short. For indexing records it means a row vanished.
- The comprehension has nowhere to put a complaint
It builds the whole dict in one expression, so there is no point at which you could notice the collision. Detecting duplicates means writing the loop out — which is the one situation where the longer form is the better one.
- Comparing lengths afterwards is the tempting shortcut
len(index) != len(records) does tell you a duplicate happened, and it cannot tell you *which* — by then the evidence has been overwritten. Catching it as it happens is what lets the error name the offending value.
The comprehension is one line and cannot report a collision. Write the loop instead, and ask one question before each assignment.
Build an empty dict. For each record, read record[key] — which raises KeyError for a broken record on its own — then check whether that value is already a key before storing.
index = {}. For each record: value = record[key]; if value in index, raise ValueError naming the value; otherwise index[value] = record. Return index. The check has to come before the assignment, or there is nothing left to detect.
def index_by(records, key): index = {} for record in records: value = record[____] if value ____ index: raise ValueError(...) index[value] = ____ return index
def index_by(records, key):
index = {}
for record in records:
value = record[key]
if value in index:
raise ValueError(
f"duplicate {key}={value!r} - two records share it"
)
index[value] = record
return index
Once index[value] = record has run, the previous record is gone and no later comparison can recover which value collided. That ordering is the whole reason this cannot be a comprehension — a one-expression form has no 'before'.
Grouping expects several records per key and collects them in a list; indexing expects exactly one and refuses more. Choosing between them is really a question about your data, and answering it wrongly is how a report ends up quietly missing rows. If you are not certain the field is unique, group.
record[key] raises KeyError naming the field for a malformed record, so there is nothing to add. That is the fourth exercise where the right amount of error handling turned out to be none — the pattern is that built-in errors are good when they already name the thing that was wrong, and worth replacing when they do not.
If the data comes from a database, index on a column with a unique constraint and let the database enforce it — you will find out at write time rather than at read time. If duplicates are expected and meaningful, use group_by instead of catching an exception you knew was coming. And for a single lookup over a small list, scanning is fine: building an index pays off when you will look things up more than once, which is the same trade as the nested-loop refactor.
- Add an on_duplicate option offering 'raise', 'first' and 'last', and decide which should be the default for an import job.
- Return the duplicates as a second value instead of raising, so a caller can report every collision rather than the first.
- Index by a computed key — a lowercased email, say — and notice that two records can now collide even though their raw fields differ.