Two lines that look obviously correct. Count how many times the predicate runs.
def partition_by(items, predicate):
return (
[item for item in items if predicate(item)],
[item for item in items if not predicate(item)],
)
# Ten items -> twenty calls, and every one of them is asked twice.
#
# If predicate() checks a cache, that doubles the misses.
# If it consumes from a queue, the second call sees a different queue.
# If it is expensive, you have doubled the cost of the function.
- A predicate is a black box, so call it once
You did not write it and you cannot see it — it may be pure, or it may count, cache, log or charge somebody. Calling it once per item is the only contract you can offer that holds regardless. This is the same reason sorted() calls its key function once per element and not once per comparison.
- `not predicate(item)` is not the opposite of `predicate(item)`
It is the opposite of a *second, independent* call. For a pure predicate those coincide; for anything stateful they do not, and an item can end up in both lists or in neither — a result that is not merely wrong but structurally impossible, which makes it very confusing to debug.
- One loop, two accumulators
Ask once, store the answer, append to whichever list it selects. The shape generalises directly: three buckets, or n buckets keyed by a function’s return value, is the same loop with a dictionary instead of two names — which is what itertools' recipes and groupby are doing underneath.
One loop, two lists. Ask the predicate once, keep the answer in a variable, and use it to choose which list to append to.
Start with matching = [] and rest = []. For each item: answer = predicate(item), then append to matching if the answer is truthy and to rest otherwise. Return them as a tuple.
`if predicate(item):` inside the loop is already once per item — the variable is only needed if you use the answer twice. Use plain truthiness rather than comparing against True, so a predicate returning a string or a list behaves the way an `if` would.
def partition_by(items, predicate): matching, rest = [], [] for item in items: if ____(item): matching.append(item) else: ____.append(item) return (____, rest)
def partition_by(items, predicate):
matching = []
rest = []
for item in items:
# Once per item. The predicate belongs to the caller and may do
# anything at all — including something different the second time.
if predicate(item):
matching.append(item)
else:
rest.append(item)
return (matching, rest)
There is a clever version using a dict keyed on bool(predicate(item)) and it is shorter, harder to read, and no faster. The value of this function is a promise about how many times the predicate runs, and the explicit loop is the version where that promise is visible on the page. Prefer the shape whose important property you can see.
Predicates in the wild return whatever is convenient: a match object, a non-empty list of errors, a string, a number. Treating the answer the way `if` does means all of those work, which is what callers expect from anything called a predicate — filter() and any() make the same choice.
matching, rest = partition_by(...) reads as one step, and a tuple says the size is fixed. It also means the two lists cannot be reordered or appended to by mistake — a caller who wants to combine them has to say so.
If the predicate is genuinely pure and cheap and you only need one of the halves, just write the comprehension — this function earns its place when you need both. For more than two groups, group-by with a key function is the right generalisation rather than nested partitions. more_itertools.partition exists and returns lazy iterators, which is better for large inputs and worse here, because it uses tee internally and buffers whichever half you have not consumed.
- Return two generators instead of two lists and work out what that does to the once-per-item promise.
- Generalise to n groups keyed by a function’s return value, and see how much of the loop survives.
- Add a limit that stops once the matching list is full, and decide what the rest should then contain.