No merging here at all — just the copy problem underneath it. Read the last two lines slowly, because this is the bug the exercise is really about.
settings = {'db': {'host': 'localhost'}}
shallow = dict(settings) # or settings.copy(), or {**settings}
shallow['db']['host'] = 'prod' # looks like it only touches the copy
print(settings['db']['host']) # -> 'prod'
# the outer dict was copied; the inner one is the same object as before
- A shallow copy copies references, not the things they point to
dict(settings) builds a new outer dictionary whose values are the very same objects. The two dicts now share every nested structure inside them, so a change reached through one is visible through the other. Nothing about the syntax warns you.
- This bug is silent and delayed
The corruption happens at whatever moment somebody writes into a nested value — usually far away, much later, in code that has every right to assume its own dictionary is its own. Tests that only check the returned value pass happily. That is why the contract here forbids sharing, not just modifying.
- copy.deepcopy is the blunt fix, and usually the right one
It rebuilds the whole structure, so nothing is shared. It costs time and memory proportional to the data, and it will choke on things like open files. For configuration — small, plain, made of dicts and lists and strings — that trade is obviously worth it.
Start from a copy of base — think hard about how deep that copy needs to be — then walk override’s items and decide, per key, whether to merge or to replace.
The merge case is narrow: it applies only when the value in override is a dict AND the value already sitting in the result is a dict. Everything else is a replacement. Use result.get(key) so a key that is missing from base does not raise.
result = copy.deepcopy(base). For each key, value in override.items(): if value is a dict and result.get(key) is a dict, set result[key] to merge_config(result[key], value) — the function calls itself, that is the recursion. Otherwise set result[key] to a deep copy of value. Return result.
import copy def merge_config(base, override): result = copy.____(base) for key, value in override.items(): if isinstance(value, dict) and isinstance(result.get(key), ____): result[key] = ____(result[key], value) else: result[key] = copy.deepcopy(____) return result
import copy
def merge_config(base, override):
result = copy.deepcopy(base)
for key, value in override.items():
if isinstance(value, dict) and isinstance(result.get(key), dict):
result[key] = merge_config(result[key], value)
else:
result[key] = copy.deepcopy(value)
return result
Testing only isinstance(value, dict) breaks the moment base holds a string where override holds a dict — you would try to merge into something with no keys. Two cases in the set exist purely to pin down this pair of checks, in both directions, because it is the condition people write half of.
Copying base is the obvious half. But `result[key] = value` in the else branch stores override’s own list or dict, so the returned config now shares mutable state with the caller’s override — and appending to a list in the result silently edits the override that produced it. No test that only compares returned values will ever catch it. Copy on the way in, and stop thinking about ownership.
Each call descends exactly one level into a finite structure, so it cannot run away — unless the config contains a cycle, which JSON and YAML cannot express. Note the recursive call passes result[key], which is already copied, so the work is not repeated: the deepcopy at the top of each call is copying an ever-smaller subtree.
Do not deep-merge lists of dicts and expect anyone to predict the result — there is no obvious answer to whether the third element of one list corresponds to the third of the other, and every library that tries picks a different rule. Do not deepcopy when the config holds live objects such as connections or open files; deepcopy will either fail or duplicate something that must not be duplicated. And if you find yourself merging four layers, reach for a library built for it — Dynaconf, Hydra or Pydantic settings — which also handle types, validation and environment variables.
- Add a `list_strategy` option offering replace, append or unique-append, then write down which one a user would expect for 'plugins' and which for 'allowed_hosts'.
- Track where each final value came from so you can answer 'why is this setting 30?' — return the merged config alongside a parallel dict of origins.
- Make it merge n layers, merge_all(defaults, file, env, flags), and decide whether that is a fold over this function or something that deserves its own implementation.