Recursion over a tree, where the thing being built up is the name rather than the value.
data = {'db': {'host': 'x', 'port': 5432}, 'debug': True}
# Each level knows only its own key and the prefix it was handed
def walk(data, prefix):
for key, value in data.items():
path = prefix + '.' + key if prefix else key
if isinstance(value, dict):
yield from walk(value, path)
else:
yield path, value
dict(walk(data, ''))
# -> {'db.host': 'x', 'db.port': 5432, 'debug': True}
# And the ambiguity, if a key contains the separator
{'a.b': {'c': 1}} and {'a': {'b.c': 1}} # both -> 'a.b.c'
- The prefix goes down, the pairs come back up
Each call is handed where it is and adds one segment before recursing. Nothing needs to know the whole tree, and there is no shared dictionary being filled in from several places — every level returns its own pairs and the caller assembles them. That is the difference between recursion that reads well and recursion that needs a diagram.
- The empty prefix is why the top level has no leading separator
`prefix + separator + key if prefix else key` is the whole special case, and it lives in one place. The alternative — starting with a separator and stripping it at the end — works and puts the correction a long way from the mistake. Handle the base case where it happens.
- Flattening is only reversible if the separator is unambiguous
Two genuinely different structures can flatten to the same key when a key contains the separator, so the operation stops being invertible and any code that later unflattens gets it wrong. You cannot fix that by escaping without inventing a whole encoding — the honest answer is to refuse the input and say why.
Write a helper that takes the data and the path so far, and have it call itself when it meets a dictionary. The outer function does the validating; the helper does the walking.
A nested value only recurses when it is a non-empty dict — an empty one has nothing to contribute and must stay a leaf. Build the child’s path as prefix + separator + key, except at the top where there is no prefix yet.
A generator helper that yields (path, value) pairs keeps the assembling in one place: dict(walk(data, '')). Check the separator once, before the walk, and check each key for the separator as you meet it — the nested keys are the ones people forget.
def flatten_dict(data, separator='.'): if not separator: raise ____('separator cannot be empty') def walk(current, prefix): for key, value in current.items(): if separator in str(key): raise ValueError(f’key {key!r} contains the separator') path = prefix + separator + key if ____ else key if isinstance(value, dict) and ____: yield from walk(value, path) else: yield path, ____ return dict(walk(data, ''))
def flatten_dict(data, separator="."):
if not separator:
raise ValueError("separator cannot be empty")
def walk(current, prefix):
for key, value in current.items():
if separator in str(key):
raise ValueError(
f"key {key!r} contains the separator {separator!r}"
)
path = prefix + separator + key if prefix else key
# An empty dict has no leaves, so recursing would lose the key.
if isinstance(value, dict) and value:
yield from walk(value, path)
else:
yield path, value
return dict(walk(data, ""))
Without it, {'a': {}} recurses into nothing, yields nothing, and the key vanishes from the output — a silent deletion that only shows up when somebody notices a section missing from their config. Deciding that an empty container is a leaf is a judgement call, and it is the one that keeps the transformation total: every input key reaches the output.
Yielding pairs and calling dict() once means no shared accumulator is passed down and mutated, so no level can overwrite another’s work by accident. yield from is what makes the recursion read as flatly as the iteration it produces — the nested calls' pairs simply appear in the stream.
Checking only the top-level keys first would miss {'a': {'b.c': 1}}, and walking the whole structure twice to validate it would duplicate the traversal logic. Checking each key at the moment you use it costs nothing and cannot go out of sync with the code it protects.
Do not flatten just to look something up — deep-get reaches into the nested structure without rebuilding it, and keeps the shape for everything else. Flatten when the destination is genuinely flat: environment variables, a metrics label set, spreadsheet columns, a diff you want to read. If you need it in both directions, write unflatten alongside it and test the round trip, because that is where the separator ambiguity bites. pandas.json_normalize does this with far more control over lists and record paths if the destination is a DataFrame.
- Write unflatten(flat, separator='.') and check that it reproduces every input here, then work out why the refused cases could not have round-tripped.
- Index into lists as well, so 'tags.0' is a key, and decide what that does to the round trip when a list is empty.
- Add a max_depth that stops flattening and leaves the remaining structure as a leaf value.