The obvious fix for KeyError has its own failure, and it is worth seeing before you write anything.
data = {'user': {'profile': None}}
data['user']['profile']['city'] # KeyError? No - TypeError
# 'NoneType' object is not subscriptable
data.get('user', {}).get('profile', {}).get('city')
# AttributeError: 'NoneType' object has no attribute 'get'
# .get('profile', {}) returned the real None, not the {} default
- A default only applies when the key is absent
get('profile', {}) returns {} when there is no profile key at all. When the key is present and holds None — which is exactly what an API sends for an empty optional block — you get None back, and the next .get in the chain fails. The default guards the wrong condition.
- Check the type, not just the key
The reliable question at each step is 'is this thing still a dictionary I can look inside?' — not 'did the last lookup succeed?'. isinstance(value, dict) answers it for None, for lists, for strings and for numbers all at once, which is why one check replaces several.
- Missing and empty are different, and the caller decides which matters
Returning default for a genuine None would make 'the field is absent' and 'the field is explicitly null' indistinguishable — and for a nullable column those mean different things. Walk the path, and only substitute the default when the walk itself fails.
Split the path into its parts, then walk them one at a time, keeping track of where you currently are in the structure. What has to be true before each step for the next one to be safe?
Hold a `current` variable that starts as data. For each key, the step is only safe if current is a dict and the key is in it — otherwise the walk has failed and you return the default.
Guard the empty path. current = data; for each key in path.split('.'): if not isinstance(current, dict) or key not in current, return default; otherwise current = current[key]. After the loop, return current — not default, because the walk succeeded.
def deep_get(data, path, default): if not path: raise ValueError(...) current = ____ for key in path.____('.'): if not isinstance(current, ____) or key not in current: return ____ current = current[key] return ____
def deep_get(data, path, default):
if not path:
raise ValueError("path cannot be empty")
current = data
for key in path.split("."):
if not isinstance(current, dict) or key not in current:
return default
current = current[key]
return current
By the time the loop ends, the whole path resolved, so whatever is there is the answer — including None, False, 0 and ''. Writing `return current or default` would quietly replace every falsy value and is the single most common bug in hand-written versions of this function. Two cases here exist only to catch it.
isinstance(current, dict) asks 'can I look inside this at all?' and `key not in current` asks 'is what I want here?'. Drop the first and None or a string raises TypeError; drop the second and a missing key raises KeyError. Both conditions, in that order, because `in` on a non-dict is not always safe either.
Returning a default is right when absence is expected — an optional profile field, a config value with a fallback. It is wrong when absence means the data is broken, because it turns a loud failure into a silent one. That is the same argument as the safe-int exercise, and it is the reason this takes default as an explicit argument rather than defaulting it to None: the caller has to think about it.
If the shape of the response is known and stable, plain subscripting is better — it documents what you expect and fails loudly when the API changes under you, which is information you want. Reach for this when the structure is genuinely optional. For anything larger, validate the payload once at the boundary with a schema library such as Pydantic and work with real objects afterwards; a codebase full of deep_get calls is a sign the parsing never happened.
- Support list indices in the path — "items.0.name" — and decide whether a string that looks like a number should ever be treated as one.
- Add deep_set that creates the intermediate dictionaries as it goes, then think about what should happen when a step exists but is not a dict.
- Accept a list of keys instead of a dotted string, and work out which real key names that makes possible that the string form cannot express.