Flow state information that is stored in a first memory among a plurality of memories for maintaining flow state information at a network device is updated based on packets ingressing the network device. The memories are arranged in a hierarchical arrangement in which memories at progressively higher levels of hierarchy are configured to maintain flow state information corresponding to progressively larger sets of flows processed by the network device. When it is determined that a fullness level of the first memory exceeds a first threshold, flow state information associated with at least one flow, among a first set of flows for which flow state information is currently being maintained in the first memory, is transferred from the first memory to a second memory, the second memory being at a higher hierarchical level than the first memory. A new flow is instantiated in space freed up in the first memory.