Aspects of the subject matter described herein relate to locally terminating an established connection. In aspects, a connection between two nodes is established. A network device lies on the path over which packets associated with the connection travel to get to either of the two nodes. After the connection is established, the network device attempts to create a safe point at which the connection can be terminated at the network device. To do so, the network device begins buffering packets and looking for information that indicates that all outstanding packets between the two nodes have been received either by one of the two nodes or have been buffered by the network device. After a safe point is reached, the network device may terminate the connection locally in a way that is transparent to the two nodes.