A method includes, with a hypervisor, receiving, with a Physical Function (PF) of a network device, a data packet destined for a virtual machine (VM), forwarding the data packet to the VM, detecting an event that indicates that a driver for a virtual network device associated with the VM is up, in response to detecting the event, programming a device address filter of the PF, and with the network device, after the device address filter is programmed, forwarding incoming data packets destined for the VM directly to a Virtual Function (VF) associated with the virtual network device.