Some embodiments provide a method of implementing capacity-aware load balancing across a set of data compute nodes (DCNs) by reducing latency for the set of DCNs. From the set of DCNs, the method identifies (1) a first subset of DCNs including DCNs that have a latency that is higher than an average latency computed for the set of DCNs and (2) a second subset of DCNs including DCNs that have a latency that is lower than the average latency computed for the set of DCNs. For each DCN in the first subset of DCNs, the method assigns to the DCN a weight value that corresponds to a target latency computed for the set of DCNs. Based on the assigned weight values for the first subset of DCNs, the method computes an excess weight value to be redistributed across the second subset of DCNs. The method redistributes the computed excess weight value across the second subset of DCNs.