A system and method for adaptive intra refresh rate setting for each video frame in a video encoder is disclosed. In one embodiment, a video encoding method includes computing channel induced distortion values at multiple intra refresh rates for a coded video frame as a function of channel loss and spatiotemporal content, computing source coding distortion values at the multiple intra refresh rates for the coded video frame as a function of the spatiotemporal content and an operating bit-rate, selecting an intra refresh rate value for one or more video frames subsequent to the encoded video frame as a function of the sum of the corresponding computed channel induced distortion values and the source coding distortion values, and encoding the one or more video frames using the selected intra refresh rate value.