Embodiments of the present invention provide techniques for coding video data efficiently based on detection of objects within video sequences. A video coder may perform object detection on the frame and when an object is detected, develop statistics of an area of the frame in which the object is located. The video coder may compare pixels adjacent to the object location to the object's statistics and may define an object region to include pixel blocks corresponding to the object's location and pixel blocks corresponding to adjacent pixels having similar statistics as the detected object. The coder may code the video frame according to a block-based compression algorithm wherein pixel blocks of the object region are coded according to coding parameters generating relatively high quality coding and pixel blocks outside the object region are coded according to coding parameters generating relatively lower quality coding.