A system and method relate to determining am initial buffer size associated with a buffer and modifying the initial buffer size when jitter is detected. The initial buffer size may be determined and modified based on a maximum initial buffer size, and the maximum initial buffer size may be determined based on delays associated with storing data to the buffer. For example, the maximum initial buffer size may correspond to a quantity of data that can be stored to the buffer during a maximum acceptable buffering delay. The maximum acceptable buffering delay may be identified by determining drop rates associated with different buffering delays and selecting, as the maximum acceptable buffering delay, one of the buffering delays associated with a maximum acceptable drop rate.