A data compression system includes: (a) a data compression module that receives a sequence of input vectors and that provides a sequence of compressed vectors; (b) a data decompression module that receives the compressed vectors to provide a sequence of output vectors; and (c) a parameter update module that receives the sequence of input vectors and the sequence of output vectors, and which learns the data compression module and data decompression module based on evaluating a loss function of the input vectors, the output vectors, and the parameters controlling the compression module and the decompression module. Each input vector and its corresponding output vector may represent digitized time-domain signals (e.g., speech, audio or video signals) over a predetermined time period. The loss function may be evaluated for each of a sequence of predetermined time periods.