Aspects of the present invention are related to systems, methods and apparatus for automatic quality assessment of a video sequence. According to a first aspect of the present invention, a quality index may be generated by combining a spatial quality index and a temporal quality index. According to a second aspect of the present invention, a spatial quality index may be calculated using a modified exponential moving average model to pool multi-scale structural similarity indices computed from test frame—reference frame pairs. According to a third aspect of the present invention, a temporal quality index may be generated by averaging multi-scale structural similarity indices computed from difference image pairs, wherein one difference image is formed between reference frames and another difference image is formed between a reference frame and a test frame.