A method and system that assesses video quality of transmitted video packet signals suffering from packet loss and delay. This packet loss and delay can create freeze events, which are observed as a jerkiness while viewing the video. The system and method compares the frames in a video transmission to determine freeze events; extracts a set of features from the locations of the freeze events and decoded video frames; and maps the set of features into a video quality score using a neural network. The video quality score provides an assessment of the effects of irregular frame freezes due to packet loss or delay on the quality of the video.