A system for generating object state distributions receives an image set and generates a set of object type classifications. The set of object type classifications includes an object type classification for one or more objects represented in the image set. The set of object type classifications is generated by utilizing the image set as input to one or more object detection modules. The system generates an object state vector for each object type classification of the set of object type classifications. The object state vector(s) is/are generated by utilizing (i) the set of object type classifications and (ii) a set of image information based on the image set as input to one or more object state description modules. The system defines an object state distribution based on collections of the object state vectors.