Determining contextual confidence of images for associative deep learning includes receiving an image including a representation of a subject. Text data related to the image is received. One or more physical properties of the image are determined. Context information of the image is determined using natural language processing. The image is classified based upon the contextual information and the one or more physical properties using a classification model to determine a classification. An emotional state of the image is determined based upon the physical properties. A confidence of the classification and emotional state is determined.