A mechanism is provided in a data processing system comprising a processor and a memory, the memory comprising instructions that are executed by the processor to specifically configure the processor to implement a medical imaging story board creation engine. The medical imaging story board creation engine executing in the data processing system receives a patient data structure comprising a medical imaging study comprising a plurality of electronic medical images. The medical imaging story board creation engine analyzes the patient data structure to determine a modality of the medical imaging study. The medical imaging story board creation engine determines, based on the determined modality of the medical imaging study, for each electronic image in the medical imaging study, at least one of an image mode or viewpoint. The medical imaging story board creation engine performs a saliency feature extraction operation on the electronic medical images in the medical imaging study based on the image mode or viewpoint of each of the electronic medical images. The medical imaging story board creation engine selects a subset of electronic medical images based on results of the saliency feature extraction. The medical imaging story board creation engine generates and outputs a collection of selected electronic medical images based on the selection of the subset of electronic medical images to form a medical imaging story board.