A global multi-label generating system is operable to receive a plurality of medical scans and a corresponding plurality of global labels that each correspond to one of a set of abnormality classes. A computer vision model is generated by training on the medical scans and the global labels. Probability matrix data, which includes a set of image patch probability values that each indicate a probability that a corresponding one of the set of abnormality classes is present in each of a set of image patches, is generated by performing an inference function that utilizes the computer vision model on a new medical scan. Global probability data that includes a set of global probability values each indicating a probability that a corresponding one of the set of abnormality classes is present in the new medical scan is generated based on the probability matrix data for transmission to a client device.