A system and method for medical image based patient-specific ischemic stroke risk prediction is disclosed. Left atrium (LA) and left atrium appendage (LAA) measurements are extracted from medical image data of a patient. Derived metrics for the LA and LAA of the patient are computed using a patient-specific computational model of cardiac function based on the LA and LAA measurements extracted from the medical image data of the patient. A stroke risk score for the patient is calculated based on the extracted LA and LAA measurements and the computed derived metrics for the LA and LAA of the patient using a trained machine learning based classifier, which inputs the extracted LA and LAA measurements and the computed derived metrics for the LA and LAA as features.