Employing a relatively simple machine learning classifier to explain evidence that led to a security action decision by a relatively complex machine learning classifier. In one embodiment, a method may include identifying training data, training a relatively complex machine learning classifier (MLC) using the training data, making a first security action decision using the relatively complex MLC, performing a security action on a network device based on the first security action decision, training a relatively simple MLC using the training data or using a subset of the training data, making a second security action decision using the relatively simple MLC, extracting evidence that led to the second security action decision by the relatively simple MLC, and explaining the first security action decision by the relatively complex MLC using the evidence extracted from the relatively simple MLC.