Systems and methods include periodically sending a probe to a cloud application where the probe simulates user activity in the cloud application; receiving a response to the probe and determining one or more statistics of the cloud application based on the response; incorporating the one or more statistics in historical statistics; and causing a notification when any of the one or more statistics exceeds a defined threshold. The probe can be a cloud-based probe from a cloud-based system, and the systems and methods can also include receiving a response to an on-premises probe that was sent by a distributed agent that is separate from the cloud-based system; and incorporating one or more statistics associated with the response to the on-premises probe in the historical statistics.