Various techniques for theft proofing autonomous driving vehicles (ADV) for transporting goods are described. In one embodiment, sensor data of a moving object representing a person within a predetermined proximity of an ADV are captured for real-time analysis by a theft detection module, to determine a moving behavior of the moving object based on the sensor data in view of a set of known moving behaviors. The theft detection module further determines whether an intention of the person is likely to remove at least some of the goods from the ADV using a process derived from historical image set, and sends an alarm to a predetermined destination in response to determining such an intention of the person. Other sensor data, for example, real time movements and weights of the ADV, can be used in conjunction with the process derived from historical image sets to determine the intention of the person.