A system for controlling vehicle systems includes a haptic actuator, a gesture detection device, a controller, and a plurality of vehicle systems. The gesture detection device captures movements of a vehicle driver or passenger and outputs data representing a gesture made by the vehicle driver or passenger. The controller is coupled to the gesture detection device and receives a signal from the gesture detection device indicative of the gesture. The controller is coupled to the haptic actuator, and selects a haptic effect based on the detected gesture and sends a signal to the haptic actuator to provide the selected haptic effect. The controller is also coupled to the vehicle systems, and based on the detected gesture, the controller sends a signal to one of the vehicle systems to perform a particular action. The particular action of the vehicle system is matched with the selected haptic effect.