There is provision of a control device for outputting an operation amount of a controlled object so as to cause a process value of the controlled object to track a target value. The control device acquires a look-ahead target value within a time series of target values; calculates a look-ahead target value deviation which is a difference between the look-ahead target value and a current process value of the controlled object; calculates an adjusted target value deviation, by calculating a difference between the look-ahead target value and a predicted value of the process value after a look-ahead time length, based on a response model of the controlled object and past change amounts of the operation amount; performs reinforcement learning based on the adjusted target value deviation; and calculates an updated operation amount based on the adjusted target value deviation.