A physical exercise assisting device includes a physical activity amount recorder recording a performed physical activity amount such that a physical exercise amount and a lifestyle activity amount can be recognized for a physical activity that has an intensity greater than or equal to a reference intensity; goal storage unit storing goals for the physical activity amount and the physical exercise amount to be performed in a predetermined unit period; and a display displaying goal achievement of the physical activity amount and the physical exercise amount based on the physical activity amount and the physical exercise amount recorded by the physical activity amount recorder and the goals stored in the goal storage unit.