A wearable apparatus includes a user interface, a motion sensor, a microprocessor and a central processing unit (CPU). In an operation mode, the motion sensor senses a current hand movement trajectory (HMT). The microprocessor generates a velocity curve along a coordinate axis according to the current HMT, and samples the velocity curve according to a first predetermined velocity and a second predetermined velocity to output velocity sampling points. The microprocessor further determines whether a matching number between the velocity sampling points and velocity feature points is greater than a threshold. The current HMT matches a predetermined HMT when the matching number is greater than the threshold. The CPU performs a system operation corresponding to the default HMT when the current HMT matches the predetermined HMT.