A user device can verify a user's identity to a server while protecting user privacy by not sharing personal data with any other device. To ensure user privacy, the user device performs an enrollment process in which the user performs an action sequence. The user device collects action data from the action sequence and uses the action data locally to generate a set of public/private key pairs (or other representation) from which information about the action sequence cannot be extracted. The public keys, but not the underlying action data, are sent to a server to store. To verify user identity, a user device can repeat the collection of action data and the generation of the key pairs. If the device can prove to the server its possession of the private keys to a sufficient degree, the user's identity can be verified.