One embodiment described herein provides a system and method for secure point-to-point communication. During operation, the system establishes a voice communication channel between a local client device and a remote client device and obtains an encryption key negotiated between the local client device and the remote client device. The system can then obtain a voice signal generated by a user associated with the local client device based on the encryption key and performs a key-validation operation by sending the voice signal from the local client device to the remote client device using the voice communication channel. In response to a successful validation of the encryption key, the system establishes a secure point-to-point communication channel between the local and remote client devices using the validated encryption key.