An electronic device such as a portable music player, cellular telephone, or computer may be provided with an audio jack system that allows audio plug position measurements to be made. Sensors such as optical sensors, magnetic sensors, mechanical sensors, electrical sensors, resistive sensors, and capacitive sensors may be used in monitoring the position and movement of the audio plug relative to the audio jack. A user may rotate the audio plug to control operations such as media playback operations, menu selection operations, and other activities in the electronic device. The audio jack may include flexible structures that allow the audio plug to be tilted relative to the audio jack. Capacitive sensors or other sensors may be used to monitor audio plug tilt and axial audio plug movement. This allows the audio plug to serve as a joystick for the electronic device.