There is provided a portable optical touch system including a first image sensing module, a second image sensing module, a connecting device, a communication interface and a processing circuit. The two image sensing modules have partially overlapped field of views and the partially overlapped field of views is for defining a touch region. The connecting device is configured to adjust or fix a distance between the first and second image sensing modules. When an object is in the touch region, the processing circuit calculates a position of the object according to images of the object acquired by the two image sensing modules and outputs information of the position of the object through the communication interface.