An optical touch system, an apparatus and a method for calculating the position of an object are provided. The apparatus includes an image sensor and a processing circuit. The processing circuit selects at least one dot from a curve segment of a brightness distribution profile obtained from a background image, wherein the curve segment corresponds to an imaging range of an object. And the processing circuit regards the column pixel brightness value corresponding to the dot as a brightness reference value. Afterwards, the processing circuit calculates brightness difference between the brightness reference value and the column pixel brightness values corresponding to the dots of a curve segment of a brightness distribution profile obtained from a sensed image, so as to further calculate a gravity of the imaging range. After that, the processing circuit regards the gravity as a representative position of the object in the sensed image.