A quick removal socket tool includes: a rod body that has a surrounding wall defining a recess for receiving a screwdriver head and formed with a retaining hole in spatial communication with the recess; and a sliding sleeve sleeved movably on the rod body, formed with an annular groove, and being movable relative to the rod body between a retaining position, where the annular groove is misaligned from the retaining hole so that the positioning ball is disposed within the retaining hole and a groove of the screwdriver head to retain the screwdriver head within the recess, and a release position, where the annular groove is aligned with the retaining hole so that the positioning ball is disposed within the retaining hole and the annular groove to allow for removal of the screwdriver head from the recess.