A system for modeling a three-dimensional structure utilizing two-dimensional segments comprising a memory and a processor in communication with the memory. The processor extracts a plurality of two-dimensional segments corresponding to the three-dimensional structure from a plurality of images indicative of different views of the three-dimensional structure. The processor determines a plurality of three-dimensional candidate segments based on the extracted plurality of two-dimensional segments and adds the plurality of three-dimensional candidate segments to a three-dimensional segment cloud. The processor transforms the three-dimensional segment cloud into a wireframe indicative of the three-dimensional structure by performing a wireframe extraction process on the three-dimensional segment cloud.