A system and method for modeling a region segmented image is described. Aspects of the present invention may include the generation of a computer model that models the region segmented image, the computer model comprising one or more nodes, wherein each node in the one or more nodes represents an arbitrarily shaped region present in the region segmented image, and each of the arbitrarily shaped regions comprises an image segment wherein the image segment is an indivisible partition in the region segmented image. The model may additionally comprise one or more logical nodes, wherein each logical node represents an image region formed by the union of two or more arbitrarily shaped image regions in the region segmented image that exhibit at least one type of spatial relationship and a hierarchical graph representation of the region segmented image. Types of spatial relationships may include hierarchical, adjacent and cohesive spatial relationships.