A system for automatically characterizing areas of interest (e.g., urban areas, forests, and/or other compound structures) in high resolution overhead imagery through manipulation of a dictionary of visual words. The pixels of an input overhead image are initially clustered into a plurality of hierarchically-arranged connected components of a first hierarchical data structure. Image descriptors (e.g., shape, spectral, etc.) of the connected components are then clustered into a plurality of hierarchically-arranged nodes of a second hierarchical data structure. The nodes at a particular level in the second hierarchical data structure become a dictionary of visual words. Subsets of the visual words may be used to label the cells of a grid over the geographic area as falling into one of a number of areas of interest. Categorization information from the grid may be mapped into a resultant image whereby pixels depict their respective type of area of interest.