Embodiments of the invention provide a method, system and article of manufacture for parallel application load balancing and distributed work management. In one embodiment, a hierarchy of master nodes may be used to coordinate the actions of pools of worker nodes. Further, the activity of the master nodes may be controlled by a “coordinator” node. A coordinator node may be configured to distribute work unit descriptions to the collection of master nodes. If needed, embodiments of the invention may be scaled to deeper hierarchies.