Disclosed herein are various methods of forming replacement gate structures for semiconductor devices. In one example, the method includes forming a sacrificial gate structure above a semiconducting substrate, removing the sacrificial gate structure to thereby define a gate cavity for a replacement gate structure, forming a gate insulation layer in the gate cavity and forming a layer of metal above the gate insulation layer. In this example, the method also includes forming a patterned etch mask layer above the metal layer that exposes substantially vertically oriented portions of the metal layer within the cavity and covers a substantially horizontally oriented portion of the metal layer within the cavity, performing an etching process through the patterned etch mask layer to reduce a thickness of the exposed substantially vertically oriented portions of the metal layer and removing the patterned etch mask layer.