A transmitter of a communications system includes a first encoder configured to apply a shaping operation to a data signal to generate a shaped data signal, a second encoder configured to encode the shaped data signal according to a forward error correction (FEC) scheme to generate an encoded signal, and a constellation mapper configured to modulate the encoded signal to symbol values according to a modulation scheme to generate a corresponding symbol stream for transmission through the communications system. The shaping operation reduces average constellation energy for constellations of the modulation scheme.