A determination is made that a memory device of a memory sub-system is to be transitioned to a sleep mode. A command is initiated to cause a standby circuit associated with the memory device to enter into a low power mode while a power supply of the memory sub-system is maintained in a powered state. In the low power mode, a reference voltage is supplied to a voltage regulator of the standby circuit to supply a standby current level to the memory device during the sleep mode.