Provided herein may be a memory system and a method of operating the memory system. The memory system may include: a semiconductor memory device including a plurality of memory cells to be programmed to an erase state and a plurality of program state; and a controller configured to control the semiconductor memory device to perform a program operation or a read operation in response to a request of a host. The controller may control the semiconductor memory device such that when, after a first program operation of the program operation has been performed, a number of program fail bits of the plurality of memory cells is greater than a maximum allowed number of ECC bits, a second program operation is performed on selected memory cells of the plurality of memory cells.