The present technology relates to a memory system and a method of operating the same. The memory system includes a memory device including a plurality of semiconductor memories, and a controller configured to control the memory device to select a victim block based on a fail bit number of some data, among data that is programmed in each of the plurality of semiconductor memories, corresponding to a specific program state, and configured to perform a garbage collection operation on the selected victim blocks.