The present invention provides a pharmaceutical composition containing hemoglobin-based therapeutic agent for treating cancer. The hemoglobin moiety can target cancer cells and the therapeutic moiety (i.e. active agent/therapeutic drug) can kill the cancer cells efficiently. The hemoglobin-based therapeutic agent used in the present invention can be used in the treatment of various cancers such as pancreatic cancer, leukemia, head and neck cancer, colorectal cancer, lung cancer, breast cancer, liver cancer, nasopharyngeal cancer, esophageal cancer, prostate cancer, stomach cancer and brain cancer. The composition can be used alone or in combination with other therapeutic agent(s) such as chemotherapeutic agent to give a synergistic effect on cancer treatment, inhibiting metastasis and/or reducing recurrence. The presently claimed hemoglobin-based 5FU-two-dye conjugate and/or hemoglobin-based 5FU-one-dye conjugate can also be used in live-cell imaging and diagnostic imaging.