A sheet conveying device that is capable of performing a stable sheet conveying operation without being influenced by a characteristic of a sheet. A sheet conveying device includes lots of conveying roller pairs configured to convey a sheet by nipping the same and conveying motors configured to drive the respective conveying roller pairs. A downstream conveying roller pair and an upstream conveying roller pair cooperate with each other to convey the sheet by nipping the same. A CPU controls the conveying motors such that the circumferential speed of the downstream conveying roller pair becomes lower than that of the upstream conveying roller pair. The speed difference in circumferential speed between the downstream conveying roller pair and the upstream conveying roller pair is changed according to a characteristic of the sheet.