A game system includes a home-console type game device, a controller device, and a portable display device. The portable display device includes an infrared light-emitting portion capable of emitting infrared light. The controller device wirelessly transmits to the game device operation date including data representing a detection result by an image-capturing section, data representing a detection result by an inertia sensor, and data representing an operation performed on an operation button. The game device receives the operation data from the controller device, and performs a game process based on the operation data.