An image sensor may include a pixel array having visible and infrared imaging pixels for simultaneously detecting light in the visible and infrared spectral ranges. The pixel array may include an array of photodiodes, an array of filter elements formed over the photodiodes, and an array of microlenses formed over the array of filter elements. The filter elements may include infrared cutoff filter elements that block infrared light while passing visible light. The infrared cutoff filter elements may be formed from a patterned layer of infrared blocking material. Each visible imaging pixel includes a portion of the infrared blocking material and a color filter element. Each infrared imaging pixel is aligned with an opening in the infrared blocking material. The opening may be filled with an infrared pass filter element that passes infrared light while blocking visible light.