A horizontal piston compressor is disclosed, including a frame with a cylinder, and a piston reciprocably received in the cylinder. The has piston h an inner chamber and first and second end walls. The piston and the cylinder form a compression chamber for compressing the gas. A valve and an orifice are disposed in the first end wall, and are configured to supply gas from the compression chamber to the inner chamber during a compression stroke of the piston. A gas bearing supports the piston relative to the frame. The gas bearing includes an opening for supplying gas from the inner chamber to a space between the piston and the cylinder such that the gas supplied to the space exerts an upward pressure on the piston. The valve may be a spring-loaded valve, and the orifice may be an orifice insert positioned between the valve and the compression chamber.