A method of processing an audio signal is disclosed. The present invention includes receiving a downmix signal including at least one object signal, and object information extracted when the downmix signal is generated, receiving mix information including mode selection information, the mix information for controlling the object signal, and generating a stereo output signal using the downmix signal or extracting a background object and at least one independent object from the downmix signal, based on the mode selection information, wherein the downmix signal corresponds to a mono signal, wherein the stereo output signal corresponds to a time-domain signal including a signal generated from decorrelating the downmix signal, and wherein the mode selection information includes information indicating which one of modes including a normal mode, a mode for controlling the background object, and a mode for controlling the at least one independent object.