Novel tools and techniques are described for auto-summarizing video and/or audio content. In some embodiments, a summary server might retrieve one or more time codes from a time code database, and might analyze the one or more time codes to determine at least one selected time code among the one or more time codes that exceeds a predetermined number of selections. The one or more time codes might correspond to one or more triggers, which might include user-inputted triggers, triggers associated with actions by a live audience, or broadcaster/distributor/producer-provided triggers (i.e., cue tones). The summary server might determine one or more selected segments of video or audio content corresponding to the at least one selected time code, and might create one or more summary tracks, where each of the one or more summary tracks might comprise each of the one or more selected segments of the video or audio content.