Techniques are disclosed for improving classification performance in supervised learning. In accordance with some embodiments, a multiclass support vector machine (SVM) having three or more classes may be converted to a plurality of binary problems that then may be reduced via one or more reduced-set methods. The resultant reduced-set (RS) vectors may be combined together in one or more joint lists, along with the original support vectors (SVs) of the different binary classes. Each binary problem may be re-trained using the joint list(s) by applying a reduction factor (RF) parameter to reduce the total quantity of RS vectors. In re-training, different kernel methods can be combined, in accordance with some embodiments. Reduction may be performed until desired classification performance is achieved. The disclosed techniques can be used, for example, to improve classification speed, accuracy, class prioritization, or a combination thereof, in the SVM training phase, in accordance with some embodiments.