A method for tuning translation parameters in statistical machine translation based on ranking of the translation parameters is disclosed. According to one embodiment, the method includes sampling pairs of candidate translation units from a set of candidate translation units corresponding to a source unit, each candidate translation unit corresponding to numeric values assigned to one or more features, receiving an initial weighting value for each feature, comparing the pairs of candidate translation units to produce binary results, and using the binary results to adjust the initial weighting values to produce modified weighting values.