Big data analysis methods and machine learning based models are used to provide offer recommendations to consumers that are probabilistically determined to be relevant to a given consumer. Machine learning based matching of user attributes and offer attributes is first performed to identify potentially relevant offers for a given consumer. A de-duplication process is then used to identify and eliminate any offers represented in the offer data that the consumer has already seen, has historically shown no interest in, has already accepted, that are directed to product or service types the user/consumer already owns, for which the user does not qualify, or that are otherwise deemed to be irrelevant to the consumer.