Context data can be used to determine the current context and/or to predict the future context of a user. When the disclosed technology knows of the user's likely context, it can prepare for object recognition (e.g., image recognition, speech recognition, etc.) by (downloading and) locally storing (i.e., holding) object data related to the context. This allows for the object recognition to be performed locally and for any additional information about the object to be provided without communication over a network, thereby reducing resources such as time, cost, and processing power. If, however, the object data related to the context is not available locally, such object data can still be downloaded from a server/cloud. In some embodiments, if a likely future context is predicted and the object data related to that future context is not available locally, the object data can be downloaded from a server/cloud prior to the future context.