A method, accelerator system, and computer program product, for prefetching data from a server system in an out-of-order processing environment. A plurality of prefetch requests associated with one or more given data sets residing on the server system are received from an application on the server system. Each prefetch request is stored in a prefetch request queue. A score is assigned to each prefetch request. A set of the prefetch requests are selected from the prefetch queue that comprise a score above a given threshold. A set of data, for each prefetch request in the set of prefetch requests, is prefetched from the server system that satisfies each prefetch request, respectively.