A method, computer program product, and system includes a processor(s) obtaining, during runtime, from a compiler, two versions of a data parallel loop for an operation. The host computing system comprises includes a CPU and a GPU is accessible to the host. The processor(s) online profiles the two versions by asynchronously executing the first version, in a profile mode, with the GPU and executing the second version, in the profile mode, with the CPU. The processor(s) generates execution times for the first version and the second version. The processor(s) stores the executions times and performance data in a storage, where the performance data comprises a size of the data parallel loop for the operation. The processor(s) update a regression model(s) to predict performance numbers for a process of an unknown loop size. The processor(s) execute the operation with the CPU or the GPU based on the performance data.