Provided are a computer program product, system, and method for determining memory access categories to use to assign tasks to processor cores to execute. A computer system has a plurality of cores, each core is comprised of a plurality of processing units and at least one cache memory shared by the processing units on the core to cache data from a memory. At task is processed to determine one of the cores on which to dispatch the task. A memory access category of a plurality of memory access categories is determined to which the processed task is assigned. The processed task is dispatched to the core assigned the determined memory access category.