Data is accessed in a multi-level hierarchical memory system. A request for data is received, including a virtual address for accessing the data. A translation buffer is queried to obtain an absolute address corresponding to the virtual address. Responsive to the translation buffer not containing an absolute address corresponding to the virtual address, the absolute address is obtained from a translation unit. A directory look-up is performed with the absolute address to determine whether a matching absolute address exists in the directory. A fetch request for the requested data is sent to a next level in the multi-level hierarchical memory system. Processing of the fetch request by the next level occurs in parallel with the directory lookup. The requested data is received in the primary cache to make the requested data available to be written to the primary cache.