Irregular codes in graph algorithms and sparse linear algebra are bottlenecked by memory and communication latency. Decoupled access/execute (DAE) is a common technique used in CPU/GPU prefetchers and specialized accelerators to tackle this problem. The technique relies on the compiler to separate memory address generation from the rest of the program, but such a separation is not always possible due to control and data dependencies between the access and execute slices, resulting in a loss of decoupling.

In this paper, we present compiler support for speculation in DAE architectures that preserves decoupling in the face of control dependencies. We propose algorithms that implement speculative memory requests in the access slice and that kill mis-speculations in the execute slice without the need for costly recovery or synchronization. Our transformations work on arbitrary, reducible control flow and are proven to preserve sequential consistency. We show that our approach applies to a wide range of architectural work on CPU/GPU prefetchers, CGRAs, and accelerators, enabling the DAE technique on a wider range of codes than before.