Non-temporal stores, and why this blog is named after them

abstract
Ordinary stores pull every line they touch into the cache. When the data will not be read again soon, that is wasted bandwidth and wasted cache. This note covers what non-temporal stores do instead, when they pay off, and where the name of this site comes from.

1Introduction

A normal store goes through the cache. The core fetches the line first (a read-for-ownership), modifies it, and writes it back whenever it gets evicted. For data you are about to read again, that is exactly what you want.

x86 also has non-temporal stores1, for data you are not about to read again. Roughly, there are three ways a write can reach memory:

  • A write-back store lands in the cache and reaches DRAM only on eviction.
  • A non-temporal store skips the cache and is merged in a write-combining buffer.
  • A streaming copy pairs ordinary loads with non-temporal stores, so the destination never enters the cache.
x86 asm
; copy 64 bytes without polluting the cache
vmovdqa   ymm0, [rsi]
vmovdqa   ymm1, [rsi+32]
vmovntdq  [rdi], ymm0
vmovntdq  [rdi+32], ymm1
sfence                 ; NT stores are weakly ordered
Listing 1. A cache-bypassing copy of one 64-byte line.2

2When it pays off

Large buffers. They help when the buffer is much larger than the last-level cache and will not be reread soon: zeroing or copying big regions, streaming output, filling frame buffers. That is why many memcpy implementations only switch to them above a size threshold.

The catch. They hurt when the data is reused shortly after, because the next read now goes all the way to DRAM.

3The name

The posts here are written once and read later, sometimes much later. Data with no temporal locality.

1On x86, the MOVNT* family: MOVNTI, MOVNTDQ, MOVNTPS and their AVX forms.
2Other cores can see non-temporal stores out of order relative to ordinary ones, hence the SFENCE before publishing the data.