)
Let’s imagine this situation: we want to verify programmatically how communication with the CPU works. We use a cache. “Cold” means empty, not speeding anything up. It is empty or contains irrelevant (or even undefined) data. “Warm” means it contains values and can speed up computations by quickly fetching the data being processed at that moment.
A cache is a small, fast memory structure that contains some values to speed up lookups: index links, memory blocks or pages, and so on. It helps spare the CPU from accessing its own (addressable) memory. A cache stores values in fast-lookup references (tables and trees are used) or for media access (RAM or SSD is used instead of HDD). A cache is needed to perform lookups quickly.
For example, consider a Linux file system. To access a file in the system, you need to identify where it begins on the disk using an index system, laid out as a table somewhere on the disk (the “superblock”).
The hotter, the better
If you need to read a file’s contents, you first have to read the index table on the HDD (10 ms). Then you analyze it, find the file’s first block, and read the file (another 10 ms). That’s about 20 ms in total. A lot!
If we add a cache (a hash table in RAM, 10 ns), everything will happen 3 orders of magnitude faster.
Each row in the hash table contains two values. At first, the cache is empty and cold. We need to fill it with values. If the index (file) is not found in the cache, the following loop is executed:
• read the index table;
• analyze it;
• read the file;
• save the index number;
• work with the disk’s first block in the cache.
The iterations continue: try to read another file, check the cache, get a “miss” (a “cold” cache), read from the HDD, add a row to the cache table.
A cold cache can’t speed anything up; you’re still reading everything from the HDD as before. Sometimes working with the system even gets slower, thanks to the extra time spent searching the table (warming up the cache).
Some values start appearing in the cache. You try to read a file, take a look—and there it is, in the cache (index found)! There’s a “starting” HDD block. You can skip reading the superblock and start reading the file you need. Your cache has “warmed up,” and you’ve cut the time by 10 ms. The cache contains the relevant data; all reads for the program go through the cache.
The hotter the cache, the faster the processing. The hottest state corresponds to real-time streaming processing. A cache like this needs regular maintenance (using update, removal, etc.); otherwise, it will grow in size (accumulating “garbage”) and lose performance.
The interface stores data in its “own” (local) cache the first time it retrieves it, then retrieves it from there.