We have a CPU, L1 cache, L2 cache and a memory. We will look at different examples of the cache organization. We will talk about what happens on a read miss, read hit, write miss and write hit.

Here are some general assumptions we will abide by:

A1. Caches are inclusive - whatever is in L1 must be in L2, whatever is in L2 must be in memory. If we expel a block A from L2 there are mechanisms that will find all blocks in L1 that match the expelled block A and expel them too. If we want to bring in a new block B into L1, we have to bring it through L2, and store it also in L2. So the inclusive claim holds always.

A2. When we have a write-through cache, every time we write a piece of data of size Y to this cache, this cache will go one level lower to write this same data. If we write one word of data to this cache it will write this same word one level lower. If we write one block to this cache, it will write this same block one level lower.

A3. We will assume that caches are always full. This only has significance for write-back caches, when we say that x% of blocks are dirty. If cache is always full, this means that every time we want to bring a block in we have to take one block out and there is exactly x% chance that this block we are taking out is dirty.

A4. Assume we want to bring a block A into a write-back cache (say we had a read miss on block A). We will assume that we first figure out which block to evict from this cache (say block B), and handle this eviction. Handling eviction means that if B was dirty we have to write B to a lower level, and then lower level has to handle this write. When this whole eviction process is done, then we will look for block A in a lower level (this results in a hit or a miss) and bring it into this cache. The overall effect of “first evict, then bring in new block” is that block B that we are evicting is always found in lower level.

A5. If a cache is write-allocate, and we want to write a piece of data of size Y to this cache, and this piece of data is in a block A that is not in this cache (i.e., we have a write miss), we will read block A as it is from a lower level and place it into this cache, then write a piece of data of size Y to this cache according to this cache's write policy (write-through or write-back).

A6. If a cache is no-write-allocate, and we want to write a piece of data of size Y to this cache, and this piece of data is in a block A that is not in this cache (i.e., we have a write miss), we will write a piece of data of size Y one level lower, bypassing this cache.

Here are different design choices we can vary for L1 and L2 each:

write-through vs. write-back write-allocate vs. no-write-allocate unified cache vs. split cache

Additionally, we may have a write buffer or not,

write buffer is placed between L1 and L2 vs. between L2 and memory vs. there is no write buffer

There are total of 192 combinations possible. We will first look at variations of a), b) and c) combinations for L1 and L2 and then we will add d) variations.

1. A cache is unified vs. we have split cache for instructions and data

The difference between these two choices is only that in the case of a unified cache we will calculate one AMAT_read and this will be how much we pay for reading anything, instruction or data. In case of split caches, we will calculate AMAT_read_for_instructions using specifications for instruction cache and AMAT_read_for_data using specifications for data cache. So only numbers we use (for block size, miss rate, hit time, etc.) will change but the formulas will stay the same. AMAT_write will be calculated for data only, regardless if the cache is split or unified. Just to note here, we may have both L1 and L2 unified, or L1 split/L2 unified, or L1 unified/L2 split (very unusual) or both L1 and L2 split. So let’s look at these combinations before we go further:

1-1: L1 and L2 are both unified:

We calculate AMAT_read using L1 and L2 specifications. This is how much we pay for reading data or instructions. We calculate AMAT_write using L1 and L2 specifications. This is how much we pay for writing data.

1-2: L1 split into L1_inst and L1_data, L2 unified:

We calculate AMAT_read_for_instructions using L1_inst and L2 specifications.

We calculate AMAT_read_for_data using L1_data and L2 specifications. We calculate AMAT_write using L1_data and L2 specifications.

1-3: L1 unified, L2 split into L2_inst and L2_data:

We calculate AMAT_read_for_instructions using L1 and L2_inst specifications.