| 10 | | The main goal being the protocol scalability, the L1 caches implement a WRITE-THROUGH policy. The coherence protocol is much simpler than the MESI protocol used in most architectures implementing a WRITE_BACK policy. With a WRITE-THROUGH policy, the main memory contains always the most recent value of a cache line, and there is NO exclusive ownership state for a L1 cache. |
| | 12 | The main goal being the protocol scalability, the L1 caches implement a WRITE-THROUGH policy. The coherence protocol |
| | 13 | is much simpler than the MESI protocol used in most architectures implementing a WRITE_BACK policy. |
| | 14 | With a WRITE-THROUGH policy, the main memory contains always the most recent value of a cache line, |
| | 15 | and there is NO exclusive ownership state for a L1 cache. |
| 14 | | In the TSAR architecture, the memory controller is distributed, as it is implemented by the distributed memory caches (one per cluster). Therefore, the global directory itself is distributed. The memory cache being inclusive: a cache line L that is present in at least one L1 cache must be present in the corresponding memory cache cache (in the home cluster). With this property, the Global Directory can be implemented as an extension of the memory cache directory. |
| | 21 | In the TSAR architecture, the memory controller is distributed, as it is implemented by the distributed memory caches |
| | 22 | (one per cluster). Therefore, the global directory itself is distributed. The memory cache being inclusive: |
| | 23 | a cache line L that is present in at least one L1 cache must be present in the corresponding memory cache cache |
| | 24 | (in the home cluster). With this property, the Global Directory can be implemented as an extension of the memory cache directory. |
| 16 | | In case of MISS, the memory cache controller must evict a victim line to bring in the missing line. In order to maintain the inclusive property, all copies of the evicted cache line in L1 caches must be invalidated. To do it, the memory cache controller must send INVALIDATE requests to all L1 caches containing a copy. |
| | 26 | In case of MISS, the memory cache controller must evict a victim line to bring in the missing line. In order to maintain the inclusive property, |
| | 27 | all copies of the evicted cache line in L1 caches must be invalidated. To do it, the memory cache controller must send |
| | 28 | INVALIDATE requests to all L1 caches containing a copy. |
| 18 | | The TSAR architecture wants to guaranty the cache coherence by hardware, for both the data and instruction L1 caches. Reflecting the different behaviour of data & instruction caches, the "hybrid" cache coherence protocol defines two different strategies, depending on the number of copies : |
| 19 | | * '''MULTICAST_UPDATE''' : the modifications of shared data are very frequent events, but the number of copies is generally not very high. When the number of copies is smaller than the DHCCP threshold, the memory cache controller registers the locations of all the copies, and send a ''multicast_update'' transaction to each concerned L1 cache in case of modification. |
| 20 | | * '''BROADCAST_INVAL''' : the modifications of shared code rare events ( self modifying code, or dynamic libraries ), but the number of replicated copies can be very large ( the exception handler, or the libc are generally replicated in all L1 caches ). When the number of copies is larger than the DHCCP threshold, the memory cache controller will simply store the number of copies (without localization) and send a ''broadcast_inval'' transaction to all L1 caches in case of modication. |
| | 30 | The TSAR architecture wants to guaranty the cache coherence by hardware, for both the data and instruction L1 caches. |
| | 31 | The modifications of shared data are very frequent events, but the number of copies is generally not very high. |
| | 32 | The modifications of shared code are very rare events ( self modifying code, or dynamic libraries ), but the number |
| | 33 | of replicated copies can be very large ( the exception handler, or the libc are generally replicated in all L1 caches ). |
| | 34 | Reflecting the different behaviour of data & instruction caches, the "hybrid" cache coherence protocol DHCCP defines two different strategies, |
| | 35 | depending on the number of copies : |
| | 36 | * '''MULTICAST_UPDATE''' : When the number of copies is smaller than the DHCCP threshold, the memory cache controller registers the locations |
| | 37 | of all the copies, and send a ''multicast_update'' transaction to each concerned L1 cache in case of modification. |
| | 38 | * '''BROADCAST_INVALIDATE''' : When the number of copies is larger than the DHCCP threshold, the memory cache controller registers only the number |
| | 39 | of copies (without localization) and send a ''broadcast_invalidate'' transaction to all L1 caches in case of modication. |
| 31 | | As a general rule, all these transactions respect the VCI advanced packet format, and there is one response packet for each command packet : For a burst transaction, a READ command packet contains one single flit, and the corresponding READ response packet contains N flits. Symmetrically, a WRITE command packet contains N flits, and the corresponding WRITE response packet contains one single flit. |
| | 50 | As a general rule, all these transactions respect the VCI advanced packet format, and there is one response packet for each command packet : |
| | 51 | For a burst transaction, a READ command packet contains one single flit, and the corresponding READ response packet contains N flits. |
| | 52 | Symmetrically, a WRITE command packet contains N flits, and the corresponding WRITE response packet contains one single flit. |
| 37 | | These transactions are initiated by a processor (actually the L1 cache controller), or by another initiator ( an I/O peripheral or hardware coprocessor with a DMA capability). This initiator can be located in any cluster. For those transactions, the target is a memory cache controller, acting as a physical memory bank, or another VCI target peripheral. This target can be located in any cluster. |
| | 59 | These transactions are initiated by a processor (actually the L1 cache controller), or by another initiator |
| | 60 | (an I/O peripheral or hardware coprocessor with a DMA capability). This initiator can be located in any cluster. For those transactions, |
| | 61 | the target is a memory cache controller, acting as a physical memory bank, or another VCI target peripheral. This target can be located in any cluster. |
| 41 | | * A '''WRITE''' transaction can be a single word request or a variable length burst request. In case of burst, the VCI command packet contains at most 8 VCI flits, with consecutive addresses. All words belong to the same half cache line, and the VCI BE field can have different values for each flit (including the zero value). The VCI response packet contains one VCI flit. A WRITE burst transaction initiated by any DMA controller must respect the same 8 aligned words constraint. The VCI CMD field contains the VCI_WRITE code. When the VCI TRDID field contains a non-zero value, it signals that the write request is “posted” : The VCI target must send a response to respect the VCI protocol, but this response can be send before the write is actually performed. This can be used by by the VCI/HT bridge. The VCI PKTID fields is not used. If the modified cache line is replicated in one or several other L1 caches, all copies must be updated or invalidated before the WRITE transaction is acknowledged. |
| | 65 | * A '''READ''' transaction can be a single word request (in case of uncached access), or a burst, corresponding to a complete cache line (16 words). |
| | 66 | A READ burst transaction initiated by any DMA controller must respect the same 16 words cache line format. |
| | 67 | For all READ transaction, the VCI command packet contains one single VCI flit. The VCI CMD field contains the VCI_READ code. |
| | 68 | The VCI PLEN field is used to define the burst length. A READ transaction has a type, encoded in the VCI TRDID field : |
| | 69 | The MSB bit of the TRDID field has the value 1. Bit 0 of the TRID field is 0 for an uncached access, and 1 for a cached access. |
| | 70 | Bit 1 of the TRDID field is 0 for a data cache request, and 1 for an instruction cache request. |
| | 71 | The response packet contains one VCI flit (single word) or 16 VCI flits (cache line). |
| | 72 | The VCI PKTID field is not used. |
| | 73 | |
| | 74 | * A '''WRITE''' transaction can be a single word request or a variable length burst request. In case of burst, the VCI command packet contains |
| | 75 | at most 8 VCI flits, with consecutive addresses. All words belong to the same half cache line, and the VCI BE field can have different values |
| | 76 | for each flit (including the zero value). The VCI response packet contains one VCI flit. |
| | 77 | A WRITE burst transaction initiated by any DMA controller must respect the same 8 aligned words constraint. |
| | 78 | The VCI CMD field contains the VCI_WRITE code. |
| | 79 | The MSB bit of the TRDID field has the value 0. The LSB bits of the TRDID field define the index of the write transaction. |
| | 80 | When these LSB bits have a non zero value, the write request is “posted” : The VCI target must send a response |
| | 81 | to respect the VCI protocol, but this response can be send before the write is actually performed. This can be used by by the VCI/HT bridge. |
| | 82 | The VCI PKTID fields is not used. |
| | 83 | If the modified cache line is replicated in one or several other L1 caches, the memory cache must guaranty that all copies have been |
| | 84 | updated or invalidated before the WRITE transaction is acknowledged. |
| 47 | | These transactions implement the DHCCP protocol : For each cache line stored in the memory cache, the memory cache implement a Registration Table that contain the copies replicated in the L1 caches. Each entry in this Registration Table contains the SRCID of the L1 cache that contains a copy, as well as the type of the copy (instruction/data). When the same cache line is replicated in both the instruction cache and the data cache of a processor, this defines two separated entries in the Registration Table. When the number copies for a given cache line L exceeds the DHCCP threshold, the corresponding Registration Table is flushed, and the memory cache registers only the number of copies. |
| | 90 | For each cache line stored in the memory cache, the memory cache implement a Registration Table that contain the copies replicated |
| | 91 | in the L1 caches. Each entry in this Registration Table contains the SRCID of the L1 cache that contains a copy, as well as the type |
| | 92 | of the copy (instruction/data). When the same cache line is replicated in both the instruction cache and the data cache of a processor, |
| | 93 | this defines two separated entries in the Registration Table. When the number copies for a given cache line L exceeds the DHCCP threshold, |
| | 94 | the corresponding Registration Table is flushed, and the memory cache registers only the number of copies. |
| 51 | | * A '''MULTICAST_UPDATE''' transaction is a multi-cast transaction sent by the memory cache controller when it receives a WRITE request to a replicated cache line and the number of copies does not exceeds the DHCCP threshold. It sends as many VCI transactions as the number of registered copies (but the writer). The VCI command packet contains (N+2) flits. The VCI ADDRESS field is constant and contains the address of the memory mapped UPDATE register in the L1 cache. The VCI CMD field contains the WRITE value. As the memory cache controller can handle several simultaneous update/invalidate transactions, the VCI TRDID field contains the transaction index. The VCI PLEN field contains the value 4*N, where N is the actual number of modified words in the cache line. The line index (34 bits) is transported in the VCI WDATA and VCI BE fields (the two LSB bits), of the first flit. The first modified word index (3 bits) is transported in the WDATA field of the second flit, and the N modified words in the WDATA and BE fields of the N following flits. For each modified word, the VCI BE field can have a different value (including the 0x0 value). The VCI response packet contains one single flit. The memory cache controller counts the number of VCI responses to detect the completion of the MULTI_UPDATE transaction. |
| | 99 | * A '''MULTICAST_UPDATE''' transaction is a multi-cast transaction sent by the memory cache controller when it receives a WRITE request |
| | 100 | to a replicated cache line and the number of copies does not exceeds the DHCCP threshold. It sends as many VCI transactions as the number |
| | 101 | of registered copies (but the writer). The VCI command packet contains (N+2) flits. The VCI ADDRESS field is constant and contains the address |
| | 102 | of the memory mapped UPDATE register in the L1 cache. The VCI CMD field contains the VCI_WRITE code. As the memory cache controller can |
| | 103 | handle several simultaneous update/invalidate transactions, the VCI TRDID field contains the transaction index. The VCI PLEN field contains the value 4*N, |
| | 104 | where N is the actual number of modified words in the cache line. The line index (34 bits) is transported in the VCI WDATA and VCI BE fields (the two LSB bits), |
| | 105 | of the first flit. The first modified word index (3 bits) is transported in the WDATA field of the second flit, and the N modified words in the WDATA and BE fields |
| | 106 | of the N following flits. For each modified word, the VCI BE field can have a different value (including the 0x0 value). |
| | 107 | The VCI response packet contains one single flit. The memory cache controller counts the number of VCI responses to detect the completion |
| | 108 | of the MULTICAST_UPDATE transaction. |
| 53 | | * A '''MULTICAST_INVAL''' transaction is a multi-cast transaction, that is composed of several VCI transactions. When a memory cache makes a cache line replacement (following a MISS), and the victim line has a number of copies smaller than the DHCCP threshold, it sends as many VCI transactions as the number of registered copies. Both the VCI command packet and the VCI response packet contain only one flit. The VCI ADDRESS field contains the address of the memory mapped INVAL register in the L1 cache. The VCI CMD field contains the WRITE value. As the memory cache controller can handle several update/invalidate transactions simultaneously, the VCI TRDID field contains the transaction index.The VCI WDATA and VCI BE (the two LSB bits) fields contain the 34 bits line index. The memory cache controller counts the number of VCI responses to detect the completion of the MULTI_INVAL transaction. |
| | 110 | * A '''MULTICAST_INVALIDATE''' transaction is a multi-cast transaction, that is composed of several VCI transactions. When a memory cache makes a cache line |
| | 111 | replacement (following a MISS in the memory cache), and the victim line has a number of copies smaller than the DHCCP threshold, it sends as many VCI transactions |
| | 112 | as the number of registered copies. Both the VCI command packet and the VCI response packet contain only one flit. The VCI ADDRESS field contains the address |
| | 113 | of the memory mapped INVAL register in the L1 cache. The VCI CMD field contains the VCI_WRITE code. As the memory cache controller can handle several |
| | 114 | update/invalidate transactions simultaneously, the VCI TRDID field contains the transaction index.The VCI WDATA and VCI BE (the two LSB bits) fields contain the |
| | 115 | 34 bits line index. The memory cache controller counts the number of VCI responses to detect the completion of the MULTI_INVAL transaction. |
| 55 | | * A '''BROADCAST_INVAL''' transaction is a broadcast transaction. This transaction is initiated when a memory cache controller replaces a line, or receives a WRITE request to a replicated cache line, that has a number of copies larger than the DHCCP threshold. The VCI command packet contains one single flit. This packet is replicated and dynamically broadcasted by the network itself. The VCI CMD field contains the WRITE value. The VCI ADDRESS field contains the global broadcast address 0x0000000003 (only the two LSB bits are set). The VCI WDATA and the VCI BE (the two LSB bits) field contain the line index. This VCI command is broadcasted to all L1 caches in the system, but only L1 caches that have a copy send a VCI response packet. All VCI response packets are independently returned to the memory cache initiator, that counts the number of VCI responses to detect the completion of the BROADCAST_INVAL transaction. If a L1 cache contains two copies of a cache line (i.e. the line is replicated in both the DATA cache, and the INSTRUCTION cache), it must send two VCI responses. |
| | 117 | * A '''BROADCAST_INVAL''' transaction is a broadcast transaction. This transaction is initiated when a memory cache controller replaces a line, |
| | 118 | or receives a WRITE request to a replicated cache line, and this cache line has a number of copies larger than the DHCCP threshold. |
| | 119 | The VCI command packet contains one single flit. This packet is replicated and dynamically broadcasted by the network itself. |
| | 120 | The VCI CMD field contains the VCI_WRITE code. The VCI ADDRESS field contains the global broadcast address 0x0000000003 |
| | 121 | (only the two LSB bits are set). The VCI WDATA and the VCI BE (the two LSB bits) field contain the line index. |
| | 122 | This VCI command is broadcasted to all L1 caches in the system, but only L1 caches that have a copy send a VCI response packet. |
| | 123 | All VCI response packets are independently returned to the memory cache initiator, that counts the number of VCI responses |
| | 124 | to detect the completion of the BROADCAST_INVAL transaction. |
| | 125 | If a L1 cache contains two copies of a cache line (i.e. the line is replicated in both the DATA cache, and the INSTRUCTION cache), it must send two VCI responses. |
| 57 | | * A '''CLEANUP''' transaction is initiated by a L1 cache controller to a memory cache controller, to signal that a cache line copy has been removed from an instruction or data cache. Both the VCI command packet and the VCI response packet contain one single flit. For a CLEANUP transaction, the VCI ADDRESS field must contain the removed cache line address, and the VCI TRDID field must contain a non zero value. |
| | 127 | * A '''CLEANUP''' transaction is initiated by a L1 cache controller to a memory cache controller, to signal that a cache line copy |
| | 128 | has been removed from an instruction or data cache. Both the VCI command packet and the VCI response packet contain one single flit. |
| | 129 | For a CLEANUP transaction, the VCI ADDRESS field must contain the removed cache line address, and the VCI TRDID field must contain a non zero value. |
| | 130 | |
| | 131 | |
| 63 | | All the external transactions use a separated ''external network'', implementing a separated address space. The memory cache and external RAM controller ports used to access the external network respect a simplified version of the VCI advanced format : the VCI fields PLEN, PKTID, CONST, CONTIG and BE are not used. The VCI ADDRESS field contains 30 bits (a 64 bytes cache line index). (30 bits). The VCI WDATA & RDATA fields contain 64 bits, in order to improve the bandwidth. The VCI SRCID field contains the memory cache index (cluster index). As the memory cache controller can process several external transactions simultaneously, the VCI TRDID field contains the transaction index. |
| | 139 | All the external transactions use a separated ''external network'', implementing a separated address space. The memory cache and |
| | 140 | the external RAM controller ports used to access the external network respect a simplified version of the VCI advanced format : |
| | 141 | the VCI fields PLEN, PKTID, CONST, CONTIG and BE are not used. The VCI ADDRESS field contains 30 bits (a 64 bytes cache line index). |
| | 142 | The VCI WDATA & RDATA fields contain 64 bits, in order to improve the bandwidth. The VCI SRCID field contains the memory cache index (cluster index). |
| | 143 | As the memory cache controller can process several external transactions simultaneously, the VCI TRDID field contains the transaction index. |
