| 13 | | ALMOS-MKH is based on the "Multi-Kernel" approach to ensure scalability, and support the distribution of system services. In this approach, each cluster of the architecture contains an instance of the kernel. Each instance controls the local resources (memory and computing cores). These multiple instances cooperate with each other to give applications the image of a single system controlling all resources. They communicate with each other using both (i) the client / server model, sending a Remote Procedure Call to a remote cluster for a complex service, (ii) the shared memory paradigm, making direct access to remote memory when required. |
| | 13 | ALMOS-MKH is based on the "Multi-Kernel" approach to ensure scalability, and support the distribution of system services. In this approach, each cluster of the architecture contains an instance of the kernel. Each instance controls the local resources (memory and computing cores). These multiple instances cooperate with each other to give applications the image of a single system controlling all resources. They communicate with each other using both (i) the client / server model, sending a remote procedure call (RPC) to a remote cluster for a complex service, (ii) the shared memory paradigm, making direct access to remote memory when required. |
| 15 | | To reduce energy consumption, ALMOS-MKH supports architectures using 32-bit cores. In this case, each cluster has a 32-bit physical address space, addressed by the LPA (Local Physical Address). To access the physical addressing space of other clusters, ALMOS-MKH defines 64-bit global physical addresses (CXY,LPA). For example, the physical space of the TSAR architecture has 40 bits, the 8 most significant bits define the target cluster identifier CXY, and the 32 LSB bits define the local physical address LPA. The ALMOS-MKH kernel thus explicitly distinguishes two types of access: |
| 16 | | * Local access (internal to a cluster) uses standard pointers. |
| 17 | | * Remote accesses (to another cluster) use extended pointers. |
| | 15 | ALMOS-MKH supports both architectures using 64-bits cores, such as the Intel based multi-core servers, or architectures using 32-bits cores, such as the TSAR-Mips32 many-core architecture. |
| 19 | | On hardware platforms containing '''32-bit cores''', such as TSAR, the ALMOS-MKH kernel runs partially in physical addressing: To reduce contention, ALMOS_MKH replicates the kernel code in all clusters, and uses - in each cluster - the Instruction MMU to map the local kernel code copy in the kernel virtual space. But the Data MMU is deactivated as soon as a core enters the kernel, and it is reactivated when it returns to user. To access the local addressable resources (memory or devices), ALMOS-MK uses standard 32 bits pointers in identity mapping. To access ressources in another cluster, ALMOS-MKH uses the ''remote_read'' and ''remote_write'' primitives implementing the 64-bit extended pointers (CXY / PTR). CXY is the target cluster identifier, and PTR is the local pointer in the target cluster. These remote access primitives are used to implement the RPC mechanism, but are also used for fast access to kernel distributed data structures, which are critical in performance. For the applications ALMOS_MKH, as any POSIX compliant OS, uses the MMU for both the instructions and the data. |
| | 17 | The main ALMOS-MKH specific feature is the following: the physical address space is supposed to be distributed between the clusters, and the MSB bits of the physical address are supposed to define the target cluster. In a physical address, the LSB bits contain the '''lpa''' (Local Physical Address), and the MSB bits define the '''cxy''' (Cluster Identifier). The physical address space is therefore described as a two levels array M[cxy][lpa]. To enforce locality, a kernel instance can use a normal pointer '''ptr''' to access the local physical memory. But, to access the physical memory of a remote cluster (cxy), it must use the specific remote access functions ''remote_read(cxy,ptr)'' and ''remote_write(cxy,ptr), where '''ptr''' is the local pointer in the target cluster. This (cxy,ptr) couple is called an ''extended pointer''. |
| 21 | | On a hardware platform containing '''64-bit cores''', such as Intel servers, it is no longer necessary for the kernel to use physical addressing to access data in remote clusters, since all the physical space can be mapped into the 64-bit virtual space. Therefore, both the local accesses and the remote access primitives uses the MMU to translate a 64 bits virtual address to a 64 bits physical address. However, to enhance access localization while minimizing contention, the ALMOS-MKH communication model continues to explicitly distinguish local and remote accesses. |
| | 19 | On a typical Intel-based hardware platform containing '''64-bit cores''', the physical address has 44 bits : the 4 MSB bits define the target cluster identifier, and the 40 LSB bits define the local physical address LPA. To avoid contention, the kernel code is replicated in all clusters to define one KCODE physical segment. and ALMOS-MKH uses the '''Instruction MMU''' to map - in each cluster - the local kernel code copy in the kernel virtual space. |
| | 20 | Regarding the data accesses, each cluster contains one KDATA and one KHEAP physical segments (the KHEAP physical segment contains all local physical memory not occupied by KCODE and KDATA). As the 48 bits virtual address space is large enough to map all these distributed KDATA[cxy] and KHEAP[cxy] segments, they can be all mapped in the kernel virtual space, and the '''Data MMU''' is used to translate both the local and the remote data accesses. |
| | 21 | |
| | 22 | On the TSAR hardware platforms containing '''32-bit cores''', the physical address has 40 bits : the 8 MSB bits define the target cluster identifier CXY, and the 32 LSB bits define the local physical address LPA. On these architectures, |
| | 23 | the virtual address is 32 bits, and this virtual space is too small to map all the distributed KDATA[cxy] and KHEAP[cxy] physical segments. On these architectures, ALMOS-MKH kernel runs partially in physical addressing: the kernel code is still replicated in all clusters, and uses the '''Instruction MMU''' to map the local kernel code copy in the kernel virtual space. |
| | 24 | But, for data accesses, the '''Data MMU'' is deactivated as soon as a core enters the kernel, and it is reactivated when it returns to user. To build a 40 bits physical address from a 32 bits pointer, ALMOS-MK uses a software controlable (TSAR-specific) register, containing a cluster identifier, that is concatained to the 32 bits pointer (in pseudo identity mapping). The default value for this register is the local cluster identifier, and is used to access the local physical memory. To access physical memory in a remote cluster, the ''remote_read'' and ''remote_write'' primitives modify the extension register before the remote memory access, and restore it after the remote memory access. |
