Changes between Version 56 and Version 57 of WikiStart


Ignore:
Timestamp:
Jan 11, 2020, 6:09:33 PM (2 years ago)
Author:
alain
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • WikiStart

    v56 v57  
    55This document describes the general principles of ALMOS-MKH, which is an operating system targeting manycore architectures with CC-NUMA (Coherent Cache, Non Uniform Memory Access) shared address space, such as the TSAR architecture which can support up to 1024 32-bit MIPS cores. ALMOS-MKH also targets INTEL / AMD multi-core architectures using 64-bit I86 cores.
    66
    7 Targeted architectures are assumed to be clustered, with one or more core and a physical memory bank per cluster. These architecture must support POSIX standard multi-threaded parallel applications.
     7Targeted architectures are assumed to be clustered, with one or more core and a physical memory bank per cluster. These architectures are supposed to support POSIX-compliant multi-threaded parallel applications.
    88
    99ALMOS-MKH inherited from the ALMOS system, developed by Ghassan Almaless. The general principles of the ALMOS system are described in his thesis.
    1010
    11 A first version of ALMOS-MKH, and in particular the distributed file system and the communication mechanism by RPC were developed by Mohamed Karaoui. The general principles of the proposed Multi-Kernel approach are described in his thesis. This system was called ALMOS-MK without H.
     11A first version of ALMOS-MKH, and in particular the distributed file system and the RPC communication mechanism were developed by Mohamed Karaoui. The general principles of the proposed Multi-Kernel approach are described in his thesis. This system was called ALMOS-MK without H.
    1212
    13 ALMOS-MKH is based on the "Multi-Kernel" approach to ensure scalability, and support the distribution of system services. In this approach, each cluster of the architecture contains an instance of the kernel. Each instance controls the local resources (memory and computing cores). These multiple instances cooperate with each other to give applications the image of a single system controlling all resources. They communicate with each other using both (i) the client / server model, sending a Remote Procedure Call to a remote cluster for a complex service, (ii) the shared memory paradigm, making direct access to remote memory when required.
     13ALMOS-MKH is based on the "Multi-Kernel" approach to ensure scalability, and support the distribution of system services. In this approach, each cluster of the architecture contains an instance of the kernel. Each instance controls the local resources (memory and computing cores). These multiple instances cooperate with each other to give applications the image of a single system controlling all resources. They communicate with each other using both (i) the client / server model, sending a remote procedure call  (RPC) to a remote cluster for a complex service, (ii) the shared memory paradigm, making direct access to remote memory when required.
    1414
    15 To reduce energy consumption, ALMOS-MKH supports architectures using 32-bit cores. In this case, each cluster has a 32-bit physical address space, addressed by the LPA (Local Physical Address). To access the physical addressing space of other clusters, ALMOS-MKH defines 64-bit global physical addresses (CXY,LPA). For example, the physical space of the TSAR architecture has 40 bits, the 8 most significant bits define the target cluster identifier CXY, and the 32 LSB bits define the local physical address LPA. The ALMOS-MKH kernel thus explicitly distinguishes two types of access:
    16   * Local access (internal to a cluster) uses standard pointers.
    17   * Remote accesses (to another cluster) use extended pointers.
     15ALMOS-MKH supports both architectures using 64-bits cores, such as the Intel based multi-core servers, or architectures using 32-bits cores, such as the TSAR-Mips32 many-core architecture.
    1816
    19 On hardware platforms containing '''32-bit cores''', such as TSAR, the ALMOS-MKH kernel runs partially in physical addressing: To reduce contention, ALMOS_MKH replicates the kernel code in all clusters, and uses - in each cluster - the Instruction MMU to map the local kernel code copy in the kernel virtual space. But the Data MMU is deactivated as soon as a core enters the kernel, and it is reactivated when it returns to user. To access the local addressable resources (memory or devices), ALMOS-MK uses standard 32 bits pointers in identity mapping.  To access ressources in another cluster, ALMOS-MKH uses the ''remote_read'' and ''remote_write'' primitives implementing the 64-bit extended pointers (CXY / PTR). CXY is the target cluster identifier, and PTR is the local pointer in the target cluster. These remote access primitives are used to implement the RPC mechanism, but are also used for fast access to kernel distributed data structures, which are critical in performance. For the applications ALMOS_MKH, as any POSIX compliant OS, uses the MMU for both the instructions and the data.
     17The main ALMOS-MKH specific feature is the following: the physical address space is supposed to be distributed between the clusters, and the MSB bits of the physical address  are supposed to define the target cluster. In a physical address, the LSB bits contain the '''lpa''' (Local Physical Address), and the MSB bits define the '''cxy''' (Cluster Identifier).  The physical address space is therefore described as a two levels array M[cxy][lpa]. To enforce locality, a kernel instance  can use a normal pointer '''ptr''' to access the local physical memory. But, to access the physical memory of a remote cluster (cxy), it must use the specific remote access functions ''remote_read(cxy,ptr)'' and ''remote_write(cxy,ptr), where '''ptr''' is the local pointer in the target cluster. This (cxy,ptr) couple is called an ''extended pointer''.
    2018
    21 On a hardware platform containing '''64-bit cores''', such as Intel servers, it is no longer necessary for the kernel to use physical addressing to access data in remote clusters, since all the physical space can be mapped into the 64-bit virtual space. Therefore, both the local accesses and the remote access primitives uses the MMU to translate a 64 bits virtual address to a 64 bits physical address. However, to enhance access localization while minimizing contention, the ALMOS-MKH communication model continues to explicitly distinguish local and remote accesses.
     19On a typical Intel-based hardware platform containing '''64-bit cores''', the physical address has 44 bits : the 4 MSB bits define the target cluster identifier, and the 40 LSB bits define the local physical address LPA. To avoid contention, the kernel code is replicated in all clusters to define one KCODE physical segment. and ALMOS-MKH uses the '''Instruction MMU''' to map - in each cluster -  the local kernel code copy in the kernel virtual space. 
     20Regarding the data accesses, each cluster contains  one KDATA and one KHEAP physical segments (the KHEAP physical segment contains all local physical memory not occupied by KCODE and KDATA). As the 48 bits virtual address space is large enough to map all these distributed KDATA[cxy] and KHEAP[cxy] segments, they can be all mapped in the kernel virtual space, and the '''Data MMU''' is used to translate both the local and the remote data accesses.
     21
     22On the TSAR hardware platforms containing '''32-bit cores''', the physical address has 40 bits : the 8 MSB bits define the target cluster identifier CXY, and the 32 LSB bits define the local physical address LPA. On these architectures,
     23the virtual address is 32 bits, and this virtual space is too small to map all the distributed KDATA[cxy] and KHEAP[cxy] physical segments. On these architectures, ALMOS-MKH kernel runs partially in physical addressing: the kernel code is still replicated in all clusters, and uses  the '''Instruction MMU''' to map the local kernel code copy in the kernel virtual space.
     24But, for data accesses, the '''Data MMU'' is deactivated as soon as a core enters the kernel, and it is reactivated when it returns to user. To build a 40 bits physical address from a 32 bits pointer, ALMOS-MK uses a software controlable (TSAR-specific) register, containing a cluster identifier, that is concatained to the 32 bits pointer (in pseudo identity mapping). The default value for this register is the local cluster identifier, and is used to access the local physical memory. To access physical memory in a remote cluster, the ''remote_read'' and ''remote_write'' primitives modify the extension register before the remote memory access, and restore it after the remote memory access.
    2225
    2326In both cases, communications between kernel instances are therefore implemented by a mix of RPCs (on the client / server model), and direct access to remote memory (when this is useful for performance). This hybrid approach is the main originality of ALMOS-MKH.