Changes between Version 55 and Version 56 of replication_distribution


Ignore:
Timestamp:
Dec 10, 2019, 1:29:51 AM (3 years ago)
Author:
alain
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • replication_distribution

    v55 v56  
    99The data to be placed are the virtual segments defined - at compilation time - in the virtual space of the various user processes currently running, or in the virtual space of the operating system itself. 
    1010
    11 == __1. general principles__ ==
     11== __1. General principles__ ==
    1212
    1313To actually control the placement of all these virtual segments on the physical memory banks, the kernel uses the paged virtual memory MMU to map a virtual segment to a given physical memory bank in a given cluster.
     
    1141141. The values contained in the N KDATA vsegs are initially identical, as they are all defined by the same ''kernel.elf'' file. But they are not read-only,  and can evolve differently in different clusters.
    1151151. The N KDATA vsegs are ''public'', and define an addressable storage space N times larger than one single KDATA vseg.  Even if most accesses are local, a thread running in cluster K must be able to access a global variable stored in another cluster X, or to send a request to another kernel instance in cluster X, or to scan a globally distributed structure, such as the DQDT or the VFS.
    116 To support this inter-cluster kernel-to-kernel communication, almos-mkh defines the ''remote_load( cxy , ptr )'' and ''remote_store( cxy , ptr ) functions, where ''ptr'' is a normal pointer (in kernel virtual space) on a variable stored in the KDATA vseg, and ''cxy'' is the remote cluster identifier. Notice that a given global variable is now identified by and extended pointer ''XPTR( cry , ptr )''. With these remote access primitives, any kernel instance in cluster K can access any global variable in any cluster. Notice that local accesses can use the normal pointers in virtual kernel space, as the virtual adresses will be simply translated by the MMU to the local physical address.
     116To support this inter-cluster kernel-to-kernel communication, almos-mkh defines the ''hal_remote_load( cxy , ptr )'' and ''hal_remote_store( cxy , ptr ) functions, where ''ptr'' is a normal pointer (in kernel virtual space) on a variable stored in the KDATA vseg, and ''cxy'' is the remote cluster identifier. Notice that a given global variable is now identified by and extended pointer ''XPTR( cry , ptr )''. With these remote access primitives, any kernel instance in cluster K can access any global variable in any cluster. Notice that local accesses can use the normal pointers in virtual kernel space, as the virtual adresses will be simply translated by the MMU to the local physical address.
    117117
    118118In other words, almost-mkh clearly distinguish the ''local accesses'', that can use standard pointers, from the ''remote access'' that '''must''' extended pointers.
     
    131131Finally the '''KDEV''' vsegs are associated to the peripheral. There is one KDEV vseg per chdev (i.e. per channel device.
    132132
    133 === 3.5. Summary ===
    134133
    135134
     
    137136== __4. Address Translation for kernel vsegs__ ==
    138137
    139 The implementation of the remote access functions introduced in section 3 depends on the target architecture.
    140 
    141 
    142  rewant to communicate withA the writable and readable bi any functionin this vseg are identical in all clusters, and defined bbut this vseg being read/write,
    143 the contentglobal data. ALMOS-MK creates one DATA vseg in each vseg, that is registered in the reference VSL(P,KREF) when the process P is created in reference cluster KREF.  In the other clusters K, the DATA vseg is registered  in VSL(P,K) when a page fault is signaled by a thread of P running in cluster K. To avoid contention, this vseg is physically distributed on all clusters, with a page granularity. For each page, the physical mapping is defined by the LSB bits of the page VPN.* The read-only segment containing the user code is replicated in all clusters where there is at least one thread using it.
    144 
    145 
    146 The TSAR implementation is available in the  [https://www-soc.lip6.fr/trac/almos-mkh/browser/trunk/hal/tsar_mips32/core/hal_remote.c  almos_mkh/hal/tsar_mips32/core.hal_remote.c] file.
     138The detailed implementation of the virtual to physical address translation depends on the target architecture.
    147139
    148140
    149141=== 4.1 TSAR-MIPS32 ===
    150142
    151 As the TSAR architecture uses 32 bits cores, to reduce the power consumption, the virtual space is much  smaller (4 Gbytes) than the physical space.
     143As the TSAR architecture uses 32 bits cores, to reduce the power consumption, the virtual space is bounded to 4 Gbytes.
    152144
    153 But the TSAR architecturethe MIPS32 core used BY the TSAR architecture HAS TWO NON STANDARDdefines another, non-standard, hardware mechanism to build a 40 bits physical address from a 32 bits virtual address. build  an al
     145But the TSAR architecture provides two non standard, but very useful features to simplify the virtual to physical address translation for kernel vsegs :
     1461. The TSAR 40 bits physical address has a specific format : it is the concaténation of an 8 bits CXY field, and a 32 bits LPADDR field, where the CXY defines
     147the cluster identifier, and the LPADDR is the local physical address inside the cluster.
     1481. the MIPS32 core used BY the TSAR architecture defines, besides the standard MMU, another non-standard,hardware mechanism for address translation : A 40  bits physical address is simply build by appending to each 32 bits virtual address a 8 bits extension contained in a software controllable register, called DATA_PADDR_EXT.
    154149
    155 This mechanism can bu used
     150In the TSAR architecture, and for any process P in any cluster K, almost-mkh registers only one extra KCODE vseg in the VMM[P,K), for kernel adressing, because almos-mkh uses the INST-MMU for instruction addresses translation, but does NOT not use the DATA-MMU for data addresses translation : When a core enters the kernel, the DATA-MMU is deactivated, and it is only reactivated when the core returns to user code.
    156151
    157 To enforce locality, there is one KDATA segment per cluster, containing a copy of all global variables statically allocated at compilation time. On the other hand, all structures dynamically allocated by the kernel (to create a new process descriptor, a new thread descriptor, a new file descriptor, etc.) are allocated in the KHEAP segment of the target cluster, and will be mainly handled by a kernel instance running in this same kernel. Therefore, most kernel memory accesses expected to be local.
     152When the value contained in the extension register is the local cluster identifier, any local kernel structures stored in the KDATA or the KHEAP segments is accessed by using directly the local physical addresses (identity mapping).
     153To access a remote kernel structure, almost-mkh '''must''' use the hardware architecture dependent remote access functions presented in section C. For the TSAR architecture these load/store functions simply modify the extension register DATA_PADDR_EXT before the memory access, and restore it after the memory access.
    158154
    159 In the - rare - situations where the kernel running in cluster K must access data in a remote cluster K' (to access a globally distributed structure such as the DQDT, or for inter-cluster client/server communication) almos-mkh uses specific remote access primitives defined in the [https://www-soc.lip6.fr/trac/almos-mkh/browser/trunk/hal/generic/hal_remote.h  hal_remote.h] file.
     155This pseudo identity mapping impose some constraints on the KCODE and the KDATA segments when compiling the kernel.
    160156
    161 
    162 
    163 In the TSAR architecture, and for any process P in any cluster K, almost-mkh registers only one extra KCODE vseg in the VMM[P,K), because almos-mkh does not use the DATA-MMU during kernel execution : Each time a core enters the kernel, to handle a sys call, an interrupt, or an exception, the DATA-MMU is deactivated, and It is reactivated when the core returns to user code.
    164 
    165 The architecture dependent [https://www-soc.lip6.fr/trac/almos-mkh/browser/trunk/hal/tsar/mips32/core/hal_remote.c  remote access] functions use the TSAR specific extension register to build a 40 bits physical address from the 32 bits virtual address.
    166 
    167 This pseudo identity mapping impose some constraints on the KCODE vseg.
     157The implementation of the hal_remote_load() and hal_remote_store() functions for the TSAR architecture is available in the  [https://www-soc.lip6.fr/trac/almos-mkh/browser/trunk/hal/tsar_mips32/core/hal_remote.c  almos_mkh/hal/tsar_mips32/core.hal_remote.c] file.
    168158
    169159
     
    172162TODO
    173163
    174 == __4. virtual space organisation__ ==
     164== __5. Virtual space organisation__ ==
    175165
    176166This section describes the almost-mkh assumptions regarding the virtual space organisation, that is strongly dependent on the size of the virtual space. 
     
    204194It is located on top of the '''mmap''' zone and starts at the address defined by the [https://www-soc.lip6.fr/trac/almos-mkh/browser/trunk/kernel/kernel_config.h CONFIG_VMM_STACK_BASE] parameter. It contains an array of fixed size slots, and each slot contains one ''stack'' vseg. The size of a slot is defined by the [https://www-soc.lip6.fr/trac/almos-mkh/browser/trunk/kernel/kernel_config.h CONFIG_VMM_STACK_SIZE]. In each slot, the first page is not mapped, in order to detect stack overflows. As threads are dynamically created and destroyed, the VMM implements a specific STACK allocator for this zone, using a bitmap vector. As the ''stack'' vsegs are private (the same virtual address can have different mappings, depending on the cluster) the number of slots in the '''stack''' zone actually defines the max number of threads for given process in a given cluster.
    205195
    206 === 4.2 Intel 64 bits ===
     196=== 5.2 Intel 64 bits ===
    207197
    208198TODO