Changes between Version 48 and Version 49 of replication_distribution


Ignore:
Timestamp:
Dec 8, 2019, 11:18:12 PM (3 years ago)
Author:
alain
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • replication_distribution

    v48 v49  
    4646This section describes the six types of user virtual segments defined by almost-mkh:
    4747
    48 || Type     ||           ||                  ||    Access     ||      Replication                                     ||    Placement                              ||  Allocation policy in user space                  ||
    49 || STACK    ||  private  || localized   || Read Write || one physical mapping per thread        || same cluster as thread using it  || dynamic (one stack allocator per cluster)   ||
    50 || CODE     ||  private  || localized   || Read Only   || one physical mapping per cluster       || same cluster as thread using it  || static  (defined in .elf file)                           ||
     48|| Type     ||           ||                  ||    Access     ||      Replication                                     ||    Placement                              ||  Allocation policy in virtual  space           ||
     49|| STACK    ||  private  || localized   || Read Write || one physical mapping per thread    || same cluster as thread using it  || dynamic (one stack allocator per cluster)   ||
     50|| CODE     ||  private  || localized   || Read Only   || one physical mapping per cluster   || same cluster as thread using it  || static  (defined in .elf file)                           ||
    5151|| DATA     ||  public   || distributed || Read Write || same mapping for all threads              || distributed on all clusters          || static  (defined in .elf file)                            ||
    5252|| ANON     ||  public   || localized   || Read Write || same mapping for all threads              || same cluster as calling thread   || dynamic (one heap allocator per process   ||
     
    6565== __ 2. kernel segments types__==
    6666
    67 For any process descriptor P in a cluster K, the VSL(P,K) and the GPT(P,K) contains not only the user vsegs, but also the kernel vsegs, because all user theads can make system calls, that must access to these kernel vsegs,
    68 with requires address translation. This section describes the three types of kernel virtual segments defined by almost-mkh
     67For any process descriptor P in a cluster K, the VSL(P,K) and the GPT(P,K) contains not only the user vsegs, but also the kernel vsegs, because all user theads can make system calls, that must access to these kernel vsegs, and this requires address translation. This section describes the three types of kernel virtual segments defined by almost-mkh
    6968
    70  1. '''KCODE''' : This '''private''' vseg contains the kernel code. It is replicated in all clusters. ALMOS-MK creates one KCODE vseg per cluster. For a process P, the CODE vseg is registered in the VSL(P,Z) when the process is created in reference cluster KREF. In the other clusters K, the CODE vseg is registered in VSL(P,K) when a page fault is signaled by a thread of P running in cluster K. In each active cluster K, the CODE vseg is localized, and physically mapped in cluster K.
    71  1. '''KDATA''' : This '''public''' vseg contains the user application global data. ALMOS-MK creates one DATA vseg, that is registered in the reference VSL(P,KREF) when the process P is created in reference cluster KREF.  In the other clusters K, the DATA vseg is registered  in VSL(P,K) when a page fault is signaled by a thread of P running in cluster K. To avoid contention, this vseg is physically distributed on all clusters, with a page granularity. For each page, the physical mapping is defined by the LSB bits of the page VPN.* The read-only segment containing the user code is replicated in all clusters where there is at least one thread using it.
    72  * The private segment containing the stack for a given thread is placed in the same cluster as the thread using it.
    73  * The shared segment containing the global data is distributed on all clusters as regularly as possible to avoid contention.
    74  * The segments dynamically allocated by the [https://www-soc.lip6.fr/trac/almos-mkh/browser/trunk/kernel/syscalls/sys_mmap.c] system call are placed  as described below.
     69|| Type        ||               ||                  ||    Access     ||      Replication                                     ||    Placement                              ||  Allocation policy in virtual space           ||
     70|| KCODE    ||  private  || localized   || Read Only   || one physical mapping per cluster       || same cluster as thread using it  || static  (defined in .elf file)                           ||
     71|| KDATA     ||  public   || localized   || Read Write || same mapping for all threads              || distributed on all cl         || static  (defined in .elf file)                            ||
     72|| KHEAP    || public
     73|| KDEV      ||  public   || localized   || Read Write || one physical mapping per thread        || same cluster as thread using it  || dynamic (one stack allocator per cluster)   ||
    7574
    76 == __3. 32 bits virtual space organisation__ ==
     75 1. '''KCODE''' : This '''private''' vseg contains the kernel code. Almost-mkh creates one KCODE vseg per cluster. For a process P, the CODE vseg is registered in the VSL(P,Z) when the process is created in reference cluster KREF. In the other clusters K, the CODE vseg is registered in VSL(P,K) when a page fault is signaled by a thread of P running in cluster K. In each active cluster K, the CODE vseg is localized, and physically mapped in cluster K.
     76 1. '''KDATA''' : This '''public''' vseg contains the global data,  statically allocated at compilation time. The initial values are identical in all clusters, but th global data. ALMOS-MK creates one DATA vseg in each vseg, that is registered in the reference VSL(P,KREF) when the process P is created in reference cluster KREF.  In the other clusters K, the DATA vseg is registered  in VSL(P,K) when a page fault is signaled by a thread of P running in cluster K. To avoid contention, this vseg is physically distributed on all clusters, with a page granularity. For each page, the physical mapping is defined by the LSB bits of the page VPN.* The read-only segment containing the user code is replicated in all clusters where there is at least one thread using it.
     77
     78To enforce locality, there is one KDATA segment per cluster, containing a copy of all global variables statically allocated at compilation time. But these vsegs are not
     79read-only, and can evolve differently in different clusters. On the other hand, all structures dynamically allocated by the kernel (to create a new process descriptor, a new thread descriptor, a new file descriptor, etc.) are allocated in the KHEAP segment of the target cluster, and will be mainly handled by a kernel instance running in this same kernel. Therefore, most kernel memory accesses expected to be local.
     80
     81In the - rare - situations where the kernel running in cluster K must access data in a remote cluster K' (to access a globally distributed structure such as the DQDT, or for inter-cluster client/server communication) almos-mkh uses specific remote access primitives defined in the [https://www-soc.lip6.fr/trac/almos-mkh/browser/trunk/hal/generic/hal_remote.h  hal_remote.h] file.
     82
     83=== 2.1 TSAR-MIPS32  implementation ===
     84
     85In the TSAR architecture, and for any process P in any cluster K, almost-mkh registers only one extra KCODE vseg in the VMM[P,K), because almos-mkh does not use the DATA-MMU during kernel execution : Each time a core enters the kernel, to handle a sys call, an interrupt, or an exception, the DATA-MMU is deactivated, and It is reactivated when the core returns to user code.
     86
     87The architecture dependent [https://www-soc.lip6.fr/trac/almos-mkh/browser/trunk/hal/tsar/mips32/core/hal_remote.c  remote access] functions use the TSAR specific extension register to build a 40 bits physical address from the 32 bits virtual address.
     88
     89This pseudo identity mapping impose some constraints on the KCODE vseg.
     90
     91
     92=== 2.2 I64 implementation ===
     93
     94TODO
     95
     96== __3. virtual space organisation__ ==
     97
     98=== 3.1 TSAR-MIP32 ===
    7799
    78100The virtual address space of an user process P is split in 5 fixed size zones, defined by configuration parameters in [https://www-soc.lip6.fr/trac/almos-mkh/browser/trunk/kernel/kernel_config.h].  Each zone contains one or several vsegs, as described below.
     
    80102=== The ''kernel'' zone ===
    81103
    82 It contains the ''kcode'' vseg (type KCODE), that must be mapped in all user process to support syscalls.
    83 It is located in the lower part of the virtual space, and starts a address 0. Its size cannot be less than a big page size (2 Mbytes for the TSAR architecture), because it will be mapped as one or several big pages.
     104It contains the ''kcode'' vseg (type KCODE), that must be mapped in all user processes.
     105It is located in the lower part of the virtual space, and starts a address 0. Its size cannot be less than a big page size (2 Mbytes for the TSAR architecture), because it will be mapped as one (or several big) pages.
    84106 
    85107=== The ''utils'' zone ===
     
    95117=== The ''heap'' zone ===
    96118
    97 It contains all vsegs dynamically allocated / released  by the [https://www-soc.lip6.fr/trac/almos-mkh/browser/trunk/kernel/syscalls/sys_mmap.c mmap()] / [https://www-soc.lip6.fr/trac/almos-mkh/browser/trunk/kernel/syscalls/sys_munmap.c munmap()] system calls (i.e. FILE / ANON / REMOTE types).
     119It contains all vsegs dynamically allocated / released  by the [https://www-soc.lip6.fr/trac/almos-mkh/browser/trunk/kernel/syscalls/sys_mmap.c mmap] / [https://www-soc.lip6.fr/trac/almos-mkh/browser/trunk/kernel/syscalls/sys_munmap.c munmap] system calls (i.e. FILE / ANON / REMOTE types).
    98120It is located on top of the '''elf''' zone, and starts at the address defined by the [https://www-soc.lip6.fr/trac/almos-mkh/browser/trunk/kernel/kernel_config.h CONFIG_VMM_HEAP_BASE]  parameter. The VMM defines a specific MMAP allocator for this zone, implementing the ''buddy'' algorithm. The mmap( FILE ) syscall maps directly a file in user space. The user level ''malloc'' library uses the mmap( ANON ) syscall to allocate virtual memory from the heap and map it in the same cluster as the calling thread. Besides the standard malloc() function, this library implements a non-standard [https://www-soc.lip6.fr/trac/almos-mkh/browser/trunk/libs/libalmosmkh/almosmkh.c remote_malloc()] function, that uses the mmap( REMOTE ) syscall to dynamically allocate virtual memory from the heap, and map it to a remote physical cluster. 
    99121