wiki:boot_procedure

Version 45 (modified by alain, 5 months ago) (diff)

--

Boot procedure

A) General Principles

The ALMOS-MKH boot procedure can be decomposed in two phases:

  • The architecture dependent phase, implemented by an architecture specific boot_loader procedure.
  • The architecture independent phase, implemented by a generic (architecture independent) kernel-init procedure.

As the generic (i.e. architecture independent) kernel initialization procedure is executed in parallel by all kernel instances in all clusters containing at least one core and one memory bank, the main task of the boot-loader is to load - in each cluster - a local copy of the ALMOS-MKH kernel code, and a description of the hardware architecture, contained in a local boot_info_t data-structure.

This fixed size boot_info_t structure is build by the boot-loader, and stored at the beginning of the local copy of the kdata segment. As it contains both general and cluster specific information, the content depends on the cluster:

  • general hardware architecture features : number of clusters, topology, etc.
  • available external (shared) peripherals : types and features.
  • number of cores in cluster,
  • available internal (private) peripherals in cluster : types and features.
  • available physical memory in cluster.

This boot_info_t structure is defined in the boot_info.h file.

To build the various boot_info_t structures (one per cluster), the boot-loader uses the arch_info_t binary structure, that is described in section Hardware Platform Definition. This binary structure is contained in the arch_info.bin file, and must be stored in the file system root directory.

This method allows an intelligent boot_loader to check and - if required - reconfigure the hardware components, to guaranty that the generated boot_info_t structures contain only functionally tested hardware components.

We describe below the boot_loader for the TSAR architecture, the boot_loader for the I86 architecture, and the generic kernel initialization procedure.

B) Boot-loader for the TSAR architecture

The TSAR boot-loader uses an OS-independent pre-loader, stored in an addressable but non-volatile device, that load the TSAR boot-loader code from an external block-device to the memory. This preloader is specific for the TSAR architecture, but independent on the Operating System. It is used by ALMOS-MKH, but also by LINUX, NetBSD, ALMOS_MKH, or the GIET-VM.

The TSAR boot_loader allocates - in each cluster containing a physical memory bank - five fixed size memory zones, to store various binary files or data structures. The two first zones are permanently allocated: The PRE_LOADER zone is only defined in cluster 0. The KERNEL_CODE zone containing the kcode and kdata sgments is directly used by the kernel when the boot_loader transfers control - in each cluster - to the kernel_init procedure. The BOOT_ICODE, ARCH_INFO, KERNEL_ELF, and BOOT_CODE zones are temporary: they are only used - in each cluster - by the boot-loader code, and the corresponding physical memory can be freely used and re-allocated by the local kernel instance.

name description base address (physical) size
PRE_LOADER pre-loader code PRELOADER_BASE (0) PRELOADER_MAX_SIZE (16KB)
KERNEL_CODE kernel code and data KERNEL_CODE_BASE (16 KB) KERNEL_CODE_MAX_SIZE (2 MB - 16 KB)
BOOT_CODE boot-loader code and data BOOT_CODE_BASE (2 MB) BOOT_CODE_MAX_SIZE (1 MB)
ARCH_INFO arch_info.bin file copy ARCH_INFO_BASE (3 MB) ARCH_INFO_MAX_SIZE (1 MB)
KERNEL_ELF kernel.elf file copy KERNEL_ELF_BASE (4 MB) KERN_ELF_MAX_SIZE (2 MB)
BOOT_STACK boot stacks (one per core) BOOT_STACK_BASE (6 MB) BOOT_STACK_MAX_SIZE (1MB)

The values given in this array are indicative. The actual values are defined by configuration parameters in the boot_config.h file. The main constraint is : the kcode'" segment (in the KERNEL_CODE zone must be entirely contained in one single big physical page (2 MBytes) because it will be mapped as one single big page in all process virtual spaces.

All cores contribute to the boot procedure, but all cores are not simultaneously activated. A core is identified by two indexes[cxy][lid] : cxy is the cluster identifier, an lid is the core local index in cluster:

  • in a first phase, only core[0][0] is running (core 0 in cluster 0).
  • In a second phase, only core[cxy][0] are running (one core per cluster).
  • in last phase, all core[cxy][lid] are running.

We describe below the four phases of the TSAR boot-loader:

B1. Preloader phase

At reset, the MMU is de-activated (for both data and instructions), and the extension address registers supporting direct access to remote memory banks (for both data and instructions) in all cores[cxy][lid] contain the 0 value. Therefore, all cores can only access the physical address space of cluster 0.

  • In the TSAR_LETI architecture, the preloader is loaded in the first 16 kbytes of the physical address space in cluster 0.
  • In the TSAR_IOB architecture, the preloader is stored in an external ROM, that is accessed throug the IO_bridge located in cluster 0.

All cores execute the same preloader code (stored in cluster 0, but the work done depends on the core identifier:

  • The core[0][0] load in the BOOT_CODE zone of cluster 0, the boot-loader code.
  • All other cores do only one task before going to sleep (i.e. low-power state): each core activates its private WTI channel in the local ICU (Interrupt Controller Unit) to be later activated by an IPI (Inter Processor Interrupt).

This shows the memory content after this pre-loader phase.

Contenu de l'espace adressable physique des clusters après phase 1

B2. Boot-loader Sequencial phase

The first instructions of the boot-loader are defined in the boot_entry.S file. This assembly code is executed by all cores entering the boot loader, but not at the same time, because in this phase, only core [0][0] is running, while all other cores are blocked in the preloaded, waiting to be activated by an IPI.

This assembly code makes the assuption that the CP0 register containing the core gid (global hardware identifier) has a fixed format: * gid == (((x << Y_WIDTH) + y) << P_WIDTH) + lid *

*

Each core running this code makes the 3 following actions: *

  • It initializes the core stack pointer depending on the lid value extracted from the gid, using the BOOT_STACK_BASE and BOOT_STACK_SIZE parameters defined in the boot_config.h file, *
  • It changes the value of the DATA address extension CP2 register, using the cxy value extracted from the gid
  • It jumps to the boot_loader() C function defined in the boot.c file, passing the two (cxy , lid) arguments.

In this sequencial phase, the core[0][0] executes the following actions:

  • The core[0][0] initializes 2 peripherals: The TTY terminal (channel 0) to display log messages, and the IOC peripheral to access the disk.
  • The core[0][0] initializes the boot-loader private FAT32 structure, allowing the boot loader to access files stored in the FAT32 file system on disk.
    • The core[0][0] load in the KERNEL_ELF zone the kernel.elf file from the disk file system..
    • Then it copies in the KERNEL_CORE zone the kcode and data segments, using the addresses contained in the .elf file (identity mapping).
    • The core[0][0] load in the ARCH_INFO zone the arch_info.bin file from the disk file system.
    • Then it builds from this arch_info.bin file the specific boot_info_t structure for cluster 0, and stores it in the kdata segment.
    • The core[0][0] send IPIs to activate all cores [i][0] in all other clusters.

This shows the memory content after this phase.

Contenu de l'espace adressable physique des clusters après phase 2

B3. Boot-loader partially parallel phase

This phase is executed by all core[cxy][0], but the core[0][0].

At this point, all INSTRUCTION address extension registers point on cluster(0,0), but the DATA extension registers point already on the local cluster to use the local stack. To access the bootloader global variables the core[cxy][0] must first copy the boot code (data and instructions) in the BOOT_CORE zone of cluster cxy.

In each cluster(i), the core[I][0] exécutes the boot-loader code (stored in physical memory of cluster 0), to do he following tasks:

  • each core[i][0] analyses the arch_info.bin structure (stored in physical memory of cluster 0), to search his own cluster identifier cxy. This is done in parallel by all cores[i][0], and can create contention.
  • each core[i][0] updates its own extended address register to access the data stored in its local physical memory. Nevertheless, it still access to the boot code stored in cluster(0), as long as the code has not been copied in local cluster.
  • each core[i][0] allocates a private stack at address BOOT_STACK_BASE in its local memory.
  • each core[i][0] copy l'image du boot-loader et le fichier arch_info.bin aux mêmes adresses, respectivement 0x100000 et 0x200000, dans la mémoire physique locale. À partir d'ici, chaque CP0 peut exécuter le code du boot-loader en local.
  • Il copie ensuite l'image du noyau à l'adresse 0x4000 de la mémoire physique locale de son cluster (c'est à dire, juste après les quatre pages réservées au prélasser).
  • Il utilise la structure arch_info.bin locale pour initialiser les différents champs de la structure boot_info_t de son cluster. Cette tâche n'utilise que des accès mémoire locaux puisque toutes les informations nécessaires sont disponibles localement.
  • Il arrive à la barrière de synchronisation, et le dernier CP0 débloque tous les CP0 (y compris bscpu),
  • Chaque CP0 envoie des IPIs pour réveiller les autres cores dans son cluster local.
  • Les CP0 se mettent en attente jusqu'à ce que tous les autres cores arrivent à ce point de rendez-vous en utilisant le mécanisme de barrière de synchronisation.

This shows the memory content after this phase.

Contenu de l'espace adressable physique des clusters après phase 3

B4. Fully parallel phase

Chaque core CPi (lid non nul), réveillé par le CP0 local de son cluster, sort du code du preloader et exécute le boot-loader dans le cluster de boot puisque ses registres d'extension d'adresse ne sont pas encore mis à jour. Une fois sortis du preloader, ces cores décrémentent le compteur de la barrière de synchronisation et débloquent les CP0. Tous ces CP0 sauf un, se mettent tout de suite en attente jusqu'à ce que les CPi finissent leur exécution du boot-loader. Le seul CP0 qui n'arrive pas encore à cette barrière de synchronisation, celui du cluster(0,0), peut maintenant écraser le code du preloader en déplaçant l'image du noyau à l'adresse 0x0 de l'espace adressable physique du cluster(0,0), puisque tous les cores sont déjà sortis du preloader. Il rejoint ensuite les autres CP0 au dernier point de rendez-vous dans le boot-loader. Les CPi, quant à eux, exécutent, pour le moment, le code du boot-loader se trouvant dans le cluster de boot car leurs registres d'extension d'adresse ont toujours la valeur 0 par défaut. Chacun de ces CPi effectue les étapes suivantes:

  • Il analyse le contenu de arch_info.bin (dans l'espace adressable physique du cluster de boot) en parcourant le tableau de descripteurs de core pour retrouver son identificateur de cluster cxy ainsi que son identificateur de core local dans son cluster lid. Notons que cette étape est exécutée parallèlement par tous les CPi, ce qui entraine une contention, encore plus forte que celle créée par les accès parallèles des CP0, au banc mémoire contenant ce tableau de descripteurs de core .
  • Il peut maintenant, à partir de son cxy, mettre à jour les valeurs dans ses registres d'extension d'adresse de code et de données. Comme le CP0 du même cluster a déjà copié les informations nécessaires dans le banc mémoire local aux mêmes adresses que du cluster de boot, il peut toujours exécuter le code du boot-loader en local.
  • Il alloue sa pile de boot en initialisant son pointeur de pile à l'adresse 0x600000 - 4K*lid dans l'espace adressable physique locale de son cluster (grâce à la nouvelle valeur dans le registre d'extension d'adresse de code).
  • La structure boot_info_t du cluster étant déjà initialisée, chacun des CPi ne fait que vérifier les informations qui le concernent.
  • Il arrive finalement au point de rendez-vous avec tous les CP0, décrémente le compteur de la barrière de synchronisation et se met en attente.
  • Dès que le dernier core arrive à ce point et débloque les autres, tous les cores se branchent à la fonction kern_init().

There is the physical memory content at boot completion.

Contenu de l'espace adressable physique des clusters après phase de boot

At this point, the boot-loader completed its job:

  • The kernel code kcode and kdata segments are loaded - in all clusters - in the first offset physical pages.
  • The hardware architecture described by the arch_info.binfile has been analyzed, and copied - in each cluster - in the boot_info_t structure, stored in the kdata segment.
  • Each local kernel instance can use all the physical memory that is not used to store the kernel kcode and kdata segments themselves.

C) Boot-loader for the I86 architecture

TODO

D) Generic kernel initialization procedure

The kernel_init( boot_info_t * info ) function is the kernel entry point when the boot_loader transfer control to the kernel. The info argument is a pointer on the fixed size boot_info_t structure, that is stored in the data kernel segment.

All cores execute this procedure in parallel, but some tasks are only executed by the CP0 core. This procedure uses two synchronisation barriers, defined as global variables in the data segment:

  • the global_barrier variable is used to synchronize all CP0 cores in all clusters containing a kernel instance.
  • the local_barrier variable is used to synchronize all cores in a given cluster.

The kernel initialization procedure execute sequentially the following steps:

D1) Core and cluster identification

Each core has an unique hardware identifier, called gid, that is hard-wired in a read-only register. From the kernel point of view a core is identified by a composite index (cxy,lid), where cxy is the cluster identifier, and lid is a local (continuous) index in the cluster. The association between the gid hardware index and the (cxy,lid) composite index is defined in the boot_info_t structure. In this first step, each core makes an associative search in the boot_info_t structure to obtain the (cxy,lid) indexes from the gid index. Then the CP0 initialize the global variable local_cxy defining the local cluster identifier.

D2) TXT0 device initialization

The core[io_cxy][0] (i.e. CP0 in I/O cluster) initializes the chdev descriptor associated to the kernel text terminal TXT0. This terminal is used by any kernel instance running on any core to display log or debug messages. This terminal is configured in non-descheduling mode : the calling thread call directly the relevant TXT driver, without using a server thread.

A first synchonization barrier is used to avoid other cores to use the TXT0 terminal before initialization completion.

D3) Cluster manager initialization

In each cluster, the CP0 makes the cluster manager initialization, namely the cores descriptors array, and the physical memory allocators. Then it initializes the local process_zero, containing al kernel threads in a given cluster.

A second synchonization barrier is used to avoid other cores to access cluster manager before initialization completion.

D4) Internal & external devices initialization

In each cluster, the CP0 makes the devices initialization. For multi-channels devices, there is one channel device (called chdev_t) per channel. For internal (replicated) devices, the khedive descriptors are allocated in the local cluster. For external (shared) devices, the chdev descriptors are regularly distributed on all clusters. These external chdev are indexed by a global index, and the host cluster is computed from this index by a modulo.

The internal devices descriptors are created first( ICU, then MMC, then DMA ), because the ICU device is used by all other devices. Then the WTI mailboxes used for IPIs (Inter Processor Interrupt) are allocated in local ICU : one WTI mailbox per core. Then each external chdev descriptor is created by the CP0 in the cluster where it must be created.

A third synchonization barrier is used to avoid other cores to access devices before initialization completion.

D5) Idle thread initialization

In this step, each core creates and initializes its private idle thread descriptor.

D6) File system initialization

The CP0 in I/O cluster) initializes the file system.

A fourth synchonization barrier is used to avoid other cores to access file system before initialization completion.

D7) Scheduler activation

Finally, each core enables its private timer IRQ to activate its private scheduler, and jump to the idle thread code.= Boot procedure =

A) General Principles

The ALMOS-MKH boot procedure can be decomposed in two phases:

  • The architecture dependent phase, implemented by an architecture specific boot_loader procedure.
  • The architecture independent phase, implemented by a generic (architecture independent) kernel-init procedure.

As the generic (i.e. architecture independent) kernel initialization procedure is executed in parallel by all kernel instances in all clusters containing at least one core and one memory bank, the main task of the boot-loader is to load - in each cluster - a local copy of the ALMOS-MKH kernel code. This code includes a description of the hardware architecture, contained in the boot_info_t data-structure.

This fixed size boot_info_t structure is build by the boot-loader, and stored at the beginning of the local copy of the kdata segment. As it contains both general and cluster specific information, the content depends on the cluster:

  • general hardware architecture features : number of clusters, topology, etc.
  • available external (shared) peripherals : types and features.
  • number of cores in cluster,
  • available internal (private) peripherals in cluster : types and features.
  • available physical memory in cluster.

This boot_info_t structure is defined in the boot_info.h file.

To build the various boot_info_t structures (one per cluster), the boot-loader uses the arch_info_t binary structure, that is described in section Hardware Platform Definition. This binary structure is contained in the arch_info.bin file, and must be stored in the file system root directory.

This method allows the boot_loader to check and reconfigure the hardware components, to guaranty that the generated boot_info_t structures contain only functionally tested hardware components.

We describe below the boot_loader for the TSAR architecture, the boot_loader for the I86 architecture, and the generic kernel initialization procedure.

B) Boot-loader for the TSAR architecture

The TSAR boot-loader uses an OS-independent pre-loader, stored in an external ROM, that load the TSAR boot-loader code from an external block-device to the memory. This preloader is specific for the TSAR architecture, but independent on the Operating System. It is used by ALMOS-MKH, but also by LINUX, NetBSD, ALMOS_MKH, or the GIET-VM.

The TSAR boot_loader allocates - in each cluster containing a physical memory bank - five fixed size memory zones, to store various binary files or data structures :

size local physical address
préloader code itself PRELOADER_MAX_SIZE (16 Kb) PRELOADER_BASE(0x0)
boot-loader code local copy BOOT_MAX_SIZE (1 Mb) BOOT_BASE (0x100000)
arch_info.bin file local copy ARCHINFO_MAX_SIZE (2 Mb) ARCHINFO_BASE (0x200000)
kernel.elf binary file KERN_MAX_SIZE (1 Mb) KERN_BASE (0x400000)
execution stacks (one per core) BOOT_STACK_SIZE (1 Mb) BOOT_STACK_BASE (0x500000)

The values given in this array are indicative. The actual values are defined by configuration parameters in the boot_config.h file. These memory zones are only used for temporary storage : when the TSAR boot_loader completes, and transfer control to the kernel_init procedure, the kernel code (i.e. the code and data segments) has been copied - in each cluster - in the lowest part of the cluster physical memory. The four pages (16 Kbytes) reserved for the prelloader are only used in cluster 0.

A core is identified by two indexes[cxy][lid] : cxy is the cluster identifier, an lid is the core local index in cluster cxy.

We describe below the four phases for the TSAR boot-loader:

B1. Preloader phase

At reset, the MMU is de-activated, and the extension address registers (for both data and instructions) in all cores[cxy][lid] contain the 0 value. Therefore, all cores can only access the physical address space of cluster 0.

  • In the TSAR_LETI architecture, the preloader is loaded in the first 16 kbytes of the RAM located in cluster 0.
  • In the TSAR_IOB architecture, the preloader is stored in an external ROM, that is accessed throug the IO_bridge located in cluster 0.

All cores execute the same preloader code, but the work done depends on the core identifier. The core[0][0] (i.e. Core0 in cluster 0) load in local memory of cluster 0, the boot-loader code. All other cores do only one task before going to sleep (low-power) state: each core activates its private WTI channel in the local ICU (Interrupt Controller Unit) to be wake-up by core [0][0], using an IPI (Inter Processor Interrupt).

This shows the memory content after this first phase.

Contenu de l'espace adressable physique des clusters après phase 1

B2. Sequencial phase

In this second phase the work is entirely done by core[0][0].

  • The core[0][0] initializes the stack pointer. The boot stack size is a configuration parameter.
  • The core[0][0] initializes 2 peripherals: The TTY terminal (channel 0) to display log info, and the IOC to access the disk.
  • The core[0][0] loads in cluster 0 the arch_info.bin file and the kernel.elf file at addresses ARCHINFO_BASE and KERN_BASE respectively.
  • The core[0][0] uses the arch_info.binstructure to initialize the local boot_info_t structure in cluster 0.
  • The core[0][0] send IPIs to activate all cores [i][0] in all other clusters.

All Core0 in all clusters synchronize through a synchronisation barrier before entering the next phase. This shows the memory content after this phase.

Contenu de l'espace adressable physique des clusters après phase 2

B3. partially parallel phase

In each cluster(i), the core[I][0] exécutes the boot-loader code (stored in physical memory of cluster 0), to do he following tasks:

  • each core[i][0] analyses the arch_info.bin structure (stored in physical memory of cluster 0), to search his own cluster identifier cxy. This is done in parallel by all cores[i][0], and can create contention.
  • each core[i][0] updates its own extended address register to access the data stored in its local physical memory. Nevertheless, it still access to the boot code stored in cluster(0), as long as the code has not been copied in local cluster.
  • each core[i][0] allocates a private stack at address BOOT_STACK_BASE in its local memory.
  • each core[i][0] copy l'image du boot-loader et le fichier arch_info.bin aux mêmes adresses, respectivement 0x100000 et 0x200000, dans la mémoire physique locale. À partir d'ici, chaque CP0 peut exécuter le code du boot-loader en local.
  • Il copie ensuite l'image du noyau à l'adresse 0x4000 de la mémoire physique locale de son cluster (c'est à dire, juste après les quatre pages réservées au prélasser).
  • Il utilise la structure arch_info.bin locale pour initialiser les différents champs de la structure boot_info_t de son cluster. Cette tâche n'utilise que des accès mémoire locaux puisque toutes les informations nécessaires sont disponibles localement.
  • Il arrive à la barrière de synchronisation, et le dernier CP0 débloque tous les CP0 (y compris bscpu),
  • Chaque CP0 envoie des IPIs pour réveiller les autres cores dans son cluster local.
  • Les CP0 se mettent en attente jusqu'à ce que tous les autres cores arrivent à ce point de rendez-vous en utilisant le mécanisme de barrière de synchronisation.

This shows the memory content after this phase.

Contenu de l'espace adressable physique des clusters après phase 3

B4. Fully parallel phase

Chaque core CPi (lid non nul), réveillé par le CP0 local de son cluster, sort du code du preloader et exécute le boot-loader dans le cluster de boot puisque ses registres d'extension d'adresse ne sont pas encore mis à jour. Une fois sortis du preloader, ces cores décrémentent le compteur de la barrière de synchronisation et débloquent les CP0. Tous ces CP0 sauf un, se mettent tout de suite en attente jusqu'à ce que les CPi finissent leur exécution du boot-loader. Le seul CP0 qui n'arrive pas encore à cette barrière de synchronisation, celui du cluster(0,0), peut maintenant écraser le code du preloader en déplaçant l'image du noyau à l'adresse 0x0 de l'espace adressable physique du cluster(0,0), puisque tous les cores sont déjà sortis du preloader. Il rejoint ensuite les autres CP0 au dernier point de rendez-vous dans le boot-loader. Les CPi, quant à eux, exécutent, pour le moment, le code du boot-loader se trouvant dans le cluster de boot car leurs registres d'extension d'adresse ont toujours la valeur 0 par défaut. Chacun de ces CPi effectue les étapes suivantes:

  • Il analyse le contenu de arch_info.bin (dans l'espace adressable physique du cluster de boot) en parcourant le tableau de descripteurs de core pour retrouver son identificateur de cluster cxy ainsi que son identificateur de core local dans son cluster lid. Notons que cette étape est exécutée parallèlement par tous les CPi, ce qui entraine une contention, encore plus forte que celle créée par les accès parallèles des CP0, au banc mémoire contenant ce tableau de descripteurs de core .
  • Il peut maintenant, à partir de son cxy, mettre à jour les valeurs dans ses registres d'extension d'adresse de code et de données. Comme le CP0 du même cluster a déjà copié les informations nécessaires dans le banc mémoire local aux mêmes adresses que du cluster de boot, il peut toujours exécuter le code du boot-loader en local.
  • Il alloue sa pile de boot en initialisant son pointeur de pile à l'adresse 0x600000 - 4K*lid dans l'espace adressable physique locale de son cluster (grâce à la nouvelle valeur dans le registre d'extension d'adresse de code).
  • La structure boot_info_t du cluster étant déjà initialisée, chacun des CPi ne fait que vérifier les informations qui le concernent.
  • Il arrive finalement au point de rendez-vous avec tous les CP0, décrémente le compteur de la barrière de synchronisation et se met en attente.
  • Dès que le dernier core arrive à ce point et débloque les autres, tous les cores se branchent à la fonction kern_init().

There is the physical memory content at boot completion.

Contenu de l'espace adressable physique des clusters après phase de boot

At this point, the boot-loader completed its job:

  • The kernel code kcode and kdata segments are loaded - in all clusters - in the first offset physical pages.
  • The hardware architecture described by the arch_info.binfile has been analyzed, and copied - in each cluster - in the boot_info_t structure, stored in the kdata segment.
  • Each local kernel instance can use all the physical memory that is not used to store the kernel kcode and kdata segments themselves.

C) Boot-loader for the I86 architecture

TODO

D) Generic kernel initialization procedure

The kernel_init( boot_info_t * info ) function is the kernel entry point when the boot_loader transfer control to the kernel. The info argument is a pointer on the fixed size boot_info_t structure, that is stored in the data kernel segment.

All cores execute this procedure in parallel, but some tasks are only executed by the CP0 core. This procedure uses two synchronisation barriers, defined as global variables in the data segment:

  • the global_barrier variable is used to synchronize all CP0 cores in all clusters containing a kernel instance.
  • the local_barrier variable is used to synchronize all cores in a given cluster.

The kernel initialization procedure execute sequentially the following steps:

D1) Core and cluster identification

Each core has an unique hardware identifier, called gid, that is hard-wired in a read-only register. From the kernel point of view a core is identified by a composite index (cxy,lid), where cxy is the cluster identifier, and lid is a local (continuous) index in the cluster. The association between the gid hardware index and the (cxy,lid) composite index is defined in the boot_info_t structure. In this first step, each core makes an associative search in the boot_info_t structure to obtain the (cxy,lid) indexes from the gid index. Then the CP0 initialize the global variable local_cxy defining the local cluster identifier.

D2) TXT0 device initialization

The core[io_cxy][0] (i.e. CP0 in I/O cluster) initializes the chdev descriptor associated to the kernel text terminal TXT0. This terminal is used by any kernel instance running on any core to display log or debug messages. This terminal is configured in non-descheduling mode : the calling thread call directly the relevant TXT driver, without using a server thread.

A first synchonization barrier is used to avoid other cores to use the TXT0 terminal before initialization completion.

D3) Cluster manager initialization

In each cluster, the CP0 makes the cluster manager initialization, namely the cores descriptors array, and the physical memory allocators. Then it initializes the local process_zero, containing al kernel threads in a given cluster.

A second synchonization barrier is used to avoid other cores to access cluster manager before initialization completion.

D4) Internal & external devices initialization

In each cluster, the CP0 makes the devices initialization. For multi-channels devices, there is one channel device (called chdev_t) per channel. For internal (replicated) devices, the khedive descriptors are allocated in the local cluster. For external (shared) devices, the chdev descriptors are regularly distributed on all clusters. These external chdev are indexed by a global index, and the host cluster is computed from this index by a modulo.

The internal devices descriptors are created first( ICU, then MMC, then DMA ), because the ICU device is used by all other devices. Then the WTI mailboxes used for IPIs (Inter Processor Interrupt) are allocated in local ICU : one WTI mailbox per core. Then each external chdev descriptor is created by the CP0 in the cluster where it must be created.

A third synchonization barrier is used to avoid other cores to access devices before initialization completion.

D5) Idle thread initialization

In this step, each core creates and initializes its private idle thread descriptor.

D6) File system initialization

The CP0 in I/O cluster) initializes the file system.

A fourth synchonization barrier is used to avoid other cores to access file system before initialization completion.

D7) Scheduler activation

Finally, each core enables its private timer IRQ to activate its private scheduler, and jump to the idle thread code.

Attachments (4)

Download all attachments as: .zip