vystem/docs/shelter/memory/page.md

Page subsystem

Introduction

The Page subsystem is a part of the memory subsystem responsible for managing initial memory map analysis, physical pages bitmap, pages tables pool management, mapping and unmapping pages as well as memory allocations before Pez initialization and memory statistics. It's defined inside shelter/lib/include/memory/page.h and implemented inside shelter/lib/src/memory/page.c

Macros and types

The Page header defines a few usefuls macros like:

• the start and end of the kernel big allocations virtual range
• the size of a page
• the max memory count in bytes as well as the max physical pages count
• the standard size of a pages table pool pages range
• all the types of EFI memory type
• the null value of physical and virtual addresses
• the flags for pages table entry

The Page header also defines the following types:

• sh_page_MEMORY_TYPE: a wrapper of sh_uint32
• sh_page_PHYSICAL_ADDRESS and sh_page_VIRTUAL_ADDRESS: wrappers of sh_uint64

Structures

The Page header defines the following structure:

• sh_page_MEMORY_MAP_ENTRY and sh_page_MEMORY_MAP_HEADER: internal structures used for memory map analysis
• sh_page_PAGE_TABLE_POOL: structure for pages table pool, containing the physical and virtual address of a pages table pool as well as the internal bitmap for the internal allocator
• sh_page_MEM_STATS: a structure filled by a specific function to get statistics on physical memory using physical pages bitmap

Pages table pool

It's recommended to read pages table pool documentation before reading this.

The Page subsystem provide the following functions regarding PTP management:

• SH_STATUS sh_page_load_boot_ptp_va(sh_page_VIRTUAL_ADDRESS pt_pool_va): load the virtual address of the boot PTP provided by bootloader. This VA is the address of the root PML4, not the address of the PTP structure. Should only be used once
• sh_page_VIRTUAL_ADDRESS sh_page_get_boot_ptp_va(): return the pointer to root PM4 stored inside boot PTP
• SH_STATUS sh_page_init_ptp(sh_page_PHYSICAL_ADDRESS ptp_pa,sh_page_VIRTUAL_ADDRESS ptp_va,sh_uint64 initial_fill_level,sh_page_PAGE_TABLE_POOL *page_table_pool): initialize the boot PTP structure with the fill level of the PTP, provided by bootloader. Doesn't allocate pages for a new PTP but it use the pages already allocated for the boot PTP. Should only be used once for boot PTP
• SH_STATUS sh_page_dump_ptp_bitmap(sh_page_PAGE_TABLE_POOL *ptp): dump the PTP bitmap on the log output, intended for debug purposes
• sh_page_PHYSICAL_ADDRESS sh_page_ptp_alloc_one_page(sh_page_PAGE_TABLE_POOL *pt_pool): allocate one page from the PTP internal bitmap allocator
• static inline sh_uint64 *sh_page_ptp_pa_to_va(sh_page_PAGE_TABLE_POOL *ptp,sh_uint64 pa): convert a PA from the PTP pages range to a usable VA using the current PTP
• SH_STATUS sh_page_ptp_va_to_pa(sh_page_PAGE_TABLE_POOL *ptp,sh_page_VIRTUAL_ADDRESS va,sh_page_PHYSICAL_ADDRESS *pa): return equivalent physical address by searching inside provided PTP

Memory map analysis

The Page subsystem is responsible for the initial memory map analysis. It provide the following functions:

• SH_STATUS sh_page_copy_memory_map(): copy the memory map from her original VA to a dedicated buffer
• SH_STATUS sh_page_check_memory_map(): check for memory map signature and detect eventual buffers overflow
• SH_STATUS sh_page_analyse_memory_map(sh_page_PAGE_TABLE_POOL *ptp): analyse memory map, initialize physical pages bitmap by allocating into a free area of the physical memory layout. This allow for dynamic sizing for physical pages bitmap.

All three functions should be run in order and only once.

Physical pages bitmap

In order to ensure a reliable source of thruth and early allocations, the Page subsystem manage a physical pages bitmap where one bit on mean one page occupied. The Page subsystem provides the following functions regarding physical pages bitmap:

• SH_STATUS sh_page_set_pages_range_bitmap(sh_uint8 *bitmap,sh_uint64 page_count_in_bitmap,sh_uint64 page_index,sh_uint64 page_count,sh_bool state): allow for manipulation of any bitmap, but should only be used for physical pages bitmap
• static inline sh_bool sh_page_is_allocated(sh_uint8 *bitmap,sh_uint64 page_index): return the status of a specific page inside the bitmap
• sh_page_VIRTUAL_ADDRESS sh_page_get_physical_bitmap_ptr(): return physical pages bitmap pointer

Pages mapping and unmapping

The Page subsystem provide the following functions regarding pages mapping:

• SH_STATUS sh_page_map_one_page_ptp(sh_page_PAGE_TABLE_POOL *ptp,sh_page_VIRTUAL_ADDRESS va,sh_page_PHYSICAL_ADDRESS pa,sh_uint64 flags): map one page inside the pages table contained in the provided PTP
• SH_STATUS sh_page_is_va_mapped_ptp(sh_page_PAGE_TABLE_POOL *ptp,sh_page_VIRTUAL_ADDRESS va): return SH_STATUS_VA_NOT_MAPPED or SH_STATUS_VA_MAPPED, which aren't errors, according to page mapping state
• SH_STATUS sh_page_is_va_range_mapped_ptp(sh_page_PAGE_TABLE_POOL *ptp,sh_page_VIRTUAL_ADDRESS va,sh_uint64 size_bytes): return SH_STATUS_VA_NOT_MAPPED, SH_STATUS_VA_FULLY_MAPPED or SH_STATUS_VA_PARTIALLY_MAPPED according to pages range mapping state
• SH_STATUS sh_page_map_contiguous_pages_range_ptp(sh_page_PAGE_TABLE_POOL *ptp,sh_page_VIRTUAL_ADDRESS va,sh_page_PHYSICAL_ADDRESS pa,sh_uint64 flags,sh_uint64 size_bytes): map a pages range that has to be contiguous virtually and physically. VAs availability is checked, not physicals pages availability
• SH_STATUS sh_page_unmap_one_page_ptp(sh_page_PAGE_TABLE_POOL *ptp,sh_page_VIRTUAL_ADDRESS va): unmap one page from the pages table contained inside provided PTP
• SH_STATUS sh_page_unmap_contiguous_pages_range_ptp(sh_page_PAGE_TABLE_POOL *ptp,sh_page_VIRTUAL_ADDRESS va,sh_uint64 size_bytes): unmap a pages range from the pages table inside provided PTP. Both physical and virtual ranges have to be contiguous. Virtual pages range is checked, not physical ranges occupation

Basic memory allocations

The Page subsystem implement basic memory allocation, available before Pez initialization, using a first fit algorithm. The Page subsystem provides the following functions regarding basic memory allocations:

• sh_uint64 sh_page_get_one_page_na(): return the index of the first available page inside physical pages bitmap
• SH_STATUS sh_page_search_available_va_range(sh_page_PAGE_TABLE_POOL *ptp,sh_page_VIRTUAL_ADDRESS range_base,sh_page_VIRTUAL_ADDRESS range_size_bytes,sh_uint64 size_bytes,sh_page_VIRTUAL_ADDRESS *address_found): search a free region inside PTP using provided size and search area boundaries provided
• SH_STATUS sh_page_search_physical_contiguous_block_na(sh_uint64 pages_needed,sh_page_PHYSICAL_ADDRESS *pa): search a free region of provided size inside physical pages bitmap
• SH_STATUS sh_page_alloc_contiguous(sh_page_PAGE_TABLE_POOL *ptp,sh_uint64 size_bytes,sh_page_VIRTUAL_ADDRESS* va): allocate a specified amount of pages and return the start VA of the allocated region
• SH_STATUS sh_page_alloc_contiguous_extended(sh_page_PAGE_TABLE_POOL *ptp,sh_uint64 size_bytes,sh_page_VIRTUAL_ADDRESS* va,DEFAULT sh_uint64 flags,DEFAULT sh_page_VIRTUAL_ADDRESS va_range_start,DEFAULT sh_uint64 va_range_size_bytes): allow for extensive allocations parameters for selecting mapping flags and VAs search range
• SH_STATUS sh_page_unalloc_one_page(sh_page_PAGE_TABLE_POOL *ptp,sh_page_VIRTUAL_ADDRESS va): free one page. Check if page is mapped before hand. PA is calculated from searching into PTP
• SH_STATUS sh_page_unalloc_contiguous(sh_page_PAGE_TABLE_POOL *ptp,sh_page_VIRTUAL_ADDRESS va,sh_uint64 size_bytes): free a contiguous memory region. Check if the entire region is allocated before hand

This role of memory allocations is lost once Pez is initialized.

Memory statistics

The Page subsystem provide the following functions regarding memory statistics:

• sh_uint64 sh_page_get_physical_memory_amount_pages(): return amount of physical memory installed in pages
• sh_uint64 sh_page_get_physical_memory_amount_bytes(): return amount of physical memory installed in bytes
• SH_STATUS sh_page_get_memory_stats(sh_page_MEM_STATS *mem_stats): provide a extensive amount of statistics on physical memory.

For a more human-readable output, the function sh_log_mem_stats can be used.