vystem/docs/shelter/tab.md

Test-and-benchmark framework

Introduction

The Shelter kernel includes his own test-and-benchmark (TAB) framework to support the measurements of performances and reliability of differents subsystems. This file describe the logic behind all the tests included inside the Shelter kernel and the embarqued payload, not the common API base described in tests utilities or the benchmark results.

A payload is a file generated at compilation, mostly random, used for accurate testing in specific situations.

Slab allocators tests

These tests are designed to test the reliability and performances of the three main slab allocators behind the Pez subsystem: physical and virtual region objects slab allocators as well as the radix tree nodes slab allocators. They are defined inside shelter/lib/include/kernel/tests/test_slabs.h and implemented inside shelter/lib/src/kernel/tests/test_slabs.c. This test doesn't require a payload.

They consist in allocating 10000 objects from these three slabs and deallocating them afterward. This allocate a few slabs for each allocator, meaning they will consume a few pages at the start but nothing too expensive for the systems. This also allow for quick allocations of the first few thousands objects of these allocators since the slabs are already allocated.

Radix tree subsystem tests

These tests are designed to test the reliability and performances of the radix tree subsystem. They are defined inside shelter/lib/include/kernel/tests/test_radix.h and implemented inside shelter/lib/src/kernel/tests/test_radix.c. They use the shelter/lib/src/kernel/tests/payloads/test_radix_payload.h.

The payload contain the following datas:

• test_keys: an array of 10000 random sh_uint64 serving as keys for the pairs
• test_values: an array of 10000 random sh_uint64 serving as values for the pairs
• test_search: an array of 10000 random sh_uint64 serving as keys for the lower bound keys for the backtracking algorithm

All tests are run on a 16-deep unique radix tree. The test are in the following order:

• Insertion: trying to insert 10000 pairs of keys-values defined inside test_keys and test_values
• Reading: trying to obtain the associated value of each of the 10000 pairs by providing the values of test_keys, checking if the correct value is returned by comparing to the values of test_values
• Searching: trying to search using the backtracking algorithm with lower bound key using the values of test_search
• Deleting: trying to delete all pairs using the keys contained inside test_keys

Pez subsystem tests

This test is designed to test the reliability and performances of the Pez subsystem. They are defined inside shelter/lib/include/kernel/tests/test_pez.h and implemented inside shelter/lib/src/kernel/tests/test_pez.c. They use the shelter/lib/src/kernel/tests/payloads/test_pez_payload.h.

This specific test has been designed to see how Pez react under high pressure. It's only apply it on Pez's physical plane, as the virtual plane uses the same logic.

The payload contain the following datas:

• test_pez_physical_size: an array of 2000 sh_uint64 serving as sizes in pages for the allocation. Half of them are 1. The other half is between 2 and 1000.
• test_pez_physical_alloc: an array of 4000 sh_uint64, serving as reference for when to allocate and when to deallocate each allocated region. These numbers range from 1 to 2000 and appear exactly 2 times for each.

The test work like this: we iterate on all the value of test_pez_physical_alloc. When we hit for the first time a number, we allocate the corresponding size inside test_pez_physical_size by using the allocation number as index. When we hit for the second time a number, we deallocate this allocation.

The test will separate single page allocations and frees from multi-pages allocations and frees when showing the results.

Malloc subsystem test

These tests are design to test the reliability and performances of the Pez subsystem. They are defined inside shelter/lib/include/kernel/tests/test_malloc.h and implemented inside shelter/lib/src/kernel/tests/test_malloc.c. They use the shelter/lib/src/kernel/tests/payloads/test_malloc_payload.h.

The payload contain the following datas:

• test_malloc_small_size: contain 10000 sizes for small allocations (under or equal to 1024 bytes)
• test_malloc_big_size: contain 1000 sizes in pages, ranging from 1 to 25, serving the same role as test_pez_physical_size
• test_malloc_big_alloc: contain 2000 values, serving the same role and generated like test_pez_physical_alloc

The malloc test combine two tests: 10000 allocations and deallocations of variously sized small objects using the generic slab allocators, and 1000 allocations and frees of pages on the heap, on the same logic of the Pez subsystem test, but adapted to the quantity used in this test.