Vystem 0.2

docs/vybuild/cache.md

@@ -0,0 +1,14 @@
+# Cache
+
+## Introduction
+
+VyBuild integrate its own cache, stored in the `.vybuild_cache` folder in the root of the project folder, identified using `getcwd` at the start of the build pipeline execution. It is recommanded to put this folder in your `.gitignore` file if you use Git, or any other equivalent source control software.
+
+## Smart cache
+
+To ensure that the cache only contains what is actually needed for efficient compilation without taking too much storage, VyBuild integrate a unique technology called smart cache. The smart cache is enabled by default and ensure that the cache remain reasonably small on disk.
+
+Instead of relying on metadata like file dates or complex dependency graphs, the smart cache acts at the root of the root of the cache subsystem, and it works even on custom build artifacts stored in cache by the user. It is built upon three simple rules:
+- any new cache object that has been created during this build pipeline execution remains in cache
+- any cache object that was already in the cache before this build pipeline execution, and that have been used/extracted from the cache during this build pipeline execution remains in cache
+- any cache object that was already in the cache before this build pipeline execution, and that hasn't been used/extracted from the cache during this build pipeline execution will be deleted at the end of this build pipeline execution

docs/vybuild/index.md

@@ -0,0 +1,30 @@
+# VyBuild Docs
+
+## Introduction
+
+VyBuild is Vystem build system. It has been designed to considerably reduce the need for external tools and the pain of writing custom scripts. It is opinionated and optimized for OS development. While it was designed for Vystem, it can be used for others OS project, but feature such as Vyx/Vyld usage and VFTM aren't guaranteed to work. VyBuild is very integrated into the Vystem ecosystem and should not be considered complete for now.
+
+## Features
+
+VyBuild is capable of the following:
+- Compiling C, C++ and Assembly file, with custom arguments, compiler, assembler, and even linker in certain cases.
+- Automatically caching build artifacts and compilation objects files, with cached objects invalidation down to modification in the compiler/assembler binary and in C/CPP files included headers
+- Automatically purging the cache to keep only newly created objects and objects of previous build that have been used during a build, preventing garbage accumulation
+- Letting the user uses the cache how they want to store custom objects
+- Running command in indicated folder, running command with auto-faillure if command return a certain status code, show output of commands live, on failure or never, save command output to provided file
+- Giving the user complete control over how the build pipeline is parralelised, with flexible options at compilation, actions or modules granularity level
+- Compiling Vyx executable with an upgraded and more powerful integration of Vyld
+- Acting on the filesystem, like moving/copying files/folder, ensure folder existence or reset
+- Letting the user uses system variables (like `%rootcallerfolder%`) or define their own to use in almost any actions field, as well as providing variable on command line argument of VyBuild
+- Running differents actions or modules depending on differents condition, like folder existence, cache presence or variable value/existence
+- Create fully valid FAT32 images (only support short file names) and fully valid disk images with GPT tables, custom type/unique partitions GUID and importing partitions from file
+- Integrating Vystem FAT Trusted Manifest support for FAT32 images generation and interaction with `keygen`
+
+## Index
+
+This documentation is intended to be read in the following order:
+1) [Modules](modules.md)
+2) [Actions](actions.md)
+3) [Variables](variables.md)
+4) [Cache](cache.md)
+5) [Usage](usage.md)

docs/vybuild/modules.md

@@ -0,0 +1,46 @@
+# Modules
+
+## Introduction
+
+Modules are the backbone of VyBuild. They allow you to precisely describe the build process of your project. They are JSON files with a very strict but clear syntax.
+
+## Syntax
+
+Modules can be executed as root module (called from command line) or submodule (called from another module).
+A module is made of several fields indexed in a dictionnary. Here is an example:
+
+``` json
+{
+  "version":"0.1",
+  "root_folder":"...",
+  "run_context":"...",
+  "name":"...",
+  "global_uuid":[
+    ...
+  ],
+  "global_vars":{
+    ...
+  },
+  "actions":[
+    ...
+  ]
+}
+```
+
+### Mandatory fields
+
+Each module has 5 mandatory fields:
+- `version`: a string indicating the version of VyBuild JSON syntax. It must be set to the current version of the VyBuild JSON syntax, which is currently `0.1`
+- `root_folder`: a string indicating the working folder of this module, all relative paths must be accessible in this folder. You can use variables in this field (see [variables docs](variables.md)). The path indicated in this field should always be absolute for maximum reliability
+- `run_context`: a string indicating in which context this module can be executed. Possible values are:
+  - `both`: the module can be executed as root module and submodule
+  - `root`: the module can only be executed as root module
+  - `sub`: the module can only be executed as submodule
+- `name`: a string indicating the name that will be displayed in VyBuild call tree
+- `actions`: a list of actions (see [actions docs](actions.md)) to execute in the module
+
+### Optional fields for root module
+
+Modules called from the command line can include the following optional fields :
+- `global_uuid`: a list of string giving variables names that will be filled with random UUIDs v4 as each build. They can be used as regular variables
+- `global_vars`: a dictionnary of string to string entries, associating variables names to their contents. For more informations, see [variables docs](variables.md)

docs/vybuild/usage.md

@@ -0,0 +1,35 @@
+# Usage
+
+## Build
+
+In order to use VyBuild, you must compile it first. For this, use the `build.sh` script located in the `vybuild` folder. This script should only be used from the root folder of the Vystem project, just like this:
+
+``` bash
+vybuild/build.sh
+```
+
+The `vybuild` binary will be compiled inside the folder of same name.
+
+## Usage
+
+VyBuild can be used like this:
+
+``` bash
+vybuild/vybuild <path to root module> <build args>
+```
+
+The first argument must be the path to the root module. Arguments that comes after that will be sorted into two categories:
+- VyBuild arguments: if VyBuild recognize an argument specific to the build system, it will remove it from the exposed arguments
+- Build arguments: these arguments will be exposed to the build pipeline using system variables
+
+Here is the list of VyBuild arguments to this day:
+- `--purge-cache`: purge the cache before starting the build pipeline
+- `--disable-smart-cache`: disable the smart cache feature for this execution of the build pipeline
+
+## Licenses
+
+External libraries used inside VyBuild:
+- ELFIO, made by Serge Lamikhov-Center, sourced from [serge1/ELFIO](https://github.com/serge1/ELFIO), under the MIT license
+- json, made by Niels Lohmann, sourced from [nlohmann/json](https://github.com/nlohmann/json), under the MIT license
+
+Others libraries are used when loading the dynamic library `commoncrypto`, which can be found [here](../boottools/commoncrypto.md)

docs/vybuild/variables.md

@@ -0,0 +1,59 @@
+# Variables
+
+## Introduction
+
+Variables in VyBuild allow for rapid changes of the build pipeline parameters and can be used almost anywhere.
+
+## Overview
+
+In VyBuild, there are two types of variables:
+- user-defined variables (also called global variables): these are defined as pairs of string in the `global_vars` field of every root module. They are usables in all parts of the build pipeline without restriction.
+- systems variables: these are special variables defined by VyBuild containing specific values. Some are usable globally and some are only usables in some fields. For more information about these ones, see [actions docs](actions.md)
+
+## Using a global variable
+
+To define a global variable with a custom value, simply add a pair of string inside the `global_vars` field of the root module of your pipeline.
+
+To define a global variable with an automated UUID v4 that changes at each build, simply add a string inside the `global_uuid` field of the root module of your pipeline.
+
+You can use variables in almost every fields that are either a string or a list of strings. You know if a field support variables if there is a boolean field in the same action with the name `<field name>_variables`. These variables fields are mandatory and indicate if VyBuild will attempt to detect and replace variables in any field where the corresponding variable field is set to `true`.
+
+To use global variables in a field that support it, just put your variable name between `$`.
+
+Example with `SOME_VARIABLE` containing `somes informations`:
+``` json
+{
+  "field_variables":true,
+  "field":"hey$SOME_VARIABLE$hey"
+}
+```
+
+In this example, the value that VyBuild sees after replacing the variable is `heysomes informationshey`.
+
+Variable names are usually in uppercases letters to avoid any confusion with others parts of a field. This is to follow the same C macros style.
+
+### Specials global variables
+
+Some global variables are a way for some parts of VyBuild to let the user add features to them. Here is a exhaustive list of these special global variables
+- `__VYBUILD_COMPILER_HASH_`: any global variables starting with this prefix will be used by actions that compile C/CPP code to include optional information on the used toolchain. These actions will consider the content of these variables as already-formated commands that they can execute and include the output of these commands into the cache keys of each cached objects. That way, any change to the toolchain can be detected and trigger a recompilation
+
+## Using systems variables
+
+Systems variables are variables created by VyBuild, usable in certain contexts. This file will only cover systems variables that aren't specific to certains actions.
+
+Systems variables can be used in almost any fields with some limitations. When used, they are framed with `%`. Here is a complete list:
+
+Name | Content | Behaviour
+-----|---------|----------
+`%rootcallerfolder%` | The full path of the parent folder of the root module of every build pipeline | Usable in every action fields that support it. Can also be used in the `root_folder` field of every module but alone (can't be combined in a path for this field).
+`%filefolder%` | The absolute path of the directory containing the current module's JSON file | Can only be used alone (can't be combined in a path for this field) in the `root_folder` of every module.
+
+Command line arguments that aren't VyBuild parameters will be exposed as systems variables under the index of the argument in the command line.
+
+Example: 
+
+``` bash
+vybuild <path to root module> img
+```
+
+In this example, the system variable `1` will be created by VyBuild with the value `img`, and can be used in any action fields that support variables using the syntax `%1%`.