CMakeLists Proposal

tools/projmgr/docs/CMakeListsProposal/CMakeListsProposal.md

@@ -0,0 +1,122 @@
+# Proposal: Solutions CMakeLists Generation
+
+In csolution it is possible to describe several projects and build-types/target-types resulting in several contexts and several build artifacts which may have inter-dependencies. To replace the `CPRJ` with `cbuild.yml` format and re-implement the CMakeLists.txt generation the following topics are being investigated:
+
+- generating CMakeLists for inter-dependent multi-context solutions
+- handling multiple toolchains in the same solution
+- modular generation of CMake targets allowing its integration with other CMake based projects and/or build plates
+- integration of existing images (that are i.e. generated using CMake)​
+- pre/post-build steps
+
+## 1. Overview
+
+The following diagram gives an overview of the current proposal considering all elements needed or recommended for reaching the above enumerated goals. A practical example addressing some of these concepts can be found in the [example](example/README.md) folder.
+
+![csolution-cmake1](images/csolution-cmake1.drawio.svg)
+
+## 2. Single vs multiple CMake Projects
+
+One common approach of describing related CMake projects, with or without inter-dependencies, is adding them to an entry point `CMakeLists.txt` using the function [`add_subdirectory`](https://cmake.org/cmake/help/latest/command/add_subdirectory.html):
+
+![csolution-cmake2](images/csolution-cmake2.drawio.svg)
+
+All CMake targets are visible and it is easy to set dependencies using [`add_dependencies`](https://cmake.org/cmake/help/latest/command/add_dependencies.html).
+However it has the following drawbacks:
+- Only one toolchain per language is supported.
+- The `configure` step is common to all targets, leading to the generation of a single [`compile_commands.json`](https://cmake.org/cmake/help/latest/variable/CMAKE_EXPORT_COMPILE_COMMANDS.html) for all subprojects.
+- It makes difficult the integration of an external stand-alone project in the same solution.
+
+
+To overcome such limitations it would be advisable to use the [`ExternalProject`](https://cmake.org/cmake/help/latest/module/ExternalProject.html#externalproject) functions:
+
+![csolution-cmake3](images/csolution-cmake3.drawio.svg)
+
+[Example](example/tmp/CMakeLists.txt#L70): SuperProject CMakeLists orchestrating contexts as external projects.
+
+This allows to run individual steps such as `configure` and `build` independently, enabling the use of a different toolchain for each context as well as the generation of a compilation database at context level. The steps can be managed as CMake targets via [`ExternalProject_Add_StepTargets`](https://cmake.org/cmake/help/latest/module/ExternalProject.html#command:externalproject_add_steptargets) as well as [`ExternalProject_Add_StepDependencies`](https://cmake.org/cmake/help/latest/module/ExternalProject.html#command:externalproject_add_stepdependencies).
+
+A possible disadvantage of this pattern is the superfluous overhead introduced by the CMake Compiler Detection phase when several contexts use the same toolchain. A custom step to optionally run the compiler detection just once per toolchain could be developed to overcome this drawback.
+
+## 3. Generating Portable CMakeLists
+
+Toolchain configuration data and CMakeLists could be generated without explicit absolute paths nor references to the CMSIS-Toolbox installation files, making them portable.
+For reaching this goal the destination platform must be properly configured: the environment variable `CMSIS_PACK_ROOT` must be set, the CMSIS-Pack repository must be initialized, the needed packs downloaded and installed, the needed toolchains should be installed and properly registered via environment variables `<name>_TOOLCHAIN_<major>_<minor>_<patch>`.
+
+## 4. Simplifying Toolchain Configuration
+
+The experience with the current Build Manager has shown communalities among the different embedded toolchains are frequent while newer versions usually bring only small deltas. Also a better machine-readable format such as YML/JSON would be preferred over *.cmake files allowing tools such as the Project Manager to easily read information that is currently only available at the CMake running phase.
+For this reason the suggestion here is to keep the common functions in *.cmake templates and shift the toolchain specific data into YML files. For example this could be a snippet of a hypothetical `AC6.yml` file:
+```yml
+toolchain-options:
+  cpu:
+    - for-compiler: AC6@>=6.6
+      Cortex-M3: -mcpu=Cortex-M3
+    - for-compiler: AC6@>=6.18
+      Cortex-M85: -mcpu=Cortex-M85
+```
+
+## 5. Using CMakeLists templates
+
+For the customization of the generated `CMakeLists.txt` at solution and/or at context level in a persistent way, avoiding overwriting when regenerating them, a possible approach is to use customizable templates.
+In a similar way the toolchain configuration *.cmake files could also benefit from customizable templates.
+
+## 6. Generating decoupled CMSIS components build information
+
+CMake targets describing components build information and user files build information could be placed in modules separately from the toolchain specific handling, the so-called build plate, enabling their use from different tools and build systems.
+
+[Example](example/tmp/project/ARMCM3/App/CMakeLists.txt#L25): [`groups.cmake`](example/tmp/project/ARMCM3/App/groups.cmake) and [`components.cmake`](example/tmp/project/ARMCM3/App/components.cmake) are generated/placed separately.
+
+## 7. Pre/post-build steps
+
+A build-run is inherently related to a context. In addition to solution level pre/post-build steps that run before/after all build-runs, individual context pre/post-build steps should be also accepted. Rather then placing `execute` nodes under `projects`, it sounds natural to specify them under the `context-set` node (TBD). 
+Input and output files should be also considered for correctly triggering pre/post-build steps, unless they are intended for running always unconditionally. The `execute` node alongside a `file` has an obvious implicit output and can be omitted.
+
+Examples of possible CMakeLists implementation:
+- [Example 1](example/tmp/CMakeLists.txt#L116): Solution level pre-build step, run always.
+- [Example 2](example/tmp/CMakeLists.txt#L126): Solution level post-build step, depends on all contexts build artifacts.
+- [Example 3](example/tmp/CMakeLists.txt#L96): Context level pre-build step, run always.
+- [Example 4](example/tmp/CMakeLists.txt#L101): Context level post-build step, depends on context build artifact.
+
+## 8. CMake hooks - extra use cases
+
+### 8.1 Add generic CMake based library
+
+A generic CMake based library does not bring a build plate, i.e. it does not have crossplatform settings for building it with any embedded toolchain in particular. It is expected to have a parent CMake project that configures the crossplatform environment.
+In the current proposal it can be added via custom templates or by extending the csolution specification to accept external projects.
+
+![csolution-cmake4](images/csolution-cmake4.drawio.svg)
+
+See [Using CMakeLists templates](#5-using-cmakelists-templates).
+
+### 8.2 Add stand-alone CMake based project
+
+Stand-alone CMake based projects are fully configured and don't need a build plate. Also in such cases it is possible to integrate it via custom templates or by extending the csolution specification.
+
+![csolution-cmake5](images/csolution-cmake5.drawio.svg)
+
+See [Using CMakeLists templates](#5-using-cmakelists-templates).
+
+### 8.3 Add CMSIS components into stand-alone CMake based project
+
+In this use case CMSIS Components are integrated into existing stand-alone CMake base projects.
+The user's CMSIS components selection/configuration needs to be translated in csolution yml files that are then processed by the Project Manager, generating cbuild.yml files and finally the CMSIS generic build info is added to the stand-alone project.
+A proof-of-concept for such CMake Module has been developed:
+[CMSIS-Pack-Utils](https://github.com/brondani/cmsis-pack-utils). 
+
+![csolution-cmake6](images/csolution-cmake6.drawio.svg)
+
+See [Generating decoupled CMSIS components build information](#6-generating-decoupled-cmsis-components-build-information).
+
+## 9. Suggestions for optimizations
+
+- The [`CMake binary directory`](https://cmake.org/cmake/help/latest/variable/CMAKE_BINARY_DIR.html) of each context could be named according to its index (ordinal number) instead of its full context name and placed right after the intermediate base directory. This strategy would lead to the shortest possible object file paths.
+
+## 10. Specification needs
+
+- The intermediate and output directory customization [`output-dirs`](https://github.com/Open-CMSIS-Pack/cmsis-toolbox/blob/main/docs/YML-Input-Format.md#output-dirs) is targeted at context level. In particular the `intdir` would need to be amended: in the multi-context scenario a solution level intermediate directory is needed to store the top-level `CMakeLists.txt`.
+- The `projects` references should accept external CMake based libraries and stand-alone projects.
+- The `execute` node should accept input and output files assignment.
+
+## 11. Test and development needs
+
+A test plan for the migration of integration test cases should be developed before the implementation of the new context-level CMakeLists, enabling its test-driven development.

tools/projmgr/docs/CMakeListsProposal/example/AC6/header.h

@@ -0,0 +1,4 @@
+
+#define HEADER_AC6
+
+int lib_ac6 (void);

tools/projmgr/docs/CMakeListsProposal/example/AC6/lib.c

@@ -0,0 +1,6 @@
+
+#include "header.h"
+
+int lib_ac6 (void) {
+  return 1;
+}

tools/projmgr/docs/CMakeListsProposal/example/GCC/header.h

@@ -0,0 +1,4 @@
+
+#define HEADER_GCC
+
+int lib_gcc (void);

tools/projmgr/docs/CMakeListsProposal/example/GCC/lib.c

@@ -0,0 +1,6 @@
+
+#include "header.h"
+
+int lib_gcc (void) {
+  return 1;
+}

tools/projmgr/docs/CMakeListsProposal/example/README.md

@@ -0,0 +1,61 @@
+# Example: Multiple contexts and toolchains CMakeLists
+
+This solution example demonstrates the use of a SuperProject CMakeLists orchestrating the build of inter-dependent contexts using multiple toolchains.
+
+The yml files are present in the `original` directory only as reference, they are not processed in this demo.
+The solution contains 3 contexts originated by 3 build types:
+- `AC6`: library compiled by AC6
+- `GCC`: library compiled by GCC
+- `App`: application using the previous libraries, compiled and linked by AC6
+
+The proposed SuperProject CMakeLists is in the `tmp` directory and the context-level CMakeLists are in the folders:
+- `tmp/project/ARMCM3/AC6`
+- `tmp/project/ARMCM3/GCC`
+- `tmp/project/ARMCM3/App`
+
+The ouput folders are configured respectively:
+- `out/project/ARMCM3/AC6`
+- `out/project/ARMCM3/GCC`
+- `out/project/ARMCM3/App`
+
+While the CMake objects are placed respectively:
+- `tmp/1`
+- `tmp/2`
+- `tmp/3`
+
+## Prerequisites
+
+Build system:
+- `CMake`
+- `Ninja`
+
+The following toolchains must be previously installed and registered via their [environment variables](https://github.com/Open-CMSIS-Pack/cmsis-toolbox/blob/main/docs/installation.md#environment-variables):
+- `AC6_TOOLCHAIN_6_19_0`
+- `GCC_TOOLCHAIN_12_2_1`
+
+The environment variable `CMSIS_PACK_ROOT` must be set, the CMSIS-Pack repository must be initialized and the following pack must be installed:
+- `ARM::CMSIS@5.9.0`
+
+## How to use
+
+On the `example` directory run CMake configure step:
+```
+cmake -G Ninja -S tmp -B tmp
+```
+
+Build all artifacts:
+```
+cmake --build tmp
+```
+
+Build artifacts of a given context:
+```
+cmake --build tmp --target <context>
+cmake --build tmp --target project.App+ARMCM3
+```
+
+Put the compilation database in the `out` directory of a given context:
+```
+cmake --build tmp --target <context>-database
+cmake --build tmp --target project.App+ARMCM3-database
+```

tools/projmgr/docs/CMakeListsProposal/example/RTE/Device/ARMCM3/ARMCM3_ac6.sct

@@ -0,0 +1,76 @@
+#! armclang -E --target=arm-arm-none-eabi -mcpu=cortex-m3 -xc
+; command above MUST be in first line (no comment above!)
+
+/*
+;-------- <<< Use Configuration Wizard in Context Menu >>> -------------------
+*/
+
+/*--------------------- Flash Configuration ----------------------------------
+; <h> Flash Configuration
+;   <o0> Flash Base Address <0x0-0xFFFFFFFF:8>
+;   <o1> Flash Size (in Bytes) <0x0-0xFFFFFFFF:8>
+; </h>
+ *----------------------------------------------------------------------------*/
+#define __ROM_BASE      0x00000000
+#define __ROM_SIZE      0x00080000
+
+/*--------------------- Embedded RAM Configuration ---------------------------
+; <h> RAM Configuration
+;   <o0> RAM Base Address    <0x0-0xFFFFFFFF:8>
+;   <o1> RAM Size (in Bytes) <0x0-0xFFFFFFFF:8>
+; </h>
+ *----------------------------------------------------------------------------*/
+#define __RAM_BASE      0x20000000
+#define __RAM_SIZE      0x00040000
+
+/*--------------------- Stack / Heap Configuration ---------------------------
+; <h> Stack / Heap Configuration
+;   <o0> Stack Size (in Bytes) <0x0-0xFFFFFFFF:8>
+;   <o1> Heap Size (in Bytes) <0x0-0xFFFFFFFF:8>
+; </h>
+ *----------------------------------------------------------------------------*/
+#define __STACK_SIZE    0x00000200
+#define __HEAP_SIZE     0x00000C00
+
+/*
+;------------- <<< end of configuration section >>> ---------------------------
+*/
+
+
+/*----------------------------------------------------------------------------
+  User Stack & Heap boundary definition
+ *----------------------------------------------------------------------------*/
+#define __STACK_TOP    (__RAM_BASE + __RAM_SIZE)    /* starts at end of RAM */
+#define __HEAP_BASE    (AlignExpr(+0, 8))           /* starts after RW_RAM section, 8 byte aligned */
+
+
+/*----------------------------------------------------------------------------
+  Scatter File Definitions definition
+ *----------------------------------------------------------------------------*/
+#define __RO_BASE       __ROM_BASE
+#define __RO_SIZE       __ROM_SIZE
+
+#define __RW_BASE       __RAM_BASE
+#define __RW_SIZE      (__RAM_SIZE - __STACK_SIZE - __HEAP_SIZE)
+
+
+LR_ROM __RO_BASE __RO_SIZE  {                       ; load region size_region
+  ER_ROM __RO_BASE __RO_SIZE  {                     ; load address = execution address
+   *.o (RESET, +First)
+   *(InRoot$$Sections)
+   .ANY (+RO)
+   .ANY (+XO)
+  }
+
+  RW_RAM __RW_BASE __RW_SIZE  {                     ; RW data
+   .ANY (+RW +ZI)
+  }
+
+#if __HEAP_SIZE > 0
+  ARM_LIB_HEAP  __HEAP_BASE EMPTY  __HEAP_SIZE  {   ; Reserve empty region for heap
+  }
+#endif
+
+  ARM_LIB_STACK __STACK_TOP EMPTY -__STACK_SIZE {   ; Reserve empty region for stack
+  }
+}

tools/projmgr/docs/CMakeListsProposal/example/RTE/Device/ARMCM3/ARMCM3_ac6.sct.base@1.0.0

@@ -0,0 +1,76 @@
+#! armclang -E --target=arm-arm-none-eabi -mcpu=cortex-m3 -xc
+; command above MUST be in first line (no comment above!)
+
+/*
+;-------- <<< Use Configuration Wizard in Context Menu >>> -------------------
+*/
+
+/*--------------------- Flash Configuration ----------------------------------
+; <h> Flash Configuration
+;   <o0> Flash Base Address <0x0-0xFFFFFFFF:8>
+;   <o1> Flash Size (in Bytes) <0x0-0xFFFFFFFF:8>
+; </h>
+ *----------------------------------------------------------------------------*/
+#define __ROM_BASE      0x00000000
+#define __ROM_SIZE      0x00080000
+
+/*--------------------- Embedded RAM Configuration ---------------------------
+; <h> RAM Configuration
+;   <o0> RAM Base Address    <0x0-0xFFFFFFFF:8>
+;   <o1> RAM Size (in Bytes) <0x0-0xFFFFFFFF:8>
+; </h>
+ *----------------------------------------------------------------------------*/
+#define __RAM_BASE      0x20000000
+#define __RAM_SIZE      0x00040000
+
+/*--------------------- Stack / Heap Configuration ---------------------------
+; <h> Stack / Heap Configuration
+;   <o0> Stack Size (in Bytes) <0x0-0xFFFFFFFF:8>
+;   <o1> Heap Size (in Bytes) <0x0-0xFFFFFFFF:8>
+; </h>
+ *----------------------------------------------------------------------------*/
+#define __STACK_SIZE    0x00000200
+#define __HEAP_SIZE     0x00000C00
+
+/*
+;------------- <<< end of configuration section >>> ---------------------------
+*/
+
+
+/*----------------------------------------------------------------------------
+  User Stack & Heap boundary definition
+ *----------------------------------------------------------------------------*/
+#define __STACK_TOP    (__RAM_BASE + __RAM_SIZE)    /* starts at end of RAM */
+#define __HEAP_BASE    (AlignExpr(+0, 8))           /* starts after RW_RAM section, 8 byte aligned */
+
+
+/*----------------------------------------------------------------------------
+  Scatter File Definitions definition
+ *----------------------------------------------------------------------------*/
+#define __RO_BASE       __ROM_BASE
+#define __RO_SIZE       __ROM_SIZE
+
+#define __RW_BASE       __RAM_BASE
+#define __RW_SIZE      (__RAM_SIZE - __STACK_SIZE - __HEAP_SIZE)
+
+
+LR_ROM __RO_BASE __RO_SIZE  {                       ; load region size_region
+  ER_ROM __RO_BASE __RO_SIZE  {                     ; load address = execution address
+   *.o (RESET, +First)
+   *(InRoot$$Sections)
+   .ANY (+RO)
+   .ANY (+XO)
+  }
+
+  RW_RAM __RW_BASE __RW_SIZE  {                     ; RW data
+   .ANY (+RW +ZI)
+  }
+
+#if __HEAP_SIZE > 0
+  ARM_LIB_HEAP  __HEAP_BASE EMPTY  __HEAP_SIZE  {   ; Reserve empty region for heap
+  }
+#endif
+
+  ARM_LIB_STACK __STACK_TOP EMPTY -__STACK_SIZE {   ; Reserve empty region for stack
+  }
+}