♻️ Refactor Packaging and part of CI (#171)

## Description This PR brings a major refactoring for the overall Python packaging as well as parts of the CI employed in the project. ### Packaging We are (finally) moving away from `setuptools` and towards modern `scikit-build-core` (https://scikit-build-core.readthedocs.io/) This brings many advantages such as - truly editable installs that also include the C++ part; this implies incredibly fast rebuilds that can be further improved by installing ccache. - completely eliminates the `setup.py` (and the `MANIFEST.in`) file and all custom build code that was required previously. As part of this change, we adopt the established best practice of a src-layout for the Python package. This implies that the Python package no longer lives in the main package directory, but is moved to `src/mqt/...`. This avoids several pitfalls when working with Python packages and makes the overall handling way easier. This also means there is no direct build-time dependency on cmake and ninja anymore as this is automatically handled by scikit-build-core on demand. As part of this rewrite, several `nox` sessions are also updated. ## CI Parts of the CI have been modernized with the latest best practises. Unfortunately, we cannot yet adopt the reusable workflows from mqt-core due to the flint dependency in this project. This will come as a follow-up ## Checklist:  - [x] The pull request only contains commits that are related to it. - [ ] I have added appropriate tests and documentation. - [ ] I have made sure that all CI jobs on GitHub pass. - [ ] The pull request introduces no new warnings and follows the project's style guidelines.
cda-tum · Sep 29, 2023 · ed3bcaf · ed3bcaf
2 parents 4ddca85 + b410a9c
commit ed3bcaf
Show file tree

Hide file tree

Showing 50 changed files with 875 additions and 678 deletions.
diff --git a/.cmake-format.yaml b/.cmake-format.yaml
@@ -1,3 +1,4 @@
 format:
+  line_width: 100
   keyword_case: "upper"
   autosort: true
diff --git a/.github/ISSUE_TEMPLATE/bug-report.yml b/.github/ISSUE_TEMPLATE/bug-report.yml
@@ -1,46 +1,28 @@
 name: 🐛 Bug report
 description: Something is not working correctly.
 title: "🐛 <title>"
-labels: ["bug"]
-assignees:
-  - lucasberent
 body:
   - type: markdown
     attributes:
       value: >-
         **Thank you for wanting to report a bug for this project!**
 
-        :warning:
-        Verify first that your issue is not [already reported on GitHub](https://github.com/cda-tum/qecc/issues).
+        ⚠
+        Verify first that your issue is not [already reported on GitHub](https://github.com/cda-tum/mqt-qecc/search?q=is%3Aissue&type=issues).
 
-  - type: markdown
-    attributes:
-      value: >-
-        **Environment**
-  - type: input
-    attributes:
-      label: mqt.qecc version
-      placeholder: For example, mqt.qecc===2.0.0
-    validations:
-      required: true
-  - type: input
+        If you have general questions, please consider [starting a discussion](https://github.com/cda-tum/mqt-qecc/discussions).
+  - type: textarea
     attributes:
-      label: OS
-      placeholder: For example, Ubuntu 22.04, macOS Big Sur, Windows etc.
+      label: Environment information
+      description: >-
+        Please provide information about your environment. For example, OS, C++ compiler, mqt.qecc version etc.
+      placeholder: |
+        - OS:
+        - C++ compiler:
+        - mqt.core version:
+        - Additional environment information:
     validations:
       required: true
-  - type: input
-    attributes:
-      label: Python version
-      placeholder: For example, Python 3.10
-  - type: input
-    attributes:
-      label: C++ compiler
-      placeholder: For example, gcc-10
-  - type: textarea
-    attributes:
-      label: Additional environment information
-      description: Feel free to add more information about your environment here.
   - type: textarea
     attributes:
       label: Description

diff --git a/.github/ISSUE_TEMPLATE/feature-request.yml b/.github/ISSUE_TEMPLATE/feature-request.yml
@@ -1,17 +1,14 @@
 name: ✨ Feature request
 description: Suggest an idea
 title: "✨ <title>"
-labels: ["enhancement"]
-assignees:
-  - lucasberent
 body:
   - type: markdown
     attributes:
       value: >
         **Thank you for wanting to suggest a feature for this project!**
 
-        :warning:
-        Verify first that your idea is not [already requested on GitHub](https://github.com/cda-tum/qecc/issues).
+        ⚠
+        Verify first that your idea is not [already requested on GitHub](https://github.com/cda-tum/qecc/search?q=is%3Aissue&type=issues).
 
   - type: textarea
     attributes:

diff --git a/.github/codecov.yml b/.github/codecov.yml
@@ -1,8 +1,8 @@
 ignore:
   - "extern/**/*"
-  - "mqt/**/*.cpp"
+  - "**/python"
   - "test/**/*"
-  - "mqt/qecc/cc_decoder/plots.py"
+  - "src/mqt/qecc/cc_decoder/plots.py"
 
 coverage:
   range: 60..90
@@ -13,10 +13,12 @@ coverage:
 
 flag_management:
   default_rules:
+    carryforward: true
     statuses:
       - type: project
         target: auto
         threshold: 0.5%
+        removed_code_behavior: adjust_base
       - type: patch
         target: 90%
         threshold: 1%
@@ -25,12 +27,15 @@ flag_management:
       paths:
         - "include"
         - "src"
+      after_n_builds: 1
     - name: python
       paths:
-        - "mqt/**/*.py"
+        - "src/mqt/**/*.py"
+      after_n_builds: 1
       statuses:
         - type: project
           threshold: 0.5%
+          removed_code_behavior: adjust_base
         - type: patch
           target: 95%
           threshold: 1%
@@ -41,6 +46,7 @@ parsers:
       conditional: no
       loop: no
 
-codecov:
-  notify:
-    after_n_builds: 2
+comment:
+  layout: "reach, diff, flags, files"
+  require_changes: true
+  show_carryforward_flags: true
diff --git a/.github/codeql-config.yml b/.github/codeql-config.yml
diff --git a/.github/dependabot.yml b/.github/dependabot.yml
@@ -2,30 +2,35 @@ version: 2
 updates:
   - package-ecosystem: "gitsubmodule"
     directory: "/"
+    groups:
+      submodules:
+        patterns:
+          - "*"
     schedule:
-      interval: "weekly"
-      day: "friday"
+      interval: "monthly"
       time: "06:00"
       timezone: "Europe/Vienna"
-    assignees:
-      - "lucasberent"
 
   - package-ecosystem: "github-actions"
     directory: "/"
+    groups:
+      github-actions:
+        patterns:
+          - "*"
     schedule:
       interval: "weekly"
       day: "friday"
       time: "06:00"
       timezone: "Europe/Vienna"
-    assignees:
-      - "lucasberent"
 
   - package-ecosystem: "pip"
     directory: "/"
+    groups:
+      python-dependencies:
+        patterns:
+          - "*"
     schedule:
       interval: "weekly"
       day: "friday"
       time: "06:00"
       timezone: "Europe/Vienna"
-    assignees:
-      - "lucasberent"
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
@@ -20,75 +20,88 @@ defaults:
     shell: bash
 
 jobs:
-  ubuntu-tests:
-    name: Tests ubuntu-latest
+  cpp-tests-ubuntu:
+    name: 🐧 Release
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
         with:
           submodules: recursive
-      - name: Setup ccache
-        uses: hendrikmuhs/ccache-action@v1.2
-        with:
-          key: "ubuntu"
       - name: Install flint
         run: sudo apt-get install libflint-dev
+      - name: Setup ccache
+        uses: Chocobo1/setup-ccache-action@v1
+        with:
+          prepend_symlinks_to_path: false
+          override_cache_key: c++-tests-ubuntu-latest
+      - name: Set up mold as linker
+        uses: rui314/setup-mold@v1
+      - name: Install Ninja
+        run: pipx install ninja
       - name: Configure CMake
-        run: cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug -DBUILD_QECC_TESTS=ON -DENABLE_COVERAGE=ON
+        run: cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
       - name: Build
-        run: cmake --build build
+        run: cmake --build build --config Release
       - name: Test
-        run: ctest -C Debug --output-on-failure --test-dir build --repeat until-pass:3 --timeout 300
-      - name: Upload coverage to Codecov
-        uses: codecov/codecov-action@v3.1.4
-        with:
-          fail_ci_if_error: true
-          flags: cpp
-          gcov: true
-          gcov_ignore: "extern/**/*"
-          token: ${{ secrets.CODECOV_TOKEN }}
+        run: ctest -C Release --output-on-failure --test-dir build --repeat until-pass:3 --timeout 600
 
-  macos-tests:
-    name: Tests macos-11
-    runs-on: macos-11
+  cpp-tests-macos:
+    name: 🍎 Release
+    runs-on: macos-latest
     steps:
       - uses: actions/checkout@v4
         with:
           submodules: recursive
+      - name: Install flint
+        run: brew install flint
       - name: Setup ccache
-        uses: hendrikmuhs/ccache-action@v1.2
+        uses: Chocobo1/setup-ccache-action@v1
         with:
-          key: "macOS"
+          prepend_symlinks_to_path: false
+          override_cache_key: c++-tests-macos-latest
+      - name: Install Ninja
+        run: pipx install ninja
+      - name: Configure CMake
+        run: cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
+      - name: Build
+        run: cmake --build build --config Release
+      - name: Test
+        run: ctest -C Release --output-on-failure --test-dir build --repeat until-pass:3 --timeout 600
+
+  coverage:
+    name: 📈 Coverage
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          submodules: recursive
       - name: Install flint
-        run: brew install flint
+        run: sudo apt-get install libflint-dev
+      - name: Setup ccache
+        uses: Chocobo1/setup-ccache-action@v1
+        with:
+          prepend_symlinks_to_path: false
+          override_cache_key: c++-coverage
+      - name: Set up mold as linker
+        uses: rui314/setup-mold@v1
+      - name: Install Ninja
+        run: pipx install ninja
       - name: Configure CMake
-        run: cmake -S . -B build -DBUILD_QECC_TESTS=ON
+        run: cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug -DENABLE_COVERAGE=ON
       - name: Build
-        run: cmake --build build
+        run: cmake --build build --config Debug
       - name: Test
-        run: ctest -C Release --output-on-failure --test-dir build --repeat until-pass:3 --timeout 300
-#  windows-tests: TODO
-#    name: Tests windows-2022
-#    runs-on: windows-2022
-#    defaults:
-#      run:
-#        shell: msys2 {0}
-#    steps:
-#      - uses: actions/checkout@v4
-#        with:
-#          submodules: recursive
-#      - uses: msys2/setup-msys2@v2
-#        with:
-#          install: mingw-w64-x86_64-cmake mingw-w64-x86_64-ninja mingw-w64-x86_64-clang mingw-w64-clang-x86_64-flint
-#      - name: Debug flint location
-#        run: pacman -Ql mingw-w64-clang-x86_64-flint
-#      - name: Configure CMake
-#        run: cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DBUILD_QECC_TESTS=ON -DBINDINGS=ON -DFLINT_ROOT=/clang64
-#        env:
-#          CC: clang
-#          CXX: clang++
-#      - name: Build
-#        run: cmake --build build --config Release --parallel 3
-#      - name: Test
-#        working-directory: build\Release\test
-#        run: ctest -C Release --output-on-failure
+        run: ctest -C Debug --output-on-failure --test-dir build --repeat until-pass:3 --timeout 600
+      - name: Upload coverage to Codecov
+        # Coverage uploads can be flaky, so we retry them a few times.
+        uses: Wandalen/wretry.action@v1.3.0
+        with:
+          action: codecov/codecov-action@v3
+          attempt_limit: 5
+          attempt_delay: 2000
+          with: |
+            fail_ci_if_error: true
+            flags: cpp
+            gcov: true
+            gcov_ignore: "extern/**/*"
+            token: ${{ secrets.token }}
diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml
@@ -3,6 +3,7 @@ name: "CodeQL"
 on:
   push:
   pull_request:
+  merge_group:
   schedule:
     - cron: "15 21 * * 6"
 
@@ -31,25 +32,47 @@ jobs:
         with:
           submodules: recursive
           fetch-depth: 0
-      - if: matrix.language == 'cpp'
-        name: Setup ccache
-        uses: hendrikmuhs/ccache-action@v1.2
+
+      - name: Setup ccache
+        uses: Chocobo1/setup-ccache-action@v1
         with:
-          key: "CodeQL"
+          prepend_symlinks_to_path: false
+          override_cache_key: codeql-${{ matrix.language }}
 
       - name: Install flint
         run: sudo apt-get install libflint-dev
 
+      - name: Set up mold as linker
+        uses: rui314/setup-mold@v1
+
+      - if: matrix.language == 'python'
+        name: Set up Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: "3.11"
+          cache: "pip"
+
+      - if: matrix.language == 'python'
+        name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip wheel setuptools
+          pip install scikit-build-core[pyproject] setuptools_scm pybind11
+          pip install --no-build-isolation -ve .
+          echo "CODEQL_PYTHON=$(which python)" >> $GITHUB_ENV
+
       # Initializes the CodeQL tools for scanning.
       - name: Initialize CodeQL
         uses: github/codeql-action/init@v2
         with:
           languages: ${{ matrix.language }}
-          config-file: .github/codeql-config.yml
+          setup-python-dependencies: false
+          config: |
+            queries:
+              - uses: security-and-quality
 
       - if: matrix.language == 'cpp'
         name: Configure CMake
-        run: cmake -S . -B build -DBINDINGS=ON -DBUILD_QECC_TESTS=ON
+        run: cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug
 
       - if: matrix.language == 'cpp'
         name: Build