Test merge (#172)

* Add basic task for generating apidocs

* Fix SPECTACTULAR_SETTINGS

- Some provided options were not correct

* Update .gitignore

* Fix for duplicated API path

- `/api/plugins/activate` routed to PluginActivate view
- Must be associated with a specific plugin ID

* By default, fail if warnings are raised

* Use GenericAPIView for GetAuthToken

* Use GenericAPIView for RolesDetail endpoint

* Refactor more endpoints to use GenericApiView

* More API cleanup

* Add extra type hints for exposed methods

* Update RoleDetails endpoint

- Specify serializer
- Use RetrieveAPI class type

* More type hints

* Export API docs as part of CI

* add more api views docs

* even more docs

* extend tests to api-version

* simplify serializer

* and more docs

* fix serializer

* added more API docs

* clean diff

* Added APISearch base

* Mention the API version bump in CONTRIBUTION.md (inventree#6436)

* Mention the API version bump in CONTRIBUTION.md

* Update CONTRIBUTING.md

---------

Co-authored-by: Oliver <oliver.henry.walters@gmail.com>

* do not assume you know the user
he might be anonymously creating the schema ;-)

* set empty serializer where no input is needed

* Fix for build.js (inventree#6437)

- Catch potential empty build_detail attribute

* Maintenance Mode Fix (inventree#6422)

* Custom maintenance mode backend

* Implement check and retries

* Update debug formatting

* Use dummy model for schema generation

* fix OpenAPI docs section

* Week number (inventree#6439)

* Support week number for batch generation

* Bug fix - hour is not minute

* Update docs

* only run if needed

* remove schema task

* Add version check

* pin version

* fix QC order

* fix assign

* refactor order

* optimize compile times

* fix assign

* onyl use install

* more fixing

* use full update cycle

* revert python change

* use api_version

* run py

* why?

* Fix for issue 6442 (inventree#6443)

- Add null option to status field for stock adjustment
- Prevent stock adjustments from setting status to OK

* clean up output

* only check for api version diff if api_version was not touched

* add schema check again

* Postgresql fix (inventree#6441)

* Assume maintenance mode is *on* if database is inaccessible

* Specify ash shell

* Update psycopg requirements

* Style fixes

* style fix - backends.py

* Update tasks.py (inventree#6446)

* Update tasks.py

- Exclude temporary settings when exporting data

* Remove duplicate code

* use seperate filter for API

* increment api_version

* Added push step

---------

Co-authored-by: Oliver Walters <oliver.henry.walters@gmail.com>
Co-authored-by: Miklós Márton <martonmiklosqdev@gmail.com>

.github/workflows/qc_checks.yaml

@@ -112,6 +112,72 @@ jobs:
           pip install linkcheckmd requests
           python -m linkcheckmd docs --recurse
 
+  apidoc:
+    name: Tests - API Documentation
+    runs-on: ubuntu-20.04
+    needs: paths-filter
+    if: needs.paths-filter.outputs.server == 'true'
+    env:
+      INVENTREE_DB_ENGINE: django.db.backends.sqlite3
+      INVENTREE_DB_NAME: ../inventree_unit_test_db.sqlite3
+      INVENTREE_ADMIN_USER: testuser
+      INVENTREE_ADMIN_PASSWORD: testpassword
+      INVENTREE_ADMIN_EMAIL: test@test.com
+      INVENTREE_PYTHON_TEST_SERVER: http://localhost:12345
+      INVENTREE_PYTHON_TEST_USERNAME: testuser
+      INVENTREE_PYTHON_TEST_PASSWORD: testpassword
+
+    steps:
+      - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # pin@v4.1.1
+      - uses: dorny/paths-filter@4512585405083f25c027a35db413c2b3b9006d50 # pin@v2.11.1
+        id: filter
+        with:
+          filters: |
+            api:
+              - 'InvenTree/InvenTree/api_version.py'
+      - name: Environment Setup
+        uses: ./.github/actions/setup
+        with:
+          apt-dependency: gettext poppler-utils
+          dev-install: true
+          update: true
+      - name: Export API Documentation
+        run: invoke apidoc --ignore-warnings
+      - name: Upload schema
+        uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # pin@v3.1.3
+        with:
+          name: api.yaml
+          path: api.yaml
+      - name: Download public schema
+        if: steps.filter.outputs.api == 'false'
+        run: |
+          pip install requests >/dev/null 2>&1
+          version="$(python3 ci/version_check.py only_version 2>&1)"
+          echo "Version: $version"
+          url="https://raw.githubusercontent.com/inventree/schema/main/export/${version}/api.yaml"
+          echo "URL: $url"
+          curl -s -o schema.yaml $url
+          echo "Downloaded schema.yaml"
+      - name: Check for differences in schemas
+        if: steps.filter.outputs.api == 'false'
+        run: |
+          diff --color -u api.yaml schema.yaml
+          diff -u api.yaml schema.yaml && echo "no difference in API schema " || echo "differences in API schema" && exit 2
+      - name: Check schema - including warnings
+        run: invoke apidoc
+        continue-on-error: true
+      - name: Push new schema if change & on master
+        if: github.ref == 'refs/heads/master' && steps.filter.outputs.api == 'true'
+        run: |
+          pip install requests >/dev/null 2>&1
+          version="$(python3 ci/version_check.py only_version 2>&1)"
+          echo "Version: $version"
+          url="https://api.github.com/repos/inventree/schema/contents/export/${version}/api.yaml"
+          echo "URL: $url"
+          content="$(cat api.yaml | base64)"
+          curl -L -X PUT -H "Accept: application/vnd.github+json" -H "Authorization: Bearer $GITHUB_TOKEN" -H "X-GitHub-Api-Version: 2022-11-28" $url -d '{"message":"Update API schema for ${version}","content":$content}'
+          echo "Uploaded api.yaml"
+
   python:
     name: Tests - inventree-python
     runs-on: ubuntu-20.04

.gitignore

@@ -70,6 +70,7 @@ secret_key.txt
 .idea/
 *.code-workspace
 .bash_history
+.DS_Store
 
 # https://github.com/github/gitignore/blob/main/Global/VisualStudioCode.gitignore
 .vscode/*
@@ -107,5 +108,8 @@ InvenTree/plugins/
 *.mo
 messages.ts
 
+# Generated API schema file
+api.yaml
+
 # web frontend (static files)
 InvenTree/web/static

CONTRIBUTING.md

@@ -79,6 +79,10 @@ The HEAD of the "stable" branch represents the latest stable release code.
 - When approved, the branch is merged back *into* stable, with an incremented PATCH number (e.g. 0.4.1 -> 0.4.2)
 - The bugfix *must* also be cherry picked into the *master* branch.
 
+## API versioning
+
+The [API version](https://github.com/inventree/InvenTree/blob/master/InvenTree/InvenTree/api_version.py) needs to be bumped every time when the API is changed.
+
 ## Environment
 ### Target version
 We are currently targeting:

Dockerfile

@@ -84,7 +84,7 @@ RUN if [ `apk --print-arch` = "armv7" ]; then \
 COPY tasks.py docker/gunicorn.conf.py docker/init.sh ./
 RUN chmod +x init.sh
 
-ENTRYPOINT ["/bin/sh", "./init.sh"]
+ENTRYPOINT ["/bin/ash", "./init.sh"]
 
 FROM inventree_base as prebuild
 

InvenTree/InvenTree/api.py

@@ -1,5 +1,7 @@
 """Main JSON interface views."""
 
+import sys
+
 from django.conf import settings
 from django.db import transaction
 from django.http import JsonResponse
@@ -8,6 +10,7 @@
 from django_q.models import OrmQ
 from drf_spectacular.utils import OpenApiResponse, extend_schema
 from rest_framework import permissions, serializers
+from rest_framework.generics import GenericAPIView
 from rest_framework.response import Response
 from rest_framework.serializers import ValidationError
 from rest_framework.views import APIView
@@ -18,6 +21,7 @@
 from InvenTree.mixins import ListCreateAPI
 from InvenTree.permissions import RolePermission
 from InvenTree.templatetags.inventree_extras import plugins_info
+from part.models import Part
 from plugin.serializers import MetadataSerializer
 from users.models import ApiToken
 
@@ -28,11 +32,41 @@
 from .views import AjaxView
 
 
+class VersionViewSerializer(serializers.Serializer):
+    """Serializer for a single version."""
+
+    class VersionSerializer(serializers.Serializer):
+        """Serializer for server version."""
+
+        server = serializers.CharField()
+        api = serializers.IntegerField()
+        commit_hash = serializers.CharField()
+        commit_date = serializers.CharField()
+        commit_branch = serializers.CharField()
+        python = serializers.CharField()
+        django = serializers.CharField()
+
+    class LinkSerializer(serializers.Serializer):
+        """Serializer for all possible links."""
+
+        doc = serializers.URLField()
+        code = serializers.URLField()
+        credit = serializers.URLField()
+        app = serializers.URLField()
+        bug = serializers.URLField()
+
+    dev = serializers.BooleanField()
+    up_to_date = serializers.BooleanField()
+    version = VersionSerializer()
+    links = LinkSerializer()
+
+
 class VersionView(APIView):
     """Simple JSON endpoint for InvenTree version information."""
 
     permission_classes = [permissions.IsAdminUser]
 
+    @extend_schema(responses={200: OpenApiResponse(response=VersionViewSerializer)})
     def get(self, request, *args, **kwargs):
         """Return information about the InvenTree server."""
         return JsonResponse({
@@ -81,6 +115,8 @@ class VersionApiSerializer(serializers.Serializer):
 class VersionTextView(ListAPI):
     """Simple JSON endpoint for InvenTree version text."""
 
+    serializer_class = VersionSerializer
+
     permission_classes = [permissions.IsAdminUser]
 
     @extend_schema(responses={200: OpenApiResponse(response=VersionApiSerializer)})
@@ -324,7 +360,17 @@ def perform_create(self, serializer):
         attachment.save()
 
 
-class APISearchView(APIView):
+class APISearchViewSerializer(serializers.Serializer):
+    """Serializer for the APISearchView."""
+
+    search = serializers.CharField()
+    search_regex = serializers.BooleanField(default=False, required=False)
+    search_whole = serializers.BooleanField(default=False, required=False)
+    limit = serializers.IntegerField(default=1, required=False)
+    offset = serializers.IntegerField(default=0, required=False)
+
+
+class APISearchView(GenericAPIView):
     """A general-purpose 'search' API endpoint.
 
     Returns hits against a number of different models simultaneously,
@@ -334,6 +380,7 @@ class APISearchView(APIView):
     """
 
     permission_classes = [permissions.IsAuthenticated]
+    serializer_class = APISearchViewSerializer
 
     def get_result_types(self):
         """Construct a list of search types we can return."""
@@ -446,4 +493,7 @@ def get_queryset(self):
 
     def get_serializer(self, *args, **kwargs):
         """Return MetadataSerializer instance."""
+        # Detect if we are currently generating the OpenAPI schema
+        if 'spectacular' in sys.argv:
+            return MetadataSerializer(Part, *args, **kwargs)
         return MetadataSerializer(self.get_model_type(), *args, **kwargs)

InvenTree/InvenTree/api_version.py

@@ -1,11 +1,14 @@
 """InvenTree API version information."""
 
 # InvenTree API version
-INVENTREE_API_VERSION = 166
+INVENTREE_API_VERSION = 167
 """Increment this API version number whenever there is a significant change to the API that any clients need to know about."""
 
 INVENTREE_API_TEXT = """
 
+v167 -> 2024-02-07: https://github.com/inventree/InvenTree/pull/6440
+    - Fixes for OpenAPI schema generation
+
 v166 -> 2024-02-04 : https://github.com/inventree/InvenTree/pull/6400
     - Adds package_name to plugin API
     - Adds mechanism for uninstalling plugins via the API

InvenTree/InvenTree/backends.py

@@ -0,0 +1,67 @@
+"""Custom backend implementations."""
+
+import logging
+import time
+
+from django.db.utils import IntegrityError, OperationalError, ProgrammingError
+
+from maintenance_mode.backends import AbstractStateBackend
+
+import common.models
+import InvenTree.helpers
+
+logger = logging.getLogger('inventree')
+
+
+class InvenTreeMaintenanceModeBackend(AbstractStateBackend):
+    """Custom backend for managing state of maintenance mode.
+
+    Stores the current state of the maintenance mode in the database,
+    using an InvenTreeSetting object.
+    """
+
+    SETTING_KEY = '_MAINTENANCE_MODE'
+
+    def get_value(self) -> bool:
+        """Get the current state of the maintenance mode.
+
+        Returns:
+            bool: True if maintenance mode is active, False otherwise.
+        """
+        try:
+            setting = common.models.InvenTreeSetting.objects.get(key=self.SETTING_KEY)
+            value = InvenTree.helpers.str2bool(setting.value)
+        except common.models.InvenTreeSetting.DoesNotExist:
+            # Database is accessible, but setting is not available - assume False
+            value = False
+        except (IntegrityError, OperationalError, ProgrammingError):
+            # Database is inaccessible - assume we are not in maintenance mode
+            logger.warning('Failed to read maintenance mode state - assuming True')
+            value = True
+
+        logger.debug('Maintenance mode state: %s', value)
+
+        return value
+
+    def set_value(self, value: bool, retries: int = 5):
+        """Set the state of the maintenance mode."""
+        logger.debug('Setting maintenance mode state: %s', value)
+
+        while retries > 0:
+            try:
+                common.models.InvenTreeSetting.set_setting(self.SETTING_KEY, value)
+
+                # Read the value back to confirm
+                if self.get_value() == value:
+                    break
+            except (IntegrityError, OperationalError, ProgrammingError):
+                # In the database is locked, then
+                logger.debug(
+                    'Failed to set maintenance mode state (%s retries left)', retries
+                )
+                time.sleep(0.1)
+
+            retries -= 1
+
+            if retries == 0:
+                logger.warning('Failed to set maintenance mode state')

InvenTree/InvenTree/magic_login.py

@@ -9,8 +9,8 @@
 
 import sesame.utils
 from rest_framework import serializers
+from rest_framework.generics import GenericAPIView
 from rest_framework.response import Response
-from rest_framework.views import APIView
 
 import InvenTree.version
 
@@ -38,7 +38,7 @@ class GetSimpleLoginSerializer(serializers.Serializer):
     email = serializers.CharField(label=_('Email'))
 
 
-class GetSimpleLoginView(APIView):
+class GetSimpleLoginView(GenericAPIView):
     """View to send a simple login link."""
 
     permission_classes = ()

InvenTree/InvenTree/serializers.py

@@ -28,6 +28,10 @@
 from InvenTree.helpers_model import download_image_from_url, get_base_url
 
 
+class EmptySerializer(serializers.Serializer):
+    """Empty serializer for use in testing."""
+
+
 class InvenTreeMoneySerializer(MoneyField):
     """Custom serializer for 'MoneyField', which ensures that passed values are numerically valid.
 

InvenTree/InvenTree/settings.py

@@ -489,12 +489,15 @@
 SPECTACULAR_SETTINGS = {
     'TITLE': 'InvenTree API',
     'DESCRIPTION': 'API for InvenTree - the intuitive open source inventory management system',
-    'LICENSE': {'MIT': 'https://github.com/inventree/InvenTree/blob/master/LICENSE'},
+    'LICENSE': {
+        'name': 'MIT',
+        'url': 'https://github.com/inventree/InvenTree/blob/master/LICENSE',
+    },
     'EXTERNAL_DOCS': {
-        'docs': 'https://docs.inventree.org',
-        'web': 'https://inventree.org',
+        'description': 'More information about InvenTree in the official docs',
+        'url': 'https://docs.inventree.org',
     },
-    'VERSION': inventreeApiVersion(),
+    'VERSION': str(inventreeApiVersion()),
     'SERVE_INCLUDE_SCHEMA': False,
 }
 
@@ -1080,8 +1083,8 @@
 IGNORED_ERRORS = [Http404, django.core.exceptions.PermissionDenied]
 
 # Maintenance mode
-MAINTENANCE_MODE_RETRY_AFTER = 60
-MAINTENANCE_MODE_STATE_BACKEND = 'maintenance_mode.backends.StaticStorageBackend'
+MAINTENANCE_MODE_RETRY_AFTER = 10
+MAINTENANCE_MODE_STATE_BACKEND = 'InvenTree.backends.InvenTreeMaintenanceModeBackend'
 
 # Are plugins enabled?
 PLUGINS_ENABLED = get_boolean_setting(