Closed issues:
+Merged pull requests:
Current version to use: v2.8.3
Important
The default for publish_on_pypi in the CD - Release workflow has changed from true to false in version 2.8.0.
To keep using the previous behaviour, set publish_on_pypi: true in the workflow file.
This change has been introduced to push for the use of PyPI's Trusted Publisher feature, which is not yet supported by reusable/callable workflows.
See the Using PyPI's Trusted Publisher section for more information on how to migrate to this feature.
Use tried and tested continuous integration (CI) and continuous deployment (CD) tools from this repository.
Currently, the repository offers GitHub Actions callable/reusable workflows and pre-commit hooks.
"},{"location":"#github-actions-callablereusable-workflows","title":"GitHub Actions callable/reusable Workflows","text":"This repository contains reusable workflows for GitHub Actions.
They are mainly for usage with modern Python package repositories.
"},{"location":"#available-workflows","title":"Available workflows","text":"The callable, reusable workflows available from this repository are described in detail in this documentation under the Workflows section.
"},{"location":"#general-usage","title":"General usage","text":"See the GitHub Docs on the topic of calling a reusable workflow to understand how one can incoporate one of these workflows in your workflow.
Note
Workflow-level set env context variables cannot be used when setting input values for the called workflow. See the GitHub documentation for more information on the env context.
Under the Workflows section for each available workflow, a usage example will be given.
"},{"location":"#pre-commit-hooks","title":"pre-commit hooks","text":"This repository contains hooks for keeping the documentation up-to-date, making available a few invoke tasks used in the reusable workflows.
By implementing and using these hooks together with the workflows, one may ensure no extra commits are created during the workflow run to update the documentation.
"},{"location":"#available-hooks","title":"Available hooks","text":"The pre-commit hooks available from this repository are described in detail in this documentation under the Hooks section.
"},{"location":"#general-usage_1","title":"General usage","text":"Add the hooks to your .pre-commit-config.yaml file. See the pre-commit webpage for more information about how to use pre-commit.
Under the Hooks section for each available hook, a usage example will be given.
"},{"location":"#license-copyright","title":"License & copyright","text":"This repository licensed under the MIT LICENSE with copyright \u00a9 2022 Casper Welzel Andersen (CasperWA) & SINTEF (on GitHub).
"},{"location":"#funding-support","title":"Funding support","text":"This repository has been supported by the following projects:
OntoTrans (2020-2024) that receives funding from the European Union\u2019s Horizon 2020 Research and Innovation Programme, under Grant Agreement n. 862136.
OpenModel (2021-2025) receives funding from the European Union\u2019s Horizon 2020 Research and Innovation Programme - DT-NMBP-11-2020 Open Innovation Platform for Materials Modelling, under Grant Agreement no: 953167.
Full Changelog
Merged pull requests:
Full Changelog
"},{"location":"CHANGELOG/#update-github-actions","title":"Update GitHub Actions","text":"Update the used GitHub Actions in the callable workflows.
"},{"location":"CHANGELOG/#dx","title":"DX","text":"Update development tools and dependencies for an improved developer experience.
Merged pull requests:
Full Changelog
"},{"location":"CHANGELOG/#support-self-hosted-runners","title":"Support self-hosted runners","text":"The runs-on key can not be specified via the runner input, which is available for all callable workflows. This means one can use the callable workflows with self-hosted runners, for example.
It is worth noting that the workflows are built with Linux/Unix systems in mind, hence specifying windows-latest may lead to issues with certain workflows. This is also true if the self-hosted runner is not Linux/Unix-based.
Implemented enhancements:
Merged pull requests:
runner input for all callable workflows #280 (CasperWA)Full Changelog
"},{"location":"CHANGELOG/#support-custom-pypi-indices","title":"Support custom PyPI indices","text":"All callable workflows now have support for setting the PIP_INDEX_URL and PIP_EXTRA_INDEX_URL environment variable whenever pip install is being invoked. Note, the PIP_EXTRA_INDEX_URL allows for multiple URLs to be provided, given they are space-delimited.
For more information on the specific workflow, see the documentation.
Implemented enhancements:
Merged pull requests:
Full Changelog
"},{"location":"CHANGELOG/#support-trusted-publishers-from-pypi","title":"Support Trusted Publishers from PyPI","text":"Trusted Publishers from PyPI is now supported via uploading the distribution(s) as artifacts (for more information about GitHub Actions artifacts, see the GitHub Docs).
Breaking change: This is not a \"true\" breaking change - but it may cause certain workflows to fail that uses the callable workflow CD - Release: The parameter publish_on_pypi has become required, meaning one must provide it in the with section of the calling workflow. For more information, see the documentation page for the CD - Release workflow.
Several fixes from the development tools have been implemented into the code base.
Implemented enhancements:
Merged pull requests:
Full Changelog
Implemented enhancements:
setver #243Merged pull requests:
Full Changelog
Fixed bugs:
git add -- . instead of git commit -a #236Merged pull requests:
Full Changelog
Fixed bugs:
Closed issues:
Merged pull requests:
Full Changelog
"},{"location":"CHANGELOG/#v270-2023-12-07","title":"v2.7.0 (2023-12-07)","text":"Full Changelog
Implemented enhancements:
Fixed bugs:
packaging.version.Version #220 (CasperWA)Closed issues:
Merged pull requests:
Full Changelog
Implemented enhancements:
update_deps.py further #148Fixed bugs:
Merged pull requests:
pyproject.toml) #213 (TEAM4-0)pyproject.toml) #206 (TEAM4-0)Full Changelog
Implemented enhancements:
Fixed bugs:
Merged pull requests:
already_handled_packages #202 (CasperWA)Full Changelog
Implemented enhancements:
latest alias MkDocs release #187Merged pull requests:
mkdocs_update_latest bool input #188 (CasperWA)Full Changelog
Fixed bugs:
--full-docs-dir input #174Merged pull requests:
Full Changelog
Fixed bugs:
pylint_options not working as intended #169Merged pull requests:
pylint_options depending on newlines #170 (CasperWA)Full Changelog
Implemented enhancements:
update-deps task #24Fixed bugs:
ci-cd update-deps #130Closed issues:
Merged pull requests:
update-pyproject pre-commit hook #128 (CasperWA)Full Changelog
Merged pull requests:
Full Changelog
Implemented enhancements:
Fixed bugs:
Merged pull requests:
Full Changelog
Fixed bugs:
Merged pull requests:
Full Changelog
Implemented enhancements:
Fixed bugs:
fail_fast should still make update-deps task fail #112Merged pull requests:
ignore option for update-deps task #111 (CasperWA)ci-cd #110 (CasperWA)Full Changelog
Implemented enhancements:
Closed issues:
Merged pull requests:
PAT prior to GITHUB_TOKEN #105 (CasperWA)Full Changelog
Implemented enhancements:
test: true actually work for \"CD - Release\" #83Fixed bugs:
Closed issues:
vMAJOR dynamic tag #81Merged pull requests:
Full Changelog
Fixed bugs:
Merged pull requests:
--strict toggleable for mkdocs build #78 (CasperWA)Full Changelog
Implemented enhancements:
Merged pull requests:
Full Changelog
Implemented enhancements:
Merged pull requests:
Full Changelog
Fixed bugs:
.pages in API ref hook #66Merged pull requests:
.pages does not get mkdocstrings option #67 (CasperWA)Full Changelog
Fixed bugs:
Merged pull requests:
Full Changelog
Implemented enhancements:
Fixed bugs:
Closed issues:
Merged pull requests:
Full Changelog
Fixed bugs:
Merged pull requests:
Full Changelog
Fixed bugs:
Merged pull requests:
Full Changelog
Implemented enhancements:
Fixed bugs:
CasperWA to SINTEF #37Closed issues:
Merged pull requests:
Full Changelog
Implemented enhancements:
Fixed bugs:
args for docs-landing-page doesn't work #27Merged pull requests:
args fix for docs-landing-page hook #31 (CasperWA)Full Changelog
Fixed bugs:
Merged pull requests:
Full Changelog
Implemented enhancements:
Merged pull requests:
Full Changelog
Fixed bugs:
Closed issues:
@v1 #15Full Changelog
Implemented enhancements:
Fixed bugs:
Closed issues:
Merged pull requests:
Full Changelog
Merged pull requests:
* This Changelog was automatically generated by github_changelog_generator
"},{"location":"LICENSE/","title":"License","text":"MIT License
Copyright (c) 2022 Casper Welzel Andersen & SINTEF
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
"},{"location":"api_reference/exceptions/","title":"exceptions","text":"CI/CD-specific exceptions.
"},{"location":"api_reference/exceptions/#ci_cd.exceptions.CICDException","title":" CICDException (Exception) ","text":"Top-level package exception class.
Source code inci_cd/exceptions.py class CICDException(Exception):\n \"\"\"Top-level package exception class.\"\"\"\n InputError (ValueError, CICDException) ","text":"There is an error with the input given to a task.
Source code inci_cd/exceptions.py class InputError(ValueError, CICDException):\n \"\"\"There is an error with the input given to a task.\"\"\"\n InputParserError (InputError) ","text":"The input could not be parsed, it may be wrongly formatted.
Source code inci_cd/exceptions.py class InputParserError(InputError):\n \"\"\"The input could not be parsed, it may be wrongly formatted.\"\"\"\n UnableToResolve (CICDException) ","text":"Unable to resolve a task or sub-task.
Source code inci_cd/exceptions.py class UnableToResolve(CICDException):\n \"\"\"Unable to resolve a task or sub-task.\"\"\"\nMain invoke Program.
See invoke documentation for more information.
"},{"location":"api_reference/tasks/api_reference_docs/","title":"api_reference_docs","text":"create_api_reference_docs task.
Create Python API reference in the documentation. This is specifically to be used with the MkDocs and mkdocstrings framework.
"},{"location":"api_reference/tasks/api_reference_docs/#ci_cd.tasks.api_reference_docs.create_api_reference_docs","title":"create_api_reference_docs(context, package_dir, pre_clean=False, pre_commit=False, root_repo_path='.', docs_folder='docs', unwanted_folder=None, unwanted_file=None, full_docs_folder=None, full_docs_file=None, special_option=None, relative=False, debug=False)","text":"Create the Python API Reference in the documentation.
Source code inci_cd/tasks/api_reference_docs.py @task(\n help={\n \"package-dir\": (\n \"Relative path to a package dir from the repository root, \"\n \"e.g., 'src/my_package'. This input option can be supplied multiple times.\"\n ),\n \"pre-clean\": \"Remove the 'api_reference' sub directory prior to (re)creation.\",\n \"pre-commit\": (\n \"Whether or not this task is run as a pre-commit hook. Will return a \"\n \"non-zero error code if changes were made.\"\n ),\n \"root-repo-path\": (\n \"A resolvable path to the root directory of the repository folder.\"\n ),\n \"docs-folder\": (\n \"The folder name for the documentation root folder. \"\n \"This defaults to 'docs'.\"\n ),\n \"unwanted-folder\": (\n \"A folder to avoid including into the Python API reference documentation. \"\n \"Note, only folder names, not paths, may be included. Note, all folders \"\n \"and their contents with this name will be excluded. Defaults to \"\n \"'__pycache__'. This input option can be supplied multiple times.\"\n ),\n \"unwanted-file\": (\n \"A file to avoid including into the Python API reference documentation. \"\n \"Note, only full file names, not paths, may be included, i.e., filename + \"\n \"file extension. Note, all files with this names will be excluded. \"\n \"Defaults to '__init__.py'. This input option can be supplied multiple \"\n \"times.\"\n ),\n \"full-docs-folder\": (\n \"A folder in which to include everything - even those without \"\n \"documentation strings. This may be useful for a module full of data \"\n \"models or to ensure all class attributes are listed. This input option \"\n \"can be supplied multiple times.\"\n ),\n \"full-docs-file\": (\n \"A full relative path to a file in which to include everything - even \"\n \"those without documentation strings. This may be useful for a file full \"\n \"of data models or to ensure all class attributes are listed. This input \"\n \"option can be supplied multiple times.\"\n ),\n \"special-option\": (\n \"A combination of a relative path to a file and a fully formed \"\n \"mkdocstrings option that should be added to the generated MarkDown file. \"\n \"The combination should be comma-separated. Example: \"\n \"'my_module/py_file.py,show_bases:false'. Encapsulate the value in double \"\n 'quotation marks (\") if including spaces ( ). Important: If multiple '\n \"package-dir options are supplied, the relative path MUST include/start \"\n \"with the package-dir value, e.g., \"\n \"'\\\"my_package/my_module/py_file.py,show_bases: false\\\"'. This input \"\n \"option can be supplied multiple times. The options will be accumulated \"\n \"for the same file, if given several times.\"\n ),\n \"relative\": (\n \"Whether or not to use relative Python import links in the API reference \"\n \"markdown files.\"\n ),\n \"debug\": \"Whether or not to print debug statements.\",\n },\n iterable=[\n \"package_dir\",\n \"unwanted_folder\",\n \"unwanted_file\",\n \"full_docs_folder\",\n \"full_docs_file\",\n \"special_option\",\n ],\n)\ndef create_api_reference_docs(\n context,\n package_dir,\n pre_clean=False,\n pre_commit=False,\n root_repo_path=\".\",\n docs_folder=\"docs\",\n unwanted_folder=None,\n unwanted_file=None,\n full_docs_folder=None,\n full_docs_file=None,\n special_option=None,\n relative=False,\n debug=False,\n):\n \"\"\"Create the Python API Reference in the documentation.\"\"\"\n if TYPE_CHECKING: # pragma: no cover\n context: Context = context # type: ignore[no-redef]\n pre_clean: bool = pre_clean # type: ignore[no-redef]\n pre_commit: bool = pre_commit # type: ignore[no-redef]\n root_repo_path: str = root_repo_path # type: ignore[no-redef]\n docs_folder: str = docs_folder # type: ignore[no-redef]\n relative: bool = relative # type: ignore[no-redef]\n debug: bool = debug # type: ignore[no-redef]\n\n if not unwanted_folder:\n unwanted_folder: list[str] = [\"__pycache__\"] # type: ignore[no-redef]\n if not unwanted_file:\n unwanted_file: list[str] = [\"__init__.py\"] # type: ignore[no-redef]\n if not full_docs_folder:\n full_docs_folder: list[str] = [] # type: ignore[no-redef]\n if not full_docs_file:\n full_docs_file: list[str] = [] # type: ignore[no-redef]\n if not special_option:\n special_option: list[str] = [] # type: ignore[no-redef]\n\n # Initialize user-given paths as pure POSIX paths\n package_dir: list[PurePosixPath] = [PurePosixPath(_) for _ in package_dir]\n root_repo_path = str(PurePosixPath(root_repo_path))\n docs_folder: PurePosixPath = PurePosixPath(docs_folder) # type: ignore[no-redef]\n full_docs_folder = [Path(PurePosixPath(_)) for _ in full_docs_folder]\n\n def write_file(full_path: Path, content: str) -> None:\n \"\"\"Write file with `content` to `full_path`\"\"\"\n if full_path.exists():\n cached_content = full_path.read_text(encoding=\"utf8\")\n if content == cached_content:\n del cached_content\n return\n del cached_content\n full_path.write_text(content, encoding=\"utf8\")\n\n if pre_commit:\n # Ensure git is installed\n result: Result = context.run(\"git --version\", hide=True)\n if result.exited != 0:\n sys.exit(\n \"Git is not installed. Please install it before running this task.\"\n )\n\n if pre_commit and root_repo_path == \".\":\n # Use git to determine repo root\n result = context.run(\"git rev-parse --show-toplevel\", hide=True)\n root_repo_path = result.stdout.strip(\"\\n\") # type: ignore[no-redef]\n\n root_repo_path: Path = Path(root_repo_path).resolve() # type: ignore[no-redef]\n package_dirs: list[Path] = [Path(root_repo_path / _) for _ in package_dir]\n docs_api_ref_dir = Path(root_repo_path / docs_folder / \"api_reference\")\n\n LOGGER.debug(\n \"\"\"package_dirs: %s\ndocs_api_ref_dir: %s\nunwanted_folder: %s\nunwanted_file: %s\nfull_docs_folder: %s\nfull_docs_file: %s\nspecial_option: %s\"\"\",\n package_dirs,\n docs_api_ref_dir,\n unwanted_folder,\n unwanted_file,\n full_docs_folder,\n full_docs_file,\n special_option,\n )\n if debug:\n print(\"package_dirs:\", package_dirs, flush=True)\n print(\"docs_api_ref_dir:\", docs_api_ref_dir, flush=True)\n print(\"unwanted_folder:\", unwanted_folder, flush=True)\n print(\"unwanted_file:\", unwanted_file, flush=True)\n print(\"full_docs_folder:\", full_docs_folder, flush=True)\n print(\"full_docs_file:\", full_docs_file, flush=True)\n print(\"special_option:\", special_option, flush=True)\n\n special_options_files = defaultdict(list)\n for special_file, option in [_.split(\",\", maxsplit=1) for _ in special_option]:\n if any(\",\" in _ for _ in (special_file, option)):\n LOGGER.error(\n \"Failing for special-option: %s\", \",\".join([special_file, option])\n )\n if debug:\n print(\n \"Failing for special-option:\",\n \",\".join([special_file, option]),\n flush=True,\n )\n sys.exit(\n \"special-option values may only include a single comma (,) to \"\n \"separate the relative file path and the mkdocstsrings option.\"\n )\n special_options_files[special_file].append(option)\n\n LOGGER.debug(\"special_options_files: %s\", special_options_files)\n if debug:\n print(\"special_options_files:\", special_options_files, flush=True)\n\n if any(os.sep in _ or \"/\" in _ for _ in unwanted_folder + unwanted_file):\n sys.exit(\"Unwanted folders and files may NOT be paths.\")\n\n pages_template = 'title: \"{name}\"\\n'\n md_template = \"# {name}\\n\\n::: {py_path}\\n\"\n no_docstring_template_addition = (\n f\"{' ' * 4}options:\\n{' ' * 6}show_if_no_docstring: true\\n\"\n )\n\n if docs_api_ref_dir.exists() and pre_clean:\n LOGGER.debug(\"Removing %s\", docs_api_ref_dir)\n if debug:\n print(f\"Removing {docs_api_ref_dir}\", flush=True)\n shutil.rmtree(docs_api_ref_dir, ignore_errors=True)\n if docs_api_ref_dir.exists():\n sys.exit(f\"{docs_api_ref_dir} should have been removed!\")\n docs_api_ref_dir.mkdir(exist_ok=True)\n\n LOGGER.debug(\"Writing file: %s\", docs_api_ref_dir / \".pages\")\n if debug:\n print(f\"Writing file: {docs_api_ref_dir / '.pages'}\", flush=True)\n write_file(\n full_path=docs_api_ref_dir / \".pages\",\n content=pages_template.format(name=\"API Reference\"),\n )\n\n single_package = len(package_dirs) == 1\n for package in package_dirs:\n for dirpath, dirnames, filenames in os.walk(package):\n for unwanted in unwanted_folder:\n LOGGER.debug(\"unwanted: %s\\ndirnames: %s\", unwanted, dirnames)\n if debug:\n print(\"unwanted:\", unwanted, flush=True)\n print(\"dirnames:\", dirnames, flush=True)\n if unwanted in dirnames:\n # Avoid walking into or through unwanted directories\n dirnames.remove(unwanted)\n\n relpath = Path(dirpath).relative_to(\n package if single_package else package.parent\n )\n abspath = (\n package / relpath if single_package else package.parent / relpath\n ).resolve()\n LOGGER.debug(\"relpath: %s\\nabspath: %s\", relpath, abspath)\n if debug:\n print(\"relpath:\", relpath, flush=True)\n print(\"abspath:\", abspath, flush=True)\n\n if not (abspath / \"__init__.py\").exists():\n # Avoid paths that are not included in the public Python API\n LOGGER.debug(\"does not exist: %s\", abspath / \"__init__.py\")\n print(\"does not exist:\", abspath / \"__init__.py\", flush=True)\n continue\n\n # Create `.pages`\n docs_sub_dir = docs_api_ref_dir / relpath\n docs_sub_dir.mkdir(exist_ok=True)\n LOGGER.debug(\"docs_sub_dir: %s\", docs_sub_dir)\n if debug:\n print(\"docs_sub_dir:\", docs_sub_dir, flush=True)\n if str(relpath) != \".\":\n LOGGER.debug(\"Writing file: %s\", docs_sub_dir / \".pages\")\n if debug:\n print(f\"Writing file: {docs_sub_dir / '.pages'}\", flush=True)\n write_file(\n full_path=docs_sub_dir / \".pages\",\n content=pages_template.format(name=relpath.name),\n )\n\n # Create markdown files\n for filename in (Path(_) for _ in filenames):\n if (\n re.match(r\".*\\.py$\", str(filename)) is None\n or str(filename) in unwanted_file\n ):\n # Not a Python file: We don't care about it!\n # Or filename is in the list of unwanted files:\n # We don't want it!\n LOGGER.debug(\n \"%s is not a Python file or is an unwanted file (through user \"\n \"input). Skipping it.\",\n filename,\n )\n if debug:\n print(\n f\"{filename} is not a Python file or is an unwanted file \"\n \"(through user input). Skipping it.\",\n flush=True,\n )\n continue\n\n py_path_root = (\n package.relative_to(root_repo_path) if relative else package.name\n )\n py_path = (\n f\"{py_path_root}/{filename.stem}\"\n if str(relpath) == \".\"\n or (str(relpath) == package.name and not single_package)\n else (\n f\"{py_path_root}/\"\n f\"{relpath if single_package else relpath.relative_to(package.name)}/\" # noqa: E501\n f\"{filename.stem}\"\n )\n )\n\n # Replace OS specific path separators with forward slashes before\n # replacing that with dots (for Python import paths).\n py_path = py_path.replace(os.sep, \"/\").replace(\"/\", \".\")\n\n LOGGER.debug(\"filename: %s\\npy_path: %s\", filename, py_path)\n if debug:\n print(\"filename:\", filename, flush=True)\n print(\"py_path:\", py_path, flush=True)\n\n relative_file_path = Path(\n str(filename) if str(relpath) == \".\" else str(relpath / filename)\n ).as_posix()\n\n # For special files we want to include EVERYTHING, even if it doesn't\n # have a doc-string\n template = md_template + (\n no_docstring_template_addition\n if relative_file_path in full_docs_file\n or relpath in full_docs_folder\n else \"\"\n )\n\n # Include special options, if any, for certain files.\n if relative_file_path in special_options_files:\n template += (\n f\"{' ' * 4}options:\\n\" if \"options:\\n\" not in template else \"\"\n )\n template += \"\\n\".join(\n f\"{' ' * 6}{option}\"\n for option in special_options_files[relative_file_path]\n )\n template += \"\\n\"\n\n LOGGER.debug(\n \"template: %s\\nWriting file: %s\",\n template,\n docs_sub_dir / filename.with_suffix(\".md\"),\n )\n if debug:\n print(\"template:\", template, flush=True)\n print(\n f\"Writing file: {docs_sub_dir / filename.with_suffix('.md')}\",\n flush=True,\n )\n\n write_file(\n full_path=docs_sub_dir / filename.with_suffix(\".md\"),\n content=template.format(name=filename.stem, py_path=py_path),\n )\n\n if pre_commit:\n # Check if there have been any changes.\n # List changes if yes.\n\n # NOTE: Concerning the weird regular expression, see:\n # http://manpages.ubuntu.com/manpages/precise/en/man1/git-status.1.html\n result = context.run(\n f'git -C \"{root_repo_path}\" status --porcelain '\n f\"{docs_api_ref_dir.relative_to(root_repo_path)}\",\n hide=True,\n )\n if result.stdout:\n for line in result.stdout.splitlines():\n if re.match(r\"^[? MARC][?MD]\", line):\n sys.exit(\n f\"{Emoji.CURLY_LOOP.value} The following files have been \"\n f\"changed/added/removed:\\n\\n{result.stdout}\\n\"\n \"Please stage them:\\n\\n\"\n f\" git add {docs_api_ref_dir.relative_to(root_repo_path)}\"\n )\n print(\n f\"{Emoji.CHECK_MARK.value} No changes - your API reference documentation \"\n \"is up-to-date !\"\n )\ncreate_docs_index task.
Create the documentation index (home) page from README.md.
create_docs_index(context, pre_commit=False, root_repo_path='.', docs_folder='docs', replacement=None, replacement_separator=',')","text":"Create the documentation index page from README.md.
Source code inci_cd/tasks/docs_index.py @task(\n help={\n \"pre-commit\": \"Whether or not this task is run as a pre-commit hook.\",\n \"root-repo-path\": (\n \"A resolvable path to the root directory of the repository folder.\"\n ),\n \"docs-folder\": (\n \"The folder name for the documentation root folder. \"\n \"This defaults to 'docs'.\"\n ),\n \"replacement\": (\n \"A replacement (mapping) to be performed on README.md when creating the \"\n \"documentation's landing page (index.md). This list ALWAYS includes \"\n \"replacing '{docs-folder}/' with an empty string, in order to correct \"\n \"relative links. This input option can be supplied multiple times.\"\n ),\n \"replacement-separator\": (\n \"String to separate a replacement's 'old' to 'new' parts.\"\n \"Defaults to a comma (,).\"\n ),\n },\n iterable=[\"replacement\"],\n)\ndef create_docs_index(\n context,\n pre_commit=False,\n root_repo_path=\".\",\n docs_folder=\"docs\",\n replacement=None,\n replacement_separator=\",\",\n):\n \"\"\"Create the documentation index page from README.md.\"\"\"\n if TYPE_CHECKING: # pragma: no cover\n context: Context = context # type: ignore[no-redef]\n pre_commit: bool = pre_commit # type: ignore[no-redef]\n root_repo_path: str = root_repo_path # type: ignore[no-redef]\n replacement_separator: str = replacement_separator # type: ignore[no-redef]\n\n docs_folder: Path = Path(docs_folder)\n\n if not replacement:\n replacement: list[str] = [] # type: ignore[no-redef]\n replacement.append(f\"{docs_folder.name}/{replacement_separator}\")\n\n if pre_commit and root_repo_path == \".\":\n # Use git to determine repo root\n result: Result = context.run(\"git rev-parse --show-toplevel\", hide=True)\n root_repo_path = result.stdout.strip(\"\\n\")\n\n root_repo_path: Path = Path(root_repo_path).resolve()\n readme = root_repo_path / \"README.md\"\n docs_index = root_repo_path / docs_folder / \"index.md\"\n\n content = readme.read_text(encoding=\"utf8\")\n\n for mapping in replacement:\n try:\n old, new = mapping.split(replacement_separator)\n except ValueError:\n sys.exit(\n \"A replacement must only include an 'old' and 'new' part, i.e., be of \"\n \"exactly length 2 when split by the '--replacement-separator'. The \"\n \"following replacement did not fulfill this requirement: \"\n f\"{mapping!r}\\n --replacement-separator={replacement_separator!r}\"\n )\n content = content.replace(old, new)\n\n docs_index.write_text(content, encoding=\"utf8\")\n\n if pre_commit:\n # Check if there have been any changes.\n # List changes if yes.\n\n # NOTE: Concerning the weird regular expression, see:\n # http://manpages.ubuntu.com/manpages/precise/en/man1/git-status.1.html\n result: Result = context.run( # type: ignore[no-redef]\n f'git -C \"{root_repo_path}\" status --porcelain '\n f\"{docs_index.relative_to(root_repo_path)}\",\n hide=True,\n )\n if result.stdout:\n for line in result.stdout.splitlines():\n if re.match(r\"^[? MARC][?MD]\", line):\n sys.exit(\n f\"{Emoji.CURLY_LOOP.value} The landing page has been updated.\"\n \"\\n\\nPlease stage it:\\n\\n\"\n f\" git add {docs_index.relative_to(root_repo_path)}\"\n )\n print(\n f\"{Emoji.CHECK_MARK.value} No changes - your landing page is up-to-date !\"\n )\nsetver task.
Set the specified version.
"},{"location":"api_reference/tasks/setver/#ci_cd.tasks.setver.setver","title":"setver(_, package_dir, version, root_repo_path='.', code_base_update=None, code_base_update_separator=',', test=False, fail_fast=False)","text":"Sets the specified version of specified Python package.
Source code inci_cd/tasks/setver.py @task(\n help={\n \"version\": \"Version to set. Must be either a SemVer or a PEP 440 version.\",\n \"package-dir\": (\n \"Relative path to package dir from the repository root, \"\n \"e.g. 'src/my_package'.\"\n ),\n \"root-repo-path\": (\n \"A resolvable path to the root directory of the repository folder.\"\n ),\n \"code-base-update\": (\n \"'--code-base-update-separator'-separated string defining 'file path', \"\n \"'pattern', 'replacement string', in that order, for something to update \"\n \"in the code base. E.g., '{package_dir}/__init__.py,__version__ *= \"\n \"*('|\\\").*('|\\\"),__version__ = \\\"{version}\\\"', where '{package_dir}' \"\n \"and {version} will be exchanged with the given '--package-dir' value and \"\n \"given '--version' value, respectively. The 'file path' must always \"\n \"either be relative to the repository root directory or absolute. The \"\n \"'pattern' should be given as a 'raw' Python string. This input option \"\n \"can be supplied multiple times.\"\n ),\n \"code-base-update-separator\": (\n \"The string separator to use for '--code-base-update' values.\"\n ),\n \"fail_fast\": (\n \"Whether to exit the task immediately upon failure or wait until the end. \"\n \"Note, no code changes will happen if an error occurs.\"\n ),\n \"test\": (\n \"Whether to do a dry run or not. If set, the task will not make any \"\n \"changes to the code base.\"\n ),\n },\n iterable=[\"code_base_update\"],\n)\ndef setver(\n _,\n package_dir,\n version,\n root_repo_path=\".\",\n code_base_update=None,\n code_base_update_separator=\",\",\n test=False,\n fail_fast=False,\n):\n \"\"\"Sets the specified version of specified Python package.\"\"\"\n if TYPE_CHECKING: # pragma: no cover\n package_dir: str = package_dir # type: ignore[no-redef]\n version: str = version # type: ignore[no-redef]\n root_repo_path: str = root_repo_path # type: ignore[no-redef]\n code_base_update: list[str] = code_base_update # type: ignore[no-redef]\n code_base_update_separator: str = code_base_update_separator # type: ignore[no-redef]\n test: bool = test # type: ignore[no-redef]\n fail_fast: bool = fail_fast # type: ignore[no-redef]\n\n # Validate inputs\n # Version\n try:\n semantic_version = SemanticVersion(version)\n except ValueError:\n msg = (\n \"Please specify version as a semantic version (SemVer) or PEP 440 version. \"\n \"The version may be prepended by a 'v'.\"\n )\n sys.exit(f\"{Emoji.CROSS_MARK.value} {error_msg(msg)}\")\n\n # Root repo path\n root_repo = Path(root_repo_path).resolve()\n if not root_repo.exists():\n msg = (\n f\"Could not find the repository root at: {root_repo} (user provided: \"\n f\"{root_repo_path!r})\"\n )\n sys.exit(f\"{Emoji.CROSS_MARK.value} {error_msg(msg)}\")\n\n # Run the task with defaults\n if not code_base_update:\n init_file = root_repo / package_dir / \"__init__.py\"\n if not init_file.exists():\n msg = (\n \"Could not find the Python package's root '__init__.py' file at: \"\n f\"{init_file}\"\n )\n sys.exit(f\"{Emoji.CROSS_MARK.value} {error_msg(msg)}\")\n\n update_file(\n init_file,\n (\n r'__version__ *= *(?:\\'|\").*(?:\\'|\")',\n f'__version__ = \"{semantic_version}\"',\n ),\n )\n\n # Success, done\n print(\n f\"{Emoji.PARTY_POPPER.value} Bumped version for {package_dir} to \"\n f\"{semantic_version}.\"\n )\n return\n\n # Code base updates were provided\n # First, validate the inputs\n validated_code_base_updates: list[tuple[Path, str, str, str]] = []\n error: bool = False\n for code_update in code_base_update:\n try:\n filepath, pattern, replacement = code_update.split(\n code_base_update_separator\n )\n except ValueError as exc:\n msg = (\n \"Could not properly extract 'file path', 'pattern', \"\n f\"'replacement string' from the '--code-base-update'={code_update}:\"\n f\"\\n{exc}\"\n )\n LOGGER.error(msg)\n LOGGER.debug(\"Traceback: %s\", traceback.format_exc())\n if fail_fast:\n sys.exit(f\"{Emoji.CROSS_MARK.value} {error_msg(msg)}\")\n print(error_msg(msg), file=sys.stderr, flush=True)\n error = True\n continue\n\n # Resolve file path\n filepath = Path(\n filepath.format(package_dir=package_dir, version=semantic_version)\n )\n\n if not filepath.is_absolute():\n filepath = root_repo / filepath\n\n if not filepath.exists():\n msg = f\"Could not find the user-provided file at: {filepath}\"\n LOGGER.error(msg)\n if fail_fast:\n sys.exit(f\"{Emoji.CROSS_MARK.value} {error_msg(msg)}\")\n print(error_msg(msg), file=sys.stderr, flush=True)\n error = True\n continue\n\n LOGGER.debug(\n \"\"\"filepath: %s\npattern: %r\nreplacement (input): %s\nreplacement (handled): %s\n\"\"\",\n filepath,\n pattern,\n replacement,\n replacement.format(package_dir=package_dir, version=semantic_version),\n )\n\n validated_code_base_updates.append(\n (\n filepath,\n pattern,\n replacement.format(package_dir=package_dir, version=semantic_version),\n replacement,\n )\n )\n\n if error:\n sys.exit(\n f\"{Emoji.CROSS_MARK.value} Errors occurred! See printed statements above.\"\n )\n\n for (\n filepath,\n pattern,\n replacement,\n input_replacement,\n ) in validated_code_base_updates:\n if test:\n print(\n f\"filepath: {filepath}\\npattern: {pattern!r}\\n\"\n f\"replacement (input): {input_replacement}\\n\"\n f\"replacement (handled): {replacement}\"\n )\n continue\n\n try:\n update_file(filepath, (pattern, replacement))\n except re.error as exc:\n if validated_code_base_updates[0] != (\n filepath,\n pattern,\n replacement,\n input_replacement,\n ):\n msg = \"Some files have already been updated !\\n\\n \"\n\n msg += (\n f\"Could not update file {filepath} according to the given input:\\n\\n \"\n f\"pattern: {pattern}\\n replacement: {replacement}\\n\\nException: \"\n f\"{exc}\"\n )\n LOGGER.error(msg)\n LOGGER.debug(\"Traceback: %s\", traceback.format_exc())\n sys.exit(f\"{Emoji.CROSS_MARK.value} {error_msg(msg)}\")\n\n # Success, done\n print(\n f\"{Emoji.PARTY_POPPER.value} Bumped version for {package_dir} to \"\n f\"{semantic_version}.\"\n )\nupdate_deps task.
Update dependencies in a pyproject.toml file.
VALID_PACKAGE_NAME_PATTERN","text":"Pattern to validate package names.
This is a valid non-normalized name, i.e., it can contain capital letters and underscores, periods, and multiples of these, including minus characters.
See PEP 508 for more information, as well as the packaging documentation: https://packaging.python.org/en/latest/specifications/name-normalization/
"},{"location":"api_reference/tasks/update_deps/#ci_cd.tasks.update_deps.update_deps","title":"update_deps(context, root_repo_path='.', fail_fast=False, pre_commit=False, ignore=None, ignore_separator='...', verbose=False, skip_unnormalized_python_package_names=False)","text":"Update dependencies in specified Python package's pyproject.toml.
ci_cd/tasks/update_deps.py @task(\n help={\n \"fail-fast\": (\n \"Fail immediately if an error occurs. Otherwise, print and ignore all \"\n \"non-critical errors.\"\n ),\n \"root-repo-path\": (\n \"A resolvable path to the root directory of the repository folder.\"\n ),\n \"pre-commit\": \"Whether or not this task is run as a pre-commit hook.\",\n \"ignore\": (\n \"Ignore-rules based on the `ignore` config option of Dependabot. It \"\n \"should be of the format: key=value...key=value, i.e., an ellipsis \"\n \"(`...`) separator and then equal-sign-separated key/value-pairs. \"\n \"Alternatively, the `--ignore-separator` can be set to something else to \"\n \"overwrite the ellipsis. The only supported keys are: `dependency-name`, \"\n \"`versions`, and `update-types`. Can be supplied multiple times per \"\n \"`dependency-name`.\"\n ),\n \"ignore-separator\": (\n \"Value to use instead of ellipsis (`...`) as a separator in `--ignore` \"\n \"key/value-pairs.\"\n ),\n \"verbose\": \"Whether or not to print debug statements.\",\n \"skip-unnormalized-python-package-names\": (\n \"Whether to skip dependencies with unnormalized Python package names. \"\n \"Normalization is outlined here: \"\n \"https://packaging.python.org/en/latest/specifications/name-normalization.\"\n ),\n },\n iterable=[\"ignore\"],\n)\ndef update_deps(\n context,\n root_repo_path=\".\",\n fail_fast=False,\n pre_commit=False,\n ignore=None,\n ignore_separator=\"...\",\n verbose=False,\n skip_unnormalized_python_package_names=False,\n):\n \"\"\"Update dependencies in specified Python package's `pyproject.toml`.\"\"\"\n if TYPE_CHECKING: # pragma: no cover\n context: Context = context # type: ignore[no-redef]\n root_repo_path: str = root_repo_path # type: ignore[no-redef]\n fail_fast: bool = fail_fast # type: ignore[no-redef]\n pre_commit: bool = pre_commit # type: ignore[no-redef]\n ignore_separator: str = ignore_separator # type: ignore[no-redef]\n verbose: bool = verbose # type: ignore[no-redef]\n skip_unnormalized_python_package_names: bool = ( # type: ignore[no-redef]\n skip_unnormalized_python_package_names\n )\n\n if not ignore:\n ignore: list[str] = [] # type: ignore[no-redef]\n\n if verbose:\n LOGGER.addHandler(logging.StreamHandler(sys.stdout))\n LOGGER.debug(\"Verbose logging enabled.\")\n\n try:\n ignore_rules = parse_ignore_entries(ignore, ignore_separator)\n except InputError as exc:\n sys.exit(\n f\"{Emoji.CROSS_MARK.value} Error: Could not parse ignore options.\\n\"\n f\"Exception: {exc}\"\n )\n LOGGER.debug(\"Parsed ignore rules: %s\", ignore_rules)\n\n if pre_commit and root_repo_path == \".\":\n # Use git to determine repo root\n result: Result = context.run(\"git rev-parse --show-toplevel\", hide=True)\n root_repo_path = result.stdout.strip(\"\\n\")\n\n pyproject_path = Path(root_repo_path).resolve() / \"pyproject.toml\"\n if not pyproject_path.exists():\n sys.exit(\n f\"{Emoji.CROSS_MARK.value} Error: Could not find the Python package \"\n f\"repository's 'pyproject.toml' file at: {pyproject_path}\"\n )\n\n # Parse pyproject.toml\n try:\n pyproject = tomlkit.parse(pyproject_path.read_bytes())\n except TOMLKitError as exc:\n sys.exit(\n f\"{Emoji.CROSS_MARK.value} Error: Could not parse the 'pyproject.toml' \"\n f\"file at: {pyproject_path}\\nException: {exc}\"\n )\n\n # Retrieve the minimum required Python version\n try:\n py_version = get_min_max_py_version(\n pyproject.get(\"project\", {}).get(\"requires-python\", \"\")\n )\n except UnableToResolve as exc:\n sys.exit(\n f\"{Emoji.CROSS_MARK.value} Error: Cannot determine minimum Python version.\"\n f\"\\nException: {exc}\"\n )\n LOGGER.debug(\"Minimum required Python version: %s\", py_version)\n\n # Retrieve the Python project's package name\n project_name: str = pyproject.get(\"project\", {}).get(\"name\", \"\")\n if not project_name:\n sys.exit(\n f\"{Emoji.CROSS_MARK.value} Error: Could not find the Python project's name\"\n \" in 'pyproject.toml'.\"\n )\n\n # Build the list of dependencies listed in pyproject.toml\n dependencies: list[str] = pyproject.get(\"project\", {}).get(\"dependencies\", [])\n for optional_deps in (\n pyproject.get(\"project\", {}).get(\"optional-dependencies\", {}).values()\n ):\n dependencies.extend(optional_deps)\n\n # Placeholder and default variables\n already_handled_packages: set[Requirement] = set()\n updated_packages: dict[str, str] = {}\n error: bool = False\n\n for dependency in dependencies:\n try:\n parsed_requirement = Requirement(dependency)\n except InvalidRequirement as exc:\n if skip_unnormalized_python_package_names:\n msg = (\n f\"Skipping requirement {dependency!r}, as unnormalized Python \"\n \"package naming is allowed by user. Note, the requirements could \"\n f\"not be parsed: {exc}\"\n )\n LOGGER.info(msg)\n print(info_msg(msg), flush=True)\n continue\n\n msg = (\n f\"Could not parse requirement {dependency!r} from pyproject.toml: \"\n f\"{exc}\"\n )\n LOGGER.error(msg)\n if fail_fast:\n sys.exit(f\"{Emoji.CROSS_MARK.value} {error_msg(msg)}\")\n print(error_msg(msg), file=sys.stderr, flush=True)\n error = True\n continue\n LOGGER.debug(\"Parsed requirement: %r\", parsed_requirement)\n\n # Skip package if already handled\n if parsed_requirement in already_handled_packages:\n continue\n\n # Skip package if it is this project (this can happen for inter-relative extra\n # dependencies)\n if parsed_requirement.name == project_name:\n msg = (\n f\"Dependency {parsed_requirement.name!r} is detected as being this \"\n \"project and will be skipped.\"\n )\n LOGGER.info(msg)\n print(info_msg(msg), flush=True)\n\n _format_and_update_dependency(\n parsed_requirement, dependency, pyproject_path\n )\n already_handled_packages.add(parsed_requirement)\n continue\n\n # Skip URL versioned dependencies\n # BUT do regenerate the dependency in order to have a consistent formatting\n if parsed_requirement.url:\n msg = (\n f\"Dependency {parsed_requirement.name!r} is pinned to a URL and \"\n \"will be skipped.\"\n )\n LOGGER.info(msg)\n print(info_msg(msg), flush=True)\n\n _format_and_update_dependency(\n parsed_requirement, dependency, pyproject_path\n )\n already_handled_packages.add(parsed_requirement)\n continue\n\n # Skip and warn if package is not version-restricted\n # BUT do regenerate the dependency in order to have a consistent formatting\n if not parsed_requirement.specifier:\n # Only warn if package name does not match project name\n if parsed_requirement.name != project_name:\n msg = (\n f\"Dependency {parsed_requirement.name!r} is not version \"\n \"restricted and will be skipped. Consider adding version \"\n \"restrictions.\"\n )\n LOGGER.warning(msg)\n print(warning_msg(msg), flush=True)\n\n _format_and_update_dependency(\n parsed_requirement, dependency, pyproject_path\n )\n already_handled_packages.add(parsed_requirement)\n continue\n\n # Examine markers for a custom set of Python version specifiers\n marker_py_version = \"\"\n if parsed_requirement.marker:\n environment_keys = default_environment().keys()\n empty_environment = {key: \"\" for key in environment_keys}\n python_version_centric_environment = empty_environment\n python_version_centric_environment.update({\"python_version\": py_version})\n\n if not parsed_requirement.marker.evaluate(\n environment=python_version_centric_environment\n ):\n # Current (minimum) Python version does NOT satisfy the marker\n marker_py_version = find_minimum_py_version(\n marker=parsed_requirement.marker,\n project_py_version=py_version,\n )\n else:\n marker_py_version = get_min_max_py_version(parsed_requirement.marker)\n\n LOGGER.debug(\"Min/max Python version from marker: %s\", marker_py_version)\n\n # Check version from PyPI's online package index\n out: Result = context.run(\n \"pip index versions \"\n f\"--python-version {marker_py_version or py_version} \"\n f\"{parsed_requirement.name}\",\n hide=True,\n )\n package_latest_version_line = out.stdout.split(sep=\"\\n\", maxsplit=1)[0]\n match = re.match(\n r\"(?P<package>\\S+) \\((?P<version>\\S+)\\)\", package_latest_version_line\n )\n if match is None:\n msg = (\n \"Could not parse package and version from 'pip index versions' output \"\n f\"for line:\\n {package_latest_version_line}\"\n )\n LOGGER.error(msg)\n if fail_fast:\n sys.exit(f\"{Emoji.CROSS_MARK.value} {error_msg(msg)}\")\n print(error_msg(msg), file=sys.stderr, flush=True)\n already_handled_packages.add(parsed_requirement)\n error = True\n continue\n\n try:\n latest_version = Version(match.group(\"version\"))\n except InvalidVersion as exc:\n msg = (\n f\"Could not parse version {match.group('version')!r} from 'pip index \"\n f\"versions' output for line:\\n {package_latest_version_line}.\\n\"\n f\"Exception: {exc}\"\n )\n LOGGER.error(msg)\n if fail_fast:\n sys.exit(f\"{Emoji.CROSS_MARK.value} {error_msg(msg)}\")\n print(error_msg(msg), file=sys.stderr, flush=True)\n error = True\n continue\n LOGGER.debug(\"Retrieved latest version: %r\", latest_version)\n\n # Here used to be a sanity check to ensure that the package name parsed from\n # pyproject.toml matches the name returned from 'pip index versions'.\n # But I cannot think of a reason why they would not match, so it has been\n # removed.\n # When checking 'pip index versions' output, it seems that the package name\n # returned is always the same as is used in the command call, e.g., if\n # 'pip index versions reQUEsts' is called, then the output will always be\n # 'reQUEsts (<latest version here>)'.\n\n # Check whether pyproject.toml already uses the latest version\n # This is expected if the latest version equals a specifier with any of the\n # operators: ==, >=, or ~=.\n split_latest_version = latest_version.base_version.split(\".\")\n _continue = False\n for specifier in parsed_requirement.specifier:\n if specifier.operator in [\"==\", \">=\", \"~=\"]:\n split_specifier_version = specifier.version.split(\".\")\n equal_length_latest_version = split_latest_version[\n : len(split_specifier_version)\n ]\n if equal_length_latest_version == split_specifier_version:\n LOGGER.debug(\n \"Package %r is already up-to-date. Specifiers: %s. \"\n \"Latest version: %s\",\n parsed_requirement.name,\n parsed_requirement.specifier,\n latest_version,\n )\n already_handled_packages.add(parsed_requirement)\n _continue = True\n if _continue:\n continue\n\n # Create ignore rules based on specifier set\n requirement_ignore_rules = create_ignore_rules(parsed_requirement.specifier)\n if requirement_ignore_rules[\"versions\"]:\n if parsed_requirement.name in ignore_rules:\n # Only \"versions\" key exists in requirement_ignore_rules\n if \"versions\" in ignore_rules[parsed_requirement.name]:\n ignore_rules[parsed_requirement.name][\"versions\"].extend(\n requirement_ignore_rules[\"versions\"]\n )\n else:\n ignore_rules[parsed_requirement.name].update(\n requirement_ignore_rules\n )\n else:\n ignore_rules[parsed_requirement.name] = requirement_ignore_rules\n LOGGER.debug(\n \"Created ignore rules (from specifier set): %s\",\n requirement_ignore_rules,\n )\n\n # Apply ignore rules\n if parsed_requirement.name in ignore_rules or \"*\" in ignore_rules:\n versions: IgnoreVersions = []\n update_types: IgnoreUpdateTypes = {}\n\n if \"*\" in ignore_rules:\n versions, update_types = parse_ignore_rules(ignore_rules[\"*\"])\n\n if parsed_requirement.name in ignore_rules:\n parsed_rules = parse_ignore_rules(ignore_rules[parsed_requirement.name])\n\n versions.extend(parsed_rules[0])\n update_types.update(parsed_rules[1])\n\n LOGGER.debug(\n \"Ignore rules:\\nversions: %s\\nupdate_types: %s\", versions, update_types\n )\n\n # Get \"current\" version from specifier set, i.e., the lowest allowed version\n # If a minimum version is not explicitly specified, use '0.0.0'\n for specifier in parsed_requirement.specifier:\n if specifier.operator in [\"==\", \">=\", \"~=\"]:\n current_version = specifier.version.split(\".\")\n break\n else:\n if latest_version.epoch == 0:\n current_version = \"0.0.0\".split(\".\")\n else:\n current_version = f\"{latest_version.epoch}!0.0.0\".split(\".\")\n\n if ignore_version(\n current=current_version,\n latest=split_latest_version,\n version_rules=versions,\n semver_rules=update_types,\n ):\n already_handled_packages.add(parsed_requirement)\n continue\n\n # Update specifier set to include the latest version.\n try:\n updated_specifier_set = update_specifier_set(\n latest_version=latest_version,\n current_specifier_set=parsed_requirement.specifier,\n )\n except UnableToResolve as exc:\n msg = (\n \"Could not determine how to update to the latest version using the \"\n f\"version range specifier set: {parsed_requirement.specifier}. \"\n f\"Package: {parsed_requirement.name}. Latest version: {latest_version}\"\n )\n LOGGER.error(\"%s. Exception: %s\", msg, exc)\n if fail_fast:\n sys.exit(f\"{Emoji.CROSS_MARK.value} {error_msg(msg)}\")\n print(error_msg(msg), file=sys.stderr, flush=True)\n already_handled_packages.add(parsed_requirement)\n error = True\n continue\n\n if not error:\n # Regenerate the full requirement string with the updated specifiers\n # Note: If any white space is present after the name (possibly incl.\n # extras) is reduced to a single space.\n match = re.search(rf\"{parsed_requirement.name}(?:\\[.*\\])?\\s+\", dependency)\n updated_dependency = regenerate_requirement(\n parsed_requirement,\n specifier=updated_specifier_set,\n post_name_space=bool(match),\n )\n LOGGER.debug(\"Updated dependency: %r\", updated_dependency)\n\n pattern_sub_line = re.escape(dependency)\n replacement_sub_line = updated_dependency.replace('\"', \"'\")\n\n LOGGER.debug(\"pattern_sub_line: %s\", pattern_sub_line)\n LOGGER.debug(\"replacement_sub_line: %s\", replacement_sub_line)\n\n # Update pyproject.toml\n update_file(pyproject_path, (pattern_sub_line, replacement_sub_line))\n already_handled_packages.add(parsed_requirement)\n updated_packages[parsed_requirement.name] = \",\".join(\n str(_)\n for _ in sorted(\n updated_specifier_set,\n key=lambda spec: spec.operator,\n reverse=True,\n )\n ) + (f\" ; {parsed_requirement.marker}\" if parsed_requirement.marker else \"\")\n\n if error:\n sys.exit(\n f\"{Emoji.CROSS_MARK.value} Errors occurred! See printed statements above.\"\n )\n\n if updated_packages:\n print(\n f\"{Emoji.PARTY_POPPER.value} Successfully updated the following \"\n \"dependencies:\\n\"\n + \"\\n\".join(\n f\" {package} ({version})\"\n for package, version in updated_packages.items()\n )\n + \"\\n\"\n )\n else:\n print(f\"{Emoji.CHECK_MARK.value} No dependency updates available.\")\nRelevant tools for printing to the console.
"},{"location":"api_reference/utils/console_printing/#ci_cd.utils.console_printing.Color","title":" Color (str, Enum) ","text":"ANSI escape sequences for colors.
Source code inci_cd/utils/console_printing.py class Color(str, Enum):\n \"\"\"ANSI escape sequences for colors.\"\"\"\n\n def __new__(cls, value: str) -> Self:\n obj = str.__new__(cls, value)\n obj._value_ = value\n return obj\n\n RED = \"\\033[91m\"\n GREEN = \"\\033[92m\"\n YELLOW = \"\\033[93m\"\n BLUE = \"\\033[94m\"\n MAGENTA = \"\\033[95m\"\n CYAN = \"\\033[96m\"\n WHITE = \"\\033[97m\"\n RESET = \"\\033[0m\"\n\n def write(self, text: str) -> str:\n \"\"\"Write the text with the color.\"\"\"\n return f\"{self.value}{text}{Color.RESET.value}\"\n Emoji (str, Enum) ","text":"Unicode strings for certain emojis.
Source code inci_cd/utils/console_printing.py class Emoji(str, Enum):\n \"\"\"Unicode strings for certain emojis.\"\"\"\n\n def __new__(cls, value: str) -> Self:\n obj = str.__new__(cls, value)\n if platform.system() == \"Windows\":\n # Windows does not support unicode emojis, so we replace them with\n # their corresponding unicode escape sequences\n obj._value_ = value.encode(\"unicode_escape\").decode(\"utf-8\")\n else:\n obj._value_ = value\n return obj\n\n PARTY_POPPER = \"\\U0001f389\"\n CHECK_MARK = \"\\u2714\"\n CROSS_MARK = \"\\u274c\"\n CURLY_LOOP = \"\\u27b0\"\n Formatting (str, Enum) ","text":"ANSI escape sequences for formatting.
Source code inci_cd/utils/console_printing.py class Formatting(str, Enum):\n \"\"\"ANSI escape sequences for formatting.\"\"\"\n\n def __new__(cls, value: str) -> Self:\n obj = str.__new__(cls, value)\n obj._value_ = value\n return obj\n\n BOLD = \"\\033[1m\"\n UNDERLINE = \"\\033[4m\"\n INVERT = \"\\033[7m\"\n RESET = \"\\033[0m\"\n\n def write(self, text: str) -> str:\n \"\"\"Write the text with the formatting.\"\"\"\n return f\"{self.value}{text}{Formatting.RESET.value}\"\nerror_msg(text)","text":"Write the text as an error message.
Source code inci_cd/utils/console_printing.py def error_msg(text: str) -> str:\n \"\"\"Write the text as an error message.\"\"\"\n return (\n f\"{Color.RED.write(Formatting.BOLD.write('ERROR'))}\"\n f\"{Color.RED.write(' - ' + text)}\"\n )\ninfo_msg(text)","text":"Write the text as an info message.
Source code inci_cd/utils/console_printing.py def info_msg(text: str) -> str:\n \"\"\"Write the text as an info message.\"\"\"\n return (\n f\"{Color.BLUE.write(Formatting.BOLD.write('INFO'))}\"\n f\"{Color.BLUE.write(' - ' + text)}\"\n )\nwarning_msg(text)","text":"Write the text as a warning message.
Source code inci_cd/utils/console_printing.py def warning_msg(text: str) -> str:\n \"\"\"Write the text as a warning message.\"\"\"\n return (\n f\"{Color.YELLOW.write(Formatting.BOLD.write('WARNING'))}\"\n f\"{Color.YELLOW.write(' - ' + text)}\"\n )\nUtilities for handling IO operations.
"},{"location":"api_reference/utils/file_io/#ci_cd.utils.file_io.update_file","title":"update_file(filename, sub_line, strip=None)","text":"Utility function for tasks to read, update, and write files
Source code inci_cd/utils/file_io.py def update_file(\n filename: Path, sub_line: tuple[str, str], strip: str | None = None\n) -> None:\n \"\"\"Utility function for tasks to read, update, and write files\"\"\"\n if strip is None and filename.suffix == \".md\":\n # Keep special white space endings for markdown files\n strip = \"\\n\"\n lines = [\n re.sub(sub_line[0], sub_line[1], line.rstrip(strip))\n for line in filename.read_text(encoding=\"utf8\").splitlines()\n ]\n filename.write_text(\"\\n\".join(lines) + \"\\n\", encoding=\"utf8\")\nHandle versions.
"},{"location":"api_reference/utils/versions/#ci_cd.utils.versions.PART_TO_LENGTH_MAPPING","title":"PART_TO_LENGTH_MAPPING","text":"Mapping of version-style name to their number of version parts.
E.g., a minor version has two parts, so the length is 2.
IgnoreEntryPair (tuple) ","text":"A key/value-pair within an ignore entry.
Source code inci_cd/utils/versions.py class IgnoreEntryPair(NamedTuple):\n \"\"\"A key/value-pair within an ignore entry.\"\"\"\n\n key: Literal[\"dependency-name\", \"versions\", \"update-types\"]\n value: str\n__getnewargs__(self) special","text":"Return self as a plain tuple. Used by copy and pickle.
Source code inci_cd/utils/versions.py def __getnewargs__(self):\n 'Return self as a plain tuple. Used by copy and pickle.'\n return _tuple(self)\n__new__(_cls, key, value) special staticmethod","text":"Create new instance of IgnoreEntryPair(key, value)
"},{"location":"api_reference/utils/versions/#ci_cd.utils.versions.IgnoreEntryPair.__repr__","title":"__repr__(self) special","text":"Return a nicely formatted representation string
Source code inci_cd/utils/versions.py def __repr__(self):\n 'Return a nicely formatted representation string'\n return self.__class__.__name__ + repr_fmt % self\n SemanticVersion (str) ","text":"A semantic version.
See SemVer.org for more information about semantic versioning.
The semantic version is in this invocation considered to build up in the following way:
<major>.<minor>.<patch>-<pre_release>+<build>\nWhere the names in carets are callable attributes for the instance.
When casting instances of SemanticVersion to str, the full version will be returned, i.e., as shown above, with a minimum of major.minor.patch.
For example, for the version 1.5, i.e., major=1, minor=5, the returned str representation will be the full major.minor.patch version: 1.5.0. The patch attribute will default to 0 while pre_release and build will be None, when asked for explicitly.
Precedence for comparing versions is done according to the rules outlined in point 11 of the specification found at SemVer.org.
Parameters:
Name Type Description Defaultmajor Union[str, int] The major version.
'' minor Optional[Union[str, int]] The minor version.
None patch Optional[Union[str, int]] The patch version.
None pre_release Optional[str] The pre-release part of the version, i.e., the part supplied after a minus (-), but before a plus (+).
None build Optional[str] The build metadata part of the version, i.e., the part supplied at the end of the version, after a plus (+).
None Attributes:
Name Type Descriptionmajor int The major version.
minor int The minor version.
patch int The patch version.
pre_release str The pre-release part of the version, i.e., the part supplied after a minus (-), but before a plus (+).
build str The build metadata part of the version, i.e., the part supplied at the end of the version, after a plus (+).
ci_cd/utils/versions.py class SemanticVersion(str):\n \"\"\"A semantic version.\n\n See [SemVer.org](https://semver.org) for more information about semantic\n versioning.\n\n The semantic version is in this invocation considered to build up in the following\n way:\n\n <major>.<minor>.<patch>-<pre_release>+<build>\n\n Where the names in carets are callable attributes for the instance.\n\n When casting instances of `SemanticVersion` to `str`, the full version will be\n returned, i.e., as shown above, with a minimum of major.minor.patch.\n\n For example, for the version `1.5`, i.e., `major=1, minor=5`, the returned `str`\n representation will be the full major.minor.patch version: `1.5.0`.\n The `patch` attribute will default to `0` while `pre_release` and `build` will be\n `None`, when asked for explicitly.\n\n Precedence for comparing versions is done according to the rules outlined in point\n 11 of the specification found at [SemVer.org](https://semver.org/#spec-item-11).\n\n Parameters:\n major (Union[str, int]): The major version.\n minor (Optional[Union[str, int]]): The minor version.\n patch (Optional[Union[str, int]]): The patch version.\n pre_release (Optional[str]): The pre-release part of the version, i.e., the\n part supplied after a minus (`-`), but before a plus (`+`).\n build (Optional[str]): The build metadata part of the version, i.e., the part\n supplied at the end of the version, after a plus (`+`).\n\n Attributes:\n major (int): The major version.\n minor (int): The minor version.\n patch (int): The patch version.\n pre_release (str): The pre-release part of the version, i.e., the part\n supplied after a minus (`-`), but before a plus (`+`).\n build (str): The build metadata part of the version, i.e., the part supplied at\n the end of the version, after a plus (`+`).\n\n \"\"\"\n\n _semver_regex = (\n r\"^(?P<major>0|[1-9]\\d*)(?:\\.(?P<minor>0|[1-9]\\d*))?(?:\\.(?P<patch>0|[1-9]\\d*))?\"\n r\"(?:-(?P<pre_release>(?:0|[1-9]\\d*|\\d*[a-zA-Z-][0-9a-zA-Z-]*)\"\n r\"(?:\\.(?:0|[1-9]\\d*|\\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?\"\n r\"(?:\\+(?P<build>[0-9a-zA-Z-]+(?:\\.[0-9a-zA-Z-]+)*))?$\"\n )\n \"\"\"The regular expression for a semantic version.\n See\n https://semver.org/#is-there-a-suggested-regular-expression-regex-to-check-a-semver-string.\"\"\"\n\n @no_type_check\n def __new__(cls, version: str | Version | None = None, **kwargs: str | int) -> Self:\n return super().__new__(\n cls, str(version) if version else cls._build_version(**kwargs)\n )\n\n def __init__(\n self,\n version: str | Version | None = None,\n *,\n major: str | int = \"\",\n minor: str | int | None = None,\n patch: str | int | None = None,\n pre_release: str | None = None,\n build: str | None = None,\n ) -> None:\n self._python_version: Version | None = None\n\n if version is not None:\n if major or minor or patch or pre_release or build:\n raise ValueError(\n \"version cannot be specified along with other parameters\"\n )\n\n if isinstance(version, Version):\n self._python_version = version\n version = \".\".join(str(_) for _ in version.release)\n\n match = re.match(self._semver_regex, version)\n if match is None:\n # Try to parse it as a Python version and try again\n try:\n _python_version = Version(version)\n except InvalidVersion as exc:\n raise ValueError(\n f\"version ({version}) cannot be parsed as a semantic version \"\n \"according to the SemVer.org regular expression\"\n ) from exc\n\n # Success. Now let's redo the SemVer.org regular expression match\n self._python_version = _python_version\n match = re.match(\n self._semver_regex,\n \".\".join(str(_) for _ in _python_version.release),\n )\n if match is None: # pragma: no cover\n # This should not really be possible at this point, as the\n # Version.releasethis is a guaranteed match.\n # But we keep it here for sanity's sake.\n raise ValueError(\n f\"version ({version}) cannot be parsed as a semantic version \"\n \"according to the SemVer.org regular expression\"\n )\n\n major, minor, patch, pre_release, build = match.groups()\n\n self._major = int(major)\n self._minor = int(minor) if minor else 0\n self._patch = int(patch) if patch else 0\n self._pre_release = pre_release if pre_release else None\n self._build = build if build else None\n\n @classmethod\n def _build_version(\n cls,\n major: str | int | None = None,\n minor: str | int | None = None,\n patch: str | int | None = None,\n pre_release: str | None = None,\n build: str | None = None,\n ) -> str:\n \"\"\"Build a version from the given parameters.\"\"\"\n if major is None:\n raise ValueError(\"At least major must be given\")\n version = str(major)\n if minor is not None:\n version += f\".{minor}\"\n if patch is not None:\n if minor is None:\n raise ValueError(\"Minor must be given if patch is given\")\n version += f\".{patch}\"\n if pre_release is not None:\n # semver spec #9: A pre-release version MAY be denoted by appending a\n # hyphen and a series of dot separated identifiers immediately following\n # the patch version.\n # https://semver.org/#spec-item-9\n if patch is None:\n raise ValueError(\"Patch must be given if pre_release is given\")\n version += f\"-{pre_release}\"\n if build is not None:\n # semver spec #10: Build metadata MAY be denoted by appending a plus sign\n # and a series of dot separated identifiers immediately following the patch\n # or pre-release version.\n # https://semver.org/#spec-item-10\n if patch is None:\n raise ValueError(\"Patch must be given if build is given\")\n version += f\"+{build}\"\n return version\n\n @property\n def major(self) -> int:\n \"\"\"The major version.\"\"\"\n return self._major\n\n @property\n def minor(self) -> int:\n \"\"\"The minor version.\"\"\"\n return self._minor\n\n @property\n def patch(self) -> int:\n \"\"\"The patch version.\"\"\"\n return self._patch\n\n @property\n def pre_release(self) -> str | None:\n \"\"\"The pre-release part of the version\n\n This is the part supplied after a minus (`-`), but before a plus (`+`).\n \"\"\"\n return self._pre_release\n\n @property\n def build(self) -> str | None:\n \"\"\"The build metadata part of the version.\n\n This is the part supplied at the end of the version, after a plus (`+`).\n \"\"\"\n return self._build\n\n @property\n def python_version(self) -> Version | None:\n \"\"\"The Python version as defined by `packaging.version.Version`.\"\"\"\n return self._python_version\n\n def as_python_version(self, shortened: bool = True) -> Version:\n \"\"\"Return the Python version as defined by `packaging.version.Version`.\"\"\"\n if not self.python_version:\n return Version(\n self.shortened()\n if shortened\n else \".\".join(str(_) for _ in (self.major, self.minor, self.patch))\n )\n\n # The SemanticVersion was generated from a Version. Return the original\n # epoch (and the rest, if the release equals the current version).\n\n # epoch\n redone_version = (\n f\"{self.python_version.epoch}!\" if self.python_version.epoch != 0 else \"\"\n )\n\n # release\n if shortened:\n redone_version += self.shortened()\n else:\n redone_version += \".\".join(\n str(_) for _ in (self.major, self.minor, self.patch)\n )\n\n if (self.major, self.minor, self.patch)[\n : len(self.python_version.release)\n ] == self.python_version.release:\n # The release is the same as the current version. Add the pre, post, dev,\n # and local parts, if any.\n if self.python_version.pre is not None:\n redone_version += \"\".join(str(_) for _ in self.python_version.pre)\n\n if self.python_version.post is not None:\n redone_version += f\".post{self.python_version.post}\"\n\n if self.python_version.dev is not None:\n redone_version += f\".dev{self.python_version.dev}\"\n\n if self.python_version.local is not None:\n redone_version += f\"+{self.python_version.local}\"\n\n return Version(redone_version)\n\n def __str__(self) -> str:\n \"\"\"Return the full version.\"\"\"\n if self.python_version:\n return str(self.as_python_version(shortened=False))\n return (\n f\"{self.major}.{self.minor}.{self.patch}\"\n f\"{f'-{self.pre_release}' if self.pre_release else ''}\"\n f\"{f'+{self.build}' if self.build else ''}\"\n )\n\n def __repr__(self) -> str:\n \"\"\"Return the string representation of the object.\"\"\"\n return f\"{self.__class__.__name__}({self.__str__()!r})\"\n\n def __getattribute__(self, name: str) -> Any:\n \"\"\"Return the attribute value.\"\"\"\n accepted_python_attributes = (\n \"epoch\",\n \"release\",\n \"pre\",\n \"post\",\n \"dev\",\n \"local\",\n \"public\",\n \"base_version\",\n \"micro\",\n )\n\n try:\n return object.__getattribute__(self, name)\n except AttributeError as exc:\n # Try returning the attribute from the Python version, if it is in a list\n # of accepted attributes\n if name not in accepted_python_attributes:\n raise AttributeError(\n f\"{self.__class__.__name__} object has no attribute {name!r}\"\n ) from exc\n\n python_version = object.__getattribute__(self, \"as_python_version\")(\n shortened=False\n )\n try:\n return getattr(python_version, name)\n except AttributeError as exc:\n raise AttributeError(\n f\"{self.__class__.__name__} object has no attribute {name!r}\"\n ) from exc\n\n def _validate_other_type(self, other: Any) -> SemanticVersion:\n \"\"\"Initial check/validation of `other` before rich comparisons.\"\"\"\n not_implemented_exc = NotImplementedError(\n f\"Rich comparison not implemented between {self.__class__.__name__} and \"\n f\"{type(other)}\"\n )\n\n if isinstance(other, self.__class__):\n return other\n\n if isinstance(other, (Version, str)):\n try:\n return self.__class__(other)\n except (TypeError, ValueError) as exc:\n raise not_implemented_exc from exc\n\n raise not_implemented_exc\n\n def __lt__(self, other: Any) -> bool:\n \"\"\"Less than (`<`) rich comparison.\"\"\"\n other_semver = self._validate_other_type(other)\n\n if self.major < other_semver.major:\n return True\n if self.major == other_semver.major:\n if self.minor < other_semver.minor:\n return True\n if self.minor == other_semver.minor:\n if self.patch < other_semver.patch:\n return True\n if self.patch == other_semver.patch:\n if self.pre_release is None:\n return False\n if other_semver.pre_release is None:\n return True\n return self.pre_release < other_semver.pre_release\n return False\n\n def __le__(self, other: Any) -> bool:\n \"\"\"Less than or equal to (`<=`) rich comparison.\"\"\"\n return self.__lt__(other) or self.__eq__(other)\n\n def __eq__(self, other: object) -> bool:\n \"\"\"Equal to (`==`) rich comparison.\"\"\"\n other_semver = self._validate_other_type(other)\n\n return (\n self.major == other_semver.major\n and self.minor == other_semver.minor\n and self.patch == other_semver.patch\n and self.pre_release == other_semver.pre_release\n )\n\n def __ne__(self, other: object) -> bool:\n \"\"\"Not equal to (`!=`) rich comparison.\"\"\"\n return not self.__eq__(other)\n\n def __ge__(self, other: Any) -> bool:\n \"\"\"Greater than or equal to (`>=`) rich comparison.\"\"\"\n return not self.__lt__(other)\n\n def __gt__(self, other: Any) -> bool:\n \"\"\"Greater than (`>`) rich comparison.\"\"\"\n return not self.__le__(other)\n\n def next_version(self, version_part: str) -> SemanticVersion:\n \"\"\"Return the next version for the specified version part.\n\n Parameters:\n version_part: The version part to increment.\n\n Returns:\n The next version.\n\n Raises:\n ValueError: If the version part is not one of `major`, `minor`, or `patch`.\n\n \"\"\"\n if version_part not in (\"major\", \"minor\", \"patch\"):\n raise ValueError(\n \"version_part must be one of 'major', 'minor', or 'patch', not \"\n f\"{version_part!r}\"\n )\n\n if version_part == \"major\":\n next_version = f\"{self.major + 1}.0.0\"\n elif version_part == \"minor\":\n next_version = f\"{self.major}.{self.minor + 1}.0\"\n else:\n next_version = f\"{self.major}.{self.minor}.{self.patch + 1}\"\n\n return self.__class__(next_version)\n\n def previous_version(\n self, version_part: str, max_filler: str | int | None = None\n ) -> SemanticVersion:\n \"\"\"Return the previous version for the specified version part.\n\n Parameters:\n version_part: The version part to decrement.\n max_filler: The maximum value for the version part to decrement.\n\n Returns:\n The previous version.\n\n Raises:\n ValueError: If the version part is not one of `major`, `minor`, or `patch`.\n\n \"\"\"\n if version_part not in (\"major\", \"minor\", \"patch\"):\n raise ValueError(\n \"version_part must be one of 'major', 'minor', or 'patch', not \"\n f\"{version_part!r}\"\n )\n\n if max_filler is None:\n max_filler = 99\n elif isinstance(max_filler, str):\n max_filler = int(max_filler)\n\n if not isinstance(max_filler, int):\n raise TypeError(\"max_filler must be an integer, string or None\")\n\n if version_part == \"major\":\n prev_version = f\"{self.major - 1}.{max_filler}.{max_filler}\"\n\n elif version_part == \"minor\" or self.patch == 0:\n prev_version = (\n f\"{self.major - 1}.{max_filler}.{max_filler}\"\n if self.minor == 0\n else f\"{self.major}.{self.minor - 1}.{max_filler}\"\n )\n\n else:\n prev_version = f\"{self.major}.{self.minor}.{self.patch - 1}\"\n\n return self.__class__(prev_version)\n\n def shortened(self) -> str:\n \"\"\"Return a shortened version of the version.\n\n The shortened version is the full version, but without the patch and/or minor\n version if they are `0`, and without the pre-release and build metadata parts.\n\n Returns:\n The shortened version.\n\n \"\"\"\n if self.patch == 0:\n if self.minor == 0:\n return str(self.major)\n return f\"{self.major}.{self.minor}\"\n return f\"{self.major}.{self.minor}.{self.patch}\"\nbuild: str | None property readonly","text":"The build metadata part of the version.
This is the part supplied at the end of the version, after a plus (+).
major: int property readonly","text":"The major version.
"},{"location":"api_reference/utils/versions/#ci_cd.utils.versions.SemanticVersion.minor","title":"minor: int property readonly","text":"The minor version.
"},{"location":"api_reference/utils/versions/#ci_cd.utils.versions.SemanticVersion.patch","title":"patch: int property readonly","text":"The patch version.
"},{"location":"api_reference/utils/versions/#ci_cd.utils.versions.SemanticVersion.pre_release","title":"pre_release: str | None property readonly","text":"The pre-release part of the version
This is the part supplied after a minus (-), but before a plus (+).
python_version: Version | None property readonly","text":"The Python version as defined by packaging.version.Version.
__eq__(self, other) special","text":"Equal to (==) rich comparison.
ci_cd/utils/versions.py def __eq__(self, other: object) -> bool:\n \"\"\"Equal to (`==`) rich comparison.\"\"\"\n other_semver = self._validate_other_type(other)\n\n return (\n self.major == other_semver.major\n and self.minor == other_semver.minor\n and self.patch == other_semver.patch\n and self.pre_release == other_semver.pre_release\n )\n__ge__(self, other) special","text":"Greater than or equal to (>=) rich comparison.
ci_cd/utils/versions.py def __ge__(self, other: Any) -> bool:\n \"\"\"Greater than or equal to (`>=`) rich comparison.\"\"\"\n return not self.__lt__(other)\n__getattribute__(self, name) special","text":"Return the attribute value.
Source code inci_cd/utils/versions.py def __getattribute__(self, name: str) -> Any:\n \"\"\"Return the attribute value.\"\"\"\n accepted_python_attributes = (\n \"epoch\",\n \"release\",\n \"pre\",\n \"post\",\n \"dev\",\n \"local\",\n \"public\",\n \"base_version\",\n \"micro\",\n )\n\n try:\n return object.__getattribute__(self, name)\n except AttributeError as exc:\n # Try returning the attribute from the Python version, if it is in a list\n # of accepted attributes\n if name not in accepted_python_attributes:\n raise AttributeError(\n f\"{self.__class__.__name__} object has no attribute {name!r}\"\n ) from exc\n\n python_version = object.__getattribute__(self, \"as_python_version\")(\n shortened=False\n )\n try:\n return getattr(python_version, name)\n except AttributeError as exc:\n raise AttributeError(\n f\"{self.__class__.__name__} object has no attribute {name!r}\"\n ) from exc\n__gt__(self, other) special","text":"Greater than (>) rich comparison.
ci_cd/utils/versions.py def __gt__(self, other: Any) -> bool:\n \"\"\"Greater than (`>`) rich comparison.\"\"\"\n return not self.__le__(other)\n__le__(self, other) special","text":"Less than or equal to (<=) rich comparison.
ci_cd/utils/versions.py def __le__(self, other: Any) -> bool:\n \"\"\"Less than or equal to (`<=`) rich comparison.\"\"\"\n return self.__lt__(other) or self.__eq__(other)\n__lt__(self, other) special","text":"Less than (<) rich comparison.
ci_cd/utils/versions.py def __lt__(self, other: Any) -> bool:\n \"\"\"Less than (`<`) rich comparison.\"\"\"\n other_semver = self._validate_other_type(other)\n\n if self.major < other_semver.major:\n return True\n if self.major == other_semver.major:\n if self.minor < other_semver.minor:\n return True\n if self.minor == other_semver.minor:\n if self.patch < other_semver.patch:\n return True\n if self.patch == other_semver.patch:\n if self.pre_release is None:\n return False\n if other_semver.pre_release is None:\n return True\n return self.pre_release < other_semver.pre_release\n return False\n__ne__(self, other) special","text":"Not equal to (!=) rich comparison.
ci_cd/utils/versions.py def __ne__(self, other: object) -> bool:\n \"\"\"Not equal to (`!=`) rich comparison.\"\"\"\n return not self.__eq__(other)\n__new__(cls, version=None, **kwargs) special staticmethod","text":"Create and return a new object. See help(type) for accurate signature.
Source code inci_cd/utils/versions.py @no_type_check\ndef __new__(cls, version: str | Version | None = None, **kwargs: str | int) -> Self:\n return super().__new__(\n cls, str(version) if version else cls._build_version(**kwargs)\n )\n__repr__(self) special","text":"Return the string representation of the object.
Source code inci_cd/utils/versions.py def __repr__(self) -> str:\n \"\"\"Return the string representation of the object.\"\"\"\n return f\"{self.__class__.__name__}({self.__str__()!r})\"\n__str__(self) special","text":"Return the full version.
Source code inci_cd/utils/versions.py def __str__(self) -> str:\n \"\"\"Return the full version.\"\"\"\n if self.python_version:\n return str(self.as_python_version(shortened=False))\n return (\n f\"{self.major}.{self.minor}.{self.patch}\"\n f\"{f'-{self.pre_release}' if self.pre_release else ''}\"\n f\"{f'+{self.build}' if self.build else ''}\"\n )\nas_python_version(self, shortened=True)","text":"Return the Python version as defined by packaging.version.Version.
ci_cd/utils/versions.py def as_python_version(self, shortened: bool = True) -> Version:\n \"\"\"Return the Python version as defined by `packaging.version.Version`.\"\"\"\n if not self.python_version:\n return Version(\n self.shortened()\n if shortened\n else \".\".join(str(_) for _ in (self.major, self.minor, self.patch))\n )\n\n # The SemanticVersion was generated from a Version. Return the original\n # epoch (and the rest, if the release equals the current version).\n\n # epoch\n redone_version = (\n f\"{self.python_version.epoch}!\" if self.python_version.epoch != 0 else \"\"\n )\n\n # release\n if shortened:\n redone_version += self.shortened()\n else:\n redone_version += \".\".join(\n str(_) for _ in (self.major, self.minor, self.patch)\n )\n\n if (self.major, self.minor, self.patch)[\n : len(self.python_version.release)\n ] == self.python_version.release:\n # The release is the same as the current version. Add the pre, post, dev,\n # and local parts, if any.\n if self.python_version.pre is not None:\n redone_version += \"\".join(str(_) for _ in self.python_version.pre)\n\n if self.python_version.post is not None:\n redone_version += f\".post{self.python_version.post}\"\n\n if self.python_version.dev is not None:\n redone_version += f\".dev{self.python_version.dev}\"\n\n if self.python_version.local is not None:\n redone_version += f\"+{self.python_version.local}\"\n\n return Version(redone_version)\nnext_version(self, version_part)","text":"Return the next version for the specified version part.
Parameters:
Name Type Description Defaultversion_part str The version part to increment.
requiredReturns:
Type DescriptionSemanticVersion The next version.
Exceptions:
Type DescriptionValueError If the version part is not one of major, minor, or patch.
ci_cd/utils/versions.py def next_version(self, version_part: str) -> SemanticVersion:\n \"\"\"Return the next version for the specified version part.\n\n Parameters:\n version_part: The version part to increment.\n\n Returns:\n The next version.\n\n Raises:\n ValueError: If the version part is not one of `major`, `minor`, or `patch`.\n\n \"\"\"\n if version_part not in (\"major\", \"minor\", \"patch\"):\n raise ValueError(\n \"version_part must be one of 'major', 'minor', or 'patch', not \"\n f\"{version_part!r}\"\n )\n\n if version_part == \"major\":\n next_version = f\"{self.major + 1}.0.0\"\n elif version_part == \"minor\":\n next_version = f\"{self.major}.{self.minor + 1}.0\"\n else:\n next_version = f\"{self.major}.{self.minor}.{self.patch + 1}\"\n\n return self.__class__(next_version)\nprevious_version(self, version_part, max_filler=None)","text":"Return the previous version for the specified version part.
Parameters:
Name Type Description Defaultversion_part str The version part to decrement.
requiredmax_filler str | int | None The maximum value for the version part to decrement.
None Returns:
Type DescriptionSemanticVersion The previous version.
Exceptions:
Type DescriptionValueError If the version part is not one of major, minor, or patch.
ci_cd/utils/versions.py def previous_version(\n self, version_part: str, max_filler: str | int | None = None\n) -> SemanticVersion:\n \"\"\"Return the previous version for the specified version part.\n\n Parameters:\n version_part: The version part to decrement.\n max_filler: The maximum value for the version part to decrement.\n\n Returns:\n The previous version.\n\n Raises:\n ValueError: If the version part is not one of `major`, `minor`, or `patch`.\n\n \"\"\"\n if version_part not in (\"major\", \"minor\", \"patch\"):\n raise ValueError(\n \"version_part must be one of 'major', 'minor', or 'patch', not \"\n f\"{version_part!r}\"\n )\n\n if max_filler is None:\n max_filler = 99\n elif isinstance(max_filler, str):\n max_filler = int(max_filler)\n\n if not isinstance(max_filler, int):\n raise TypeError(\"max_filler must be an integer, string or None\")\n\n if version_part == \"major\":\n prev_version = f\"{self.major - 1}.{max_filler}.{max_filler}\"\n\n elif version_part == \"minor\" or self.patch == 0:\n prev_version = (\n f\"{self.major - 1}.{max_filler}.{max_filler}\"\n if self.minor == 0\n else f\"{self.major}.{self.minor - 1}.{max_filler}\"\n )\n\n else:\n prev_version = f\"{self.major}.{self.minor}.{self.patch - 1}\"\n\n return self.__class__(prev_version)\nshortened(self)","text":"Return a shortened version of the version.
The shortened version is the full version, but without the patch and/or minor version if they are 0, and without the pre-release and build metadata parts.
Returns:
Type Descriptionstr The shortened version.
Source code inci_cd/utils/versions.py def shortened(self) -> str:\n \"\"\"Return a shortened version of the version.\n\n The shortened version is the full version, but without the patch and/or minor\n version if they are `0`, and without the pre-release and build metadata parts.\n\n Returns:\n The shortened version.\n\n \"\"\"\n if self.patch == 0:\n if self.minor == 0:\n return str(self.major)\n return f\"{self.major}.{self.minor}\"\n return f\"{self.major}.{self.minor}.{self.patch}\"\ncreate_ignore_rules(specifier_set)","text":"Create ignore rules based on version specifier set.
The only ignore rules needed are related to versions that should be explicitly avoided, i.e., the != operator. All other specifiers should require an explicit ignore rule by the user, if no update should be suggested.
ci_cd/utils/versions.py def create_ignore_rules(specifier_set: SpecifierSet) -> IgnoreRules:\n \"\"\"Create ignore rules based on version specifier set.\n\n The only ignore rules needed are related to versions that should be explicitly\n avoided, i.e., the `!=` operator. All other specifiers should require an explicit\n ignore rule by the user, if no update should be suggested.\n \"\"\"\n return {\n \"versions\": [\n f\"=={specifier.version}\"\n for specifier in specifier_set\n if specifier.operator == \"!=\"\n ]\n }\nfind_minimum_py_version(marker, project_py_version)","text":"Find the minimum Python version from a marker.
Source code inci_cd/utils/versions.py def find_minimum_py_version(marker: Marker, project_py_version: str) -> str:\n \"\"\"Find the minimum Python version from a marker.\"\"\"\n split_py_version = project_py_version.split(\".\")\n\n def _next_version(_version: SemanticVersion) -> SemanticVersion:\n if len(split_py_version) == PART_TO_LENGTH_MAPPING[\"major\"]:\n return _version.next_version(\"major\")\n if len(split_py_version) == PART_TO_LENGTH_MAPPING[\"minor\"]:\n return _version.next_version(\"minor\")\n return _version.next_version(\"patch\")\n\n min_py_version = SemanticVersion(project_py_version)\n\n environment_keys = default_environment().keys()\n empty_environment = {key: \"\" for key in environment_keys}\n python_version_centric_environment = empty_environment\n python_version_centric_environment.update({\"python_version\": min_py_version})\n\n while not _semi_valid_python_version(min_py_version) or not marker.evaluate(\n environment=python_version_centric_environment\n ):\n min_py_version = _next_version(min_py_version)\n python_version_centric_environment.update({\"python_version\": min_py_version})\n\n return min_py_version.shortened()\nget_min_max_py_version(requires_python)","text":"Get minimum or maximum Python version from requires_python.
This also works for parsing requirement markers specifying validity for a specific python_version.
A minimum version will be preferred.
This means all the minimum operators will be checked first, and if none of them match, then all the maximum operators will be checked. See below to understand which operators are minimum and which are maximum.
Note
The first operator to match will be used, so if a minimum operator matches, then the maximum operators will not be checked.
Whether it will minimum or maximum will depend on the operator: Minimum: >=, ==, ~=, > Maximum: <=, <
Returns:
Type Descriptionstr The minimum or maximum Python version. E.g., if requires_python is \">=3.6\", then \"3.6\" (min) will be returned, and if requires_python is Marker(\"python_version < '3.6'\"), then \"3.5.99\" (max) will be returned.
ci_cd/utils/versions.py def get_min_max_py_version(\n requires_python: str | Marker,\n) -> str:\n \"\"\"Get minimum or maximum Python version from `requires_python`.\n\n This also works for parsing requirement markers specifying validity for a specific\n `python_version`.\n\n _A minimum version will be preferred._\n\n This means all the minimum operators will be checked first, and if none of them\n match, then all the maximum operators will be checked.\n See below to understand which operators are minimum and which are maximum.\n\n Note:\n The first operator to match will be used, so if a minimum operator matches,\n then the maximum operators will not be checked.\n\n Whether it will minimum or maximum will depend on the operator:\n Minimum: `>=`, `==`, `~=`, `>`\n Maximum: `<=`, `<`\n\n Returns:\n The minimum or maximum Python version.\n E.g., if `requires_python` is `\">=3.6\"`, then `\"3.6\"` (min) will be returned,\n and if `requires_python` is `Marker(\"python_version < '3.6'\")`, then `\"3.5.99\"`\n (max) will be returned.\n\n \"\"\"\n if isinstance(requires_python, Marker):\n match = re.search(\n r\"python_version\\s*\"\n r\"(?P<operator>==|!=|<=|>=|<|>|~=)\\s*\"\n r\"('|\\\")(?P<version>[0-9]+(?:\\.[0-9]+)*)('|\\\")\",\n str(requires_python),\n )\n\n if match is None:\n raise UnableToResolve(\"Could not retrieve 'python_version' marker.\")\n\n requires_python = f\"{match.group('operator')}{match.group('version')}\"\n\n try:\n specifier_set = SpecifierSet(requires_python)\n except InvalidSpecifier as exc:\n raise UnableToResolve(\n \"Cannot parse 'requires_python' as a specifier set.\"\n ) from exc\n\n py_version = \"\"\n min_or_max = \"min\"\n\n length_to_part_mapping = {\n 1: \"major\",\n 2: \"minor\",\n 3: \"patch\",\n }\n\n # Minimum\n for specifier in specifier_set:\n if specifier.operator in [\">=\", \"==\", \"~=\"]:\n py_version = specifier.version\n break\n\n if specifier.operator == \">\":\n split_version = specifier.version.split(\".\")\n parsed_version = SemanticVersion(specifier.version)\n\n if len(split_version) == PART_TO_LENGTH_MAPPING[\"major\"]:\n py_version = str(parsed_version.next_version(\"major\").major)\n elif len(split_version) == PART_TO_LENGTH_MAPPING[\"minor\"]:\n py_version = \".\".join(\n parsed_version.next_version(\"minor\").split(\".\")[:2]\n )\n elif len(split_version) == PART_TO_LENGTH_MAPPING[\"patch\"]:\n py_version = str(parsed_version.next_version(\"patch\"))\n\n break\n else:\n # Maximum\n min_or_max = \"max\"\n\n for specifier in specifier_set:\n if specifier.operator == \"<=\":\n py_version = specifier.version\n break\n\n if specifier.operator == \"<\":\n split_version = specifier.version.split(\".\")\n parsed_version = SemanticVersion(specifier.version)\n\n if parsed_version == SemanticVersion(\"0\"):\n raise UnableToResolve(\n f\"{specifier} is not a valid Python version specifier.\"\n )\n\n if len(split_version) not in length_to_part_mapping:\n raise UnableToResolve(\n f\"{specifier} is not a valid Python version specifier. It was \"\n \"expected to be a major, minor, or patch version specifier.\"\n )\n\n # Fill with 0's and shorten. This is an attempt to return a full range\n # of previous versions.\n py_version = str(\n parsed_version.previous_version(\n length_to_part_mapping[len(split_version)],\n max_filler=0,\n ).shortened()\n )\n\n break\n else:\n raise UnableToResolve(\n \"Cannot determine min/max Python version from version specifier(s): \"\n f\"{specifier_set}\"\n )\n\n if py_version not in specifier_set:\n split_py_version = py_version.split(\".\")\n parsed_py_version = SemanticVersion(py_version)\n\n # See the _semi_valid_python_version() function for these values\n largest_value_for_a_patch_part = 18\n largest_value_for_a_minor_part = 12\n largest_value_for_a_major_part = 3\n largest_value_for_any_part = max(\n largest_value_for_a_patch_part,\n largest_value_for_a_minor_part,\n largest_value_for_a_major_part,\n )\n\n while (\n not _semi_valid_python_version(parsed_py_version)\n or py_version not in specifier_set\n ):\n if min_or_max == \"min\":\n if parsed_py_version.patch >= largest_value_for_a_patch_part:\n parsed_py_version = parsed_py_version.next_version(\"minor\")\n elif parsed_py_version.minor >= largest_value_for_a_minor_part:\n parsed_py_version = parsed_py_version.next_version(\"major\")\n else:\n parsed_py_version = parsed_py_version.next_version(\"patch\")\n else:\n parsed_py_version = parsed_py_version.previous_version(\n length_to_part_mapping[len(split_py_version)],\n max_filler=largest_value_for_any_part,\n )\n\n py_version = parsed_py_version.shortened()\n split_py_version = py_version.split(\".\")\n\n return py_version\nignore_version(current, latest, version_rules, semver_rules)","text":"Determine whether the latest version can be ignored.
Parameters:
Name Type Description Defaultcurrent list[str] The current version as a list of version parts. It's expected, but not required, the version is a semantic version.
requiredlatest list[str] The latest version as a list of version parts. It's expected, but not required, the version is a semantic version.
requiredversion_rules IgnoreVersions Version ignore rules.
requiredsemver_rules IgnoreUpdateTypes Semantic version ignore rules.
requiredReturns:
Type Descriptionbool Whether or not the latest version can be ignored based on the version and semantic version ignore rules.
Source code inci_cd/utils/versions.py def ignore_version(\n current: list[str],\n latest: list[str],\n version_rules: IgnoreVersions,\n semver_rules: IgnoreUpdateTypes,\n) -> bool:\n \"\"\"Determine whether the latest version can be ignored.\n\n Parameters:\n current: The current version as a list of version parts. It's expected, but not\n required, the version is a semantic version.\n latest: The latest version as a list of version parts. It's expected, but not\n required, the version is a semantic version.\n version_rules: Version ignore rules.\n semver_rules: Semantic version ignore rules.\n\n Returns:\n Whether or not the latest version can be ignored based on the version and\n semantic version ignore rules.\n\n \"\"\"\n # ignore all updates\n if not version_rules and not semver_rules:\n # A package name has been specified without specific rules, ignore all updates\n # for package.\n return True\n\n # version rules\n if _ignore_version_rules_specifier_set(latest, version_rules):\n return True\n\n # semver rules\n return bool(\n \"version-update\" in semver_rules\n and _ignore_semver_rules(current, latest, semver_rules)\n )\nparse_ignore_entries(entries, separator)","text":"Parser for the --ignore option.
The --ignore option values are given as key/value-pairs in the form: key=value...key=value. Here ... is the separator value supplied by --ignore-separator.
Parameters:
Name Type Description Defaultentries list[str] The list of supplied --ignore options.
separator str The supplied --ignore-separator value.
Returns:
Type DescriptionIgnoreRulesCollection A parsed mapping of dependencies to ignore rules.
Source code inci_cd/utils/versions.py def parse_ignore_entries(entries: list[str], separator: str) -> IgnoreRulesCollection:\n \"\"\"Parser for the `--ignore` option.\n\n The `--ignore` option values are given as key/value-pairs in the form:\n `key=value...key=value`. Here `...` is the separator value supplied by\n `--ignore-separator`.\n\n Parameters:\n entries: The list of supplied `--ignore` options.\n separator: The supplied `--ignore-separator` value.\n\n Returns:\n A parsed mapping of dependencies to ignore rules.\n\n \"\"\"\n ignore_entries: IgnoreRulesCollection = {}\n\n for entry in entries:\n pairs = entry.split(separator, maxsplit=2)\n for pair in pairs:\n if separator in pair:\n raise InputParserError(\n \"More than three key/value-pairs were given for an `--ignore` \"\n \"option, while there are only three allowed key names. Input \"\n f\"value: --ignore={entry!r}\"\n )\n\n ignore_entry: IgnoreEntry = {}\n for pair in pairs:\n match = re.match(\n r\"^(?P<key>dependency-name|versions|update-types)=(?P<value>.*)$\",\n pair,\n )\n if match is None:\n raise InputParserError(\n f\"Could not parse ignore configuration: {pair!r} (part of the \"\n f\"ignore option: {entry!r})\"\n )\n\n parsed_pair = IgnoreEntryPair(**match.groupdict()) # type: ignore[arg-type]\n\n if parsed_pair.key in ignore_entry:\n raise InputParserError(\n \"An ignore configuration can only be given once per option. The \"\n f\"configuration key {parsed_pair.key!r} was found multiple \"\n f\"times in the option {entry!r}\"\n )\n\n ignore_entry[parsed_pair.key] = parsed_pair.value.strip()\n\n if \"dependency-name\" not in ignore_entry:\n raise InputError(\n \"Ignore option entry missing required 'dependency-name' \"\n f\"configuration. Ignore option entry: {entry}\"\n )\n\n dependency_name = ignore_entry[\"dependency-name\"]\n if dependency_name not in ignore_entries:\n ignore_entries[dependency_name] = {\n key: [value]\n for key, value in ignore_entry.items()\n if key != \"dependency-name\"\n }\n else:\n for key, value in ignore_entry.items():\n if key != \"dependency-name\":\n ignore_entries[dependency_name][key].append(value)\n\n return ignore_entries\nparse_ignore_rules(rules)","text":"Parser for a specific set of ignore rules.
Parameters:
Name Type Description Defaultrules IgnoreRules A set of ignore rules for one or more packages.
requiredReturns:
Type Descriptiontuple[IgnoreVersions, IgnoreUpdateTypes] A tuple of the parsed 'versions' and 'update-types' entries as dictionaries.
Source code inci_cd/utils/versions.py def parse_ignore_rules(\n rules: IgnoreRules,\n) -> tuple[IgnoreVersions, IgnoreUpdateTypes]:\n \"\"\"Parser for a specific set of ignore rules.\n\n Parameters:\n rules: A set of ignore rules for one or more packages.\n\n Returns:\n A tuple of the parsed 'versions' and 'update-types' entries as dictionaries.\n\n \"\"\"\n if not rules:\n # Ignore package altogether\n return [{\"operator\": \">=\", \"version\": \"0\"}], {}\n\n versions: IgnoreVersions = []\n update_types: IgnoreUpdateTypes = {}\n\n if \"versions\" in rules:\n for versions_entry in rules[\"versions\"]:\n match = re.match(\n r\"^(?P<operator>>|<|<=|>=|==|!=|~=)\\s*(?P<version>\\S+)$\",\n versions_entry,\n )\n if match is None:\n raise InputParserError(\n \"Ignore option's 'versions' value cannot be parsed. It \"\n \"must be a single operator followed by a version number.\\n\"\n f\"Unparseable 'versions' value: {versions_entry!r}\"\n )\n versions.append(match.groupdict()) # type: ignore[arg-type]\n\n if \"update-types\" in rules:\n update_types[\"version-update\"] = []\n for update_type_entry in rules[\"update-types\"]:\n match = re.match(\n r\"^version-update:semver-(?P<semver_part>major|minor|patch)$\",\n update_type_entry,\n )\n if match is None:\n raise InputParserError(\n \"Ignore option's 'update-types' value cannot be parsed.\"\n \" It must be either: 'version-update:semver-major', \"\n \"'version-update:semver-minor' or \"\n \"'version-update:semver-patch'.\\nUnparseable 'update-types' \"\n f\"value: {update_type_entry!r}\"\n )\n update_types[\"version-update\"].append(match.group(\"semver_part\")) # type: ignore[arg-type]\n\n return versions, update_types\nregenerate_requirement(requirement, *, name=None, extras=None, specifier=None, url=None, marker=None, post_name_space=False)","text":"Regenerate a requirement string including the given parameters.
Parameters:
Name Type Description Defaultrequirement Requirement The requirement to regenerate and fallback to.
requiredname str | None A new name to use for the requirement.
None extras set[str] | None New extras to use for the requirement.
None specifier SpecifierSet | str | None A new specifier set to use for the requirement.
None url str | None A new URL to use for the requirement.
None marker Marker | str | None A new marker to use for the requirement.
None post_name_space bool Whether or not to add a single space after the name (possibly including extras), but before the specifier.
False Returns:
Type Descriptionstr The regenerated requirement string.
Source code inci_cd/utils/versions.py def regenerate_requirement(\n requirement: Requirement,\n *,\n name: str | None = None,\n extras: set[str] | None = None,\n specifier: SpecifierSet | str | None = None,\n url: str | None = None,\n marker: Marker | str | None = None,\n post_name_space: bool = False,\n) -> str:\n \"\"\"Regenerate a requirement string including the given parameters.\n\n Parameters:\n requirement: The requirement to regenerate and fallback to.\n name: A new name to use for the requirement.\n extras: New extras to use for the requirement.\n specifier: A new specifier set to use for the requirement.\n url: A new URL to use for the requirement.\n marker: A new marker to use for the requirement.\n post_name_space: Whether or not to add a single space after the name (possibly\n including extras), but before the specifier.\n\n Returns:\n The regenerated requirement string.\n\n \"\"\"\n updated_dependency = name or requirement.name\n\n if extras or requirement.extras:\n formatted_extras = \",\".join(sorted(extras or requirement.extras))\n updated_dependency += f\"[{formatted_extras}]\"\n\n if post_name_space:\n updated_dependency += \" \"\n\n if specifier or requirement.specifier:\n if specifier and not isinstance(specifier, SpecifierSet):\n specifier = SpecifierSet(specifier)\n updated_dependency += \",\".join(\n str(_)\n for _ in sorted(\n specifier or requirement.specifier,\n key=lambda spec: spec.operator,\n reverse=True,\n )\n )\n\n if url or requirement.url:\n updated_dependency += f\"@ {url or requirement.url}\"\n if marker or requirement.marker:\n updated_dependency += \" \"\n\n if marker or requirement.marker:\n updated_dependency += f\"; {marker or requirement.marker}\"\n\n return updated_dependency\nupdate_specifier_set(latest_version, current_specifier_set)","text":"Update the specifier set to include the latest version.
Source code inci_cd/utils/versions.py def update_specifier_set(\n latest_version: SemanticVersion | Version | str, current_specifier_set: SpecifierSet\n) -> SpecifierSet:\n \"\"\"Update the specifier set to include the latest version.\"\"\"\n logger = logging.getLogger(__name__)\n\n latest_version = SemanticVersion(latest_version)\n\n new_specifier_set = set(current_specifier_set)\n updated_specifiers = []\n split_latest_version = (\n latest_version.as_python_version(shortened=False).base_version.split(\".\")\n if latest_version.python_version\n else latest_version.split(\".\")\n )\n current_version_epochs = {Version(_.version).epoch for _ in current_specifier_set}\n\n logger.debug(\n \"Received latest version: %s and current specifier set: %s\",\n latest_version,\n current_specifier_set,\n )\n\n if latest_version in current_specifier_set:\n # The latest version is already included in the specifier set.\n # Update specifier set if the latest version is included via a `~=` or a `==`\n # operator.\n for specifier in current_specifier_set:\n if specifier.operator in [\"~=\", \"==\"]:\n split_specifier_version = specifier.version.split(\".\")\n updated_version = \".\".join(\n split_latest_version[: len(split_specifier_version)]\n )\n updated_specifiers.append(f\"{specifier.operator}{updated_version}\")\n new_specifier_set.remove(specifier)\n break\n else:\n # The latest version is already included in the specifier set, and the set\n # does not need updating. To communicate this, make updated_specifiers\n # non-empty, but include only an empty string.\n updated_specifiers.append(\"\")\n\n elif (\n latest_version.python_version\n and latest_version.as_python_version().epoch not in current_version_epochs\n ):\n # The latest version is *not* included in the specifier set.\n # And the latest version is NOT in the same epoch as the current version range.\n\n # Sanity check that the latest version's epoch is larger than the largest\n # epoch in the specifier set.\n if current_version_epochs and latest_version.as_python_version().epoch < max(\n current_version_epochs\n ):\n raise UnableToResolve(\n \"The latest version's epoch is smaller than the largest epoch in \"\n \"the specifier set.\"\n )\n\n # Simply add the latest version as a specifier.\n updated_specifiers.append(f\"=={latest_version}\")\n\n else:\n # The latest version is *not* included in the specifier set.\n # But we're in the right epoch.\n\n # Expect the latest version to be greater than the current version range.\n for specifier in current_specifier_set:\n # Simply expand the range if the version range is capped through a specifier\n # using either of the `<` or `<=` operators.\n if specifier.operator == \"<=\":\n split_specifier_version = specifier.version.split(\".\")\n updated_version = \".\".join(\n split_latest_version[: len(split_specifier_version)]\n )\n\n updated_specifiers.append(f\"{specifier.operator}{updated_version}\")\n new_specifier_set.remove(specifier)\n break\n\n if specifier.operator == \"<\":\n # Update to include latest version by upping to the next\n # version up from the latest version\n split_specifier_version = specifier.version.split(\".\")\n\n updated_version = \"\"\n\n # Add epoch if present\n if (\n latest_version.python_version\n and latest_version.as_python_version().epoch != 0\n ):\n updated_version += f\"{latest_version.as_python_version().epoch}!\"\n\n # Up only the last version segment of the latest version according to\n # what version segments are defined in the specifier version.\n if len(split_specifier_version) == PART_TO_LENGTH_MAPPING[\"major\"]:\n updated_version += str(latest_version.next_version(\"major\").major)\n elif len(split_specifier_version) == PART_TO_LENGTH_MAPPING[\"minor\"]:\n updated_version += \".\".join(\n latest_version.next_version(\"minor\").split(\".\")[:2]\n )\n elif len(split_specifier_version) == PART_TO_LENGTH_MAPPING[\"patch\"]:\n updated_version += latest_version.next_version(\"patch\")\n else:\n raise UnableToResolve(\n \"Invalid/unable to handle number of version parts: \"\n f\"{len(split_specifier_version)}\"\n )\n\n updated_specifiers.append(f\"{specifier.operator}{updated_version}\")\n new_specifier_set.remove(specifier)\n break\n\n if specifier.operator == \"~=\":\n # Expand and change ~= to >= and < operators if the latest version\n # changes major version. Otherwise, update to include latest version as\n # the minimum version\n current_version = SemanticVersion(specifier.version)\n\n # Add epoch if present\n epoch = \"\"\n if (\n latest_version.python_version\n and latest_version.as_python_version().epoch != 0\n ):\n epoch += f\"{latest_version.as_python_version().epoch}!\"\n\n if latest_version.major > current_version.major:\n # Expand and change ~= to >= and < operators\n\n # >= current_version (fully padded)\n updated_specifiers.append(f\">={current_version}\")\n\n # < next major version up from latest_version\n updated_specifiers.append(\n f\"<{epoch}{latest_version.next_version('major').major!s}\"\n )\n else:\n # Keep the ~= operator, but update to include the latest version as\n # the minimum version\n split_specifier_version = specifier.version.split(\".\")\n updated_version = \".\".join(\n split_latest_version[: len(split_specifier_version)]\n )\n updated_specifiers.append(f\"{specifier.operator}{updated_version}\")\n\n new_specifier_set.remove(specifier)\n break\n\n # Finally, add updated specifier(s) to new specifier set or raise.\n if updated_specifiers:\n # If updated_specifiers includes only an empty string, it means that the\n # current specifier set is valid as is and already includes the latest version\n if updated_specifiers != [\"\"]:\n # Otherwise, add updated specifier(s) to new specifier set\n new_specifier_set |= {Specifier(_) for _ in updated_specifiers}\n else:\n raise UnableToResolve(\n \"Cannot resolve how to update specifier set to include latest version.\"\n )\n\n return SpecifierSet(\",\".join(str(_) for _ in new_specifier_set))\npre-commit is an excellent tool for running CI tasks prior to committing new changes. The tasks are called \"hooks\" and are run in a separate virtual environment. The hooks usually change files in-place, meaning after pre-commit is run during a git commit command, the changed files can be reviewed and re-staged and committed.
Through SINTEF/ci-cd several hooks are available, mainly related to the GitHub Actions callable/reusable workflows that are also available in this repository.
This section contains all the available pre-commit hooks:
pyproject.tomlpre-commit hook id: docs-api-reference
Run this hook to update the API Reference section of your documentation.
The hook walks through a package directory, finding all Python files and creating a markdown file matching it along with recreating the Python API tree under the docs/api_reference/ folder.
The hook will run when any Python file is changed in the repository.
The hook expects the documentation to be setup with the MkDocs framework, including the mkdocstrings plugin for parsing in the Python class/function and method doc-strings, as well as the awesome-pages plugin for providing proper titles in the table-of-contents navigation.
"},{"location":"hooks/docs_api_reference/#using-it-together-with-cicd-workflows","title":"Using it together with CI/CD workflows","text":"If this hook is being used together with the workflow CI - Tests, to test if the documentation can be built, or CD - Release and/or CI/CD - New updates to default branch, to build and publish the documentation upon a release or push to the default branch, it is necessary to understand the way the Python API modules are referenced in the markdown files under docs/api_reference/.
By default, the references refer to the Python import path of a module. However, should a package be installed as an editable installation, i.e., using pip install -e, then the relative path from the repository root will be used.
This differentiation is only relevant for repositories, where these two cases are not aligned, such as when the Python package folder is in a nested folder, e.g., src/my_package/.
In order to remedy this, there are a single configuration in each workflow and this hooks that needs to be set to the same value. For this hook, the option name is --relative and the value for using the relative path, i.e., an editable installation, is to simply include this toggle option. If the option is not included, then a non-editable installation is assumed, i.e., the -e option is not used when installing the package, and a proper resolvable Python import statement is used as a link in the API reference markdown files. The latter is the default.
For the workflows, one should set the configuration option relative to true to use the relative path, i.e., an editable installation. And likewise set relative to false if a proper resolvable Python import statement is to be used, without forcing the -e option. The latter is the default.
It is required to specify the --package-dir argument at least once through the args key.
Otherwise, as noted above, without the proper framework, the created markdown files will not bring about the desired result in a built documentation.
"},{"location":"hooks/docs_api_reference/#options","title":"Options","text":"Any of these options can be given through the args key when defining the hook.
--package-dir Relative path to a package dir from the repository root, e.g., 'src/my_package'.This input option can be supplied multiple times. Yes string --docs-folder The folder name for the documentation root folder. No string docs --unwanted-folder A folder to avoid including into the Python API reference documentation. If this is not supplied, it will default to __pycache__.Note: Only folder names, not paths, may be included.Note: All folders and their contents with these names will be excluded.This input option can be supplied multiple times. No string __pycache__ --unwanted-file A file to avoid including into the Python API reference documentation. If this is not supplied, it will default to __init__.pyNote: Only full file names, not paths, may be included, i.e., filename + file extension.Note: All files with these names will be excluded.This input option can be supplied multiple times. No string __init__.py --full-docs-folder A folder in which to include everything - even those without documentation strings. This may be useful for a module full of data models or to ensure all class attributes are listed.This input option can be supplied multiple times. No string --full-docs-file A full relative path to a file in which to include everything - even those without documentation strings. This may be useful for a file full of data models or to ensure all class attributes are listed.This input option can be supplied multiple times. No string --special-option A combination of a relative path to a file and a fully formed mkdocstrings option that should be added to the generated MarkDown file. The combination should be comma-separated.Example: my_module/py_file.py,show_bases:false.Encapsulate the value in double quotation marks (\") if including spaces ( ).Important: If multiple package-dir options are supplied, the relative path MUST include/start with the package-dir value, e.g., \"my_package/my_module/py_file.py,show_bases: false\".This input option can be supplied multiple times. The options will be accumulated for the same file, if given several times. No string --relative Whether or not to use relative Python import links in the API reference markdown files. See section Using it together with CI/CD workflows above. No flag --debug Whether or not to print debug statements. No flag"},{"location":"hooks/docs_api_reference/#usage-example","title":"Usage example","text":"The following is an example of how an addition of the Update API Reference in Documentation hook into a .pre-commit-config.yaml file may look. It is meant to be complete as is.
repos:\n - repo: https://github.com/SINTEF/ci-cd\n rev: v2.8.3\n hooks:\n - id: docs-api-reference\n args:\n - --package-dir\n - src/my_python_package\n - --package-dir\n - src/my_other_python_package\n - --full-docs-folder\n - models\n - --full-docs-folder\n - data\npre-commit hook id: docs-landing-page
Run this hook to update the landing page (root index.md file) for your documentation.
The hook copies the root README.md file into the root of your documentation folder, renaming it to index.md and implementing any replacements specified.
The hook will run when the root README.md file is changed in the repository.
The hook expects the documentation to be a framework that can build markdown files for deploying a documentation site.
"},{"location":"hooks/docs_landing_page/#expectations","title":"Expectations","text":"It is required that the root README.md exists and the documentation's landing page is named index.md and can be found in the root of the documentation folder.
Any of these options can be given through the args key when defining the hook.
--docs-folder The folder name for the documentation root folder. No string docs --replacement A replacement (mapping) to be performed on README.md when creating the documentation's landing page (index.md). This list always includes replacing '--docs-folder/' with an empty string, in order to correct relative links. By default the value (LICENSE),(LICENSE.md) is set, but this will be overwritten if args is set.This input option can be supplied multiple times. No string (LICENSE),(LICENSE.md) --replacement-separator String to separate a replacement's 'old' to 'new' parts. Defaults to a comma (,). No string ,"},{"location":"hooks/docs_landing_page/#usage-example","title":"Usage example","text":"The following is an example of how an addition of the Update Landing Page (index.md) for Documentation hook into a .pre-commit-config.yaml file may look. It is meant to be complete as is.
repos:\n - repo: https://github.com/SINTEF/ci-cd\n rev: v2.8.3\n hooks:\n - id: docs-landing-page\n args:\n # Replace `(LICENSE)` with `(LICENSE.md)` (i.e., don't overwrite the default)\n - '--replacement'\n - '(LICENSE);(LICENSE.md)'\n # Replace `(tools/` with `(`\n - '--replacement'\n - '(tools/;('\n - '--replacement-separator'\n - ';'\npyproject.toml","text":"pre-commit hook id: update-pyproject
Run this hook to update the dependencies in your pyproject.toml file.
The hook utilizes pip index versions to determine the latest version available for all required and optional dependencies listed in your pyproject.toml file. It checks this based on the Python version listed as the minimum supported Python version by the package (defined through the requires-python key in your pyproject.toml file).
To ignore or configure how specific dependencies should be updated, the --ignore argument option can be utilized. This is done by specifying a line per dependency that contains --ignore-separator-separated (defaults to ellipsis (...)) key/value-pairs of:
dependency-name Ignore updates for dependencies with matching names, optionally using * to match zero or more characters. versions Ignore specific versions or ranges of versions. Examples: ~=1.0.5, >= 1.0.5,<2, >=0.1.1. update-types Ignore types of updates, such as SemVer major, minor, patch updates on version updates (for example: version-update:semver-patch will ignore patch updates). This can be combined with dependency-name=* to ignore particular update-types for all dependencies. Supported update-types values
Currently, only version-update:semver-major, version-update:semver-minor, and version-update:semver-patch are supported options for update-types.
The --ignore option is essentially similar to the ignore option of Dependabot. If versions and update-types are used together, they will both be respected jointly.
Here are some examples of different values that may be given for the --ignore option that accomplishes different things:
Value: dependency-name=Sphinx...versions=>=4.5.0 Accomplishes: For Sphinx, ignore all updates for/from version 4.5.0 and up / keep the minimum version for Sphinx at 4.5.0.
Value: dependency-name=pydantic...update-types=version-update:semver-patch Accomplishes: For pydantic, ignore all patch updates.
Value: dependency-name=numpy Accomplishes: For NumPy, ignore any and all updates.
Below is a usage example, where some of the example values above are implemented.
"},{"location":"hooks/update_pyproject/#expectations","title":"Expectations","text":"It is required that the root pyproject.toml exists.
A minimum Python version for the Python package should be specified in the pyproject.toml file through the requires-python key.
An active internet connection and for PyPI not to be down.
"},{"location":"hooks/update_pyproject/#options","title":"Options","text":"Any of these options can be given through the args key when defining the hook.
--root-repo-path A resolvable path to the root directory of the repository folder, where the pyproject.toml file can be found. No string . --fail-fast Fail immediately if an error occurs. Otherwise, print and ignore all non-critical errors. No flag --ignore Ignore-rules based on the ignore config option of Dependabot.It should be of the format: key=value...key=value, i.e., an ellipsis (...) separator and then equal-sign-separated key/value-pairs.Alternatively, the --ignore-separator can be set to something else to overwrite the ellipsis.The only supported keys are: dependency-name, versions, and update-types.Can be supplied multiple times per dependency-name. No string --ignore-separator Value to use instead of ellipsis (...) as a separator in --ignore key/value-pairs. No string --verbose Whether or not to print debug statements. No flag --skip-unnormalized-python-package-names Whether to skip dependencies with unnormalized Python package names. Normalization is outlined here. No flag"},{"location":"hooks/update_pyproject/#usage-example","title":"Usage example","text":"The following is an example of how an addition of the Update dependencies in pyproject.toml hook into a .pre-commit-config.yaml file may look. It is meant to be complete as is.
repos:\n - repo: https://github.com/SINTEF/ci-cd\n rev: v2.8.3\n hooks:\n - id: update-pyproject\n args:\n - --fail-fast\n - --ignore-separator=//\n - --ignore\n - dependency-name=Sphinx//versions=>=4.5.0\n - --ignore\n - dependency-name=numpy\nThis section contains all the available callable/reusable workflows:
cd_release.yml)ci_automerge_prs.yml)ci_cd_updated_default_branch.yml))ci_check_pyproject_dependencies.yml)ci_tests.yml)ci_update_dependencies.yml)For inputs specifying single or multi-line input values, the following rules apply:
If only \"single\" is mentioned, it means that the input value must be a single line (the workflow might fail if it is not).
Note
There is currently no input parameter that is explicitly single line only. Instead, one should consider the input parameter to be single line only if it is not explicitly mentioned as multi-line.
If both \"single\" and \"multi-line\" is mentioned, it means that multiple values can be specified, but they must be separated either over several, separate lines or within a single line by a space.
Here are some examples:
"},{"location":"workflows/#multi-line-input","title":"Multi-line input","text":"Accepted input styles:
# Two separate version update changes:\nversion_update_changes: |\n \"file/path,pattern,replacement string\"\n \"another/file/path,pattern,replacement string\"\n\n# A single version update change\nversion_update_changes: |\n \"file/path,pattern,replacement string\"\n\n# A single version update change, different formatting for input\nversion_update_changes: \"file/path,pattern,replacement string\"\nDisallowed input styles:
# Two separate version update changes:\nversion_update_changes: \"file/path,pattern,replacement string another/file/path,pattern,replacement string\"\nAccepted input styles:
# A single git username:\ngit_username: \"Casper Welzel Andersen\"\n\n# A single git username, different formatting for input\ngit_username: |\n \"Casper Welzel Andersen\"\nDisallowed input styles:
# Two separate git usernames:\ngit_username: |\n \"Casper Welzel Andersen\"\n \"Francesca L. Bleken\"\n\n# Two separate git usernames, different formatting for input\ngit_username: \"Casper Welzel Andersen Francesca L. Bleken\"\nWarning
It is important to note that the disallowed examples will work without fault in this case (might not always be true for other parameters). But the git username will be a single string, combining the names in succession, instead of being two separate values.
"},{"location":"workflows/#single-or-multi-line-input","title":"Single or multi-line input","text":"Accepted input styles:
# A single system dependency:\nsystem_dependencies: \"graphviz\"\n\n# A single system dependency, different formatting for input\nsystem_dependencies: |\n \"graphviz\"\n\n# Two separate system dependencies:\nsystem_dependencies: |\n \"graphviz\"\n \"Sphinx\"\n\n# Two separate system dependencies, different formatting for input\nsystem_dependencies: \"graphviz Sphinx\"\nDisallowed input styles:
# Use of custom separator:\nsystem_dependencies: \"graphviz,Sphinx\"\nImportant
The default for publish_on_pypi has changed from true to false in version 2.8.0.
To keep using the previous behaviour, set publish_on_pypi: true in the workflow file.
This change has been introduced to push for the use of PyPI's Trusted Publisher feature, which is not yet supported by reusable/callable workflows.
See the Using PyPI's Trusted Publisher section for more information on how to migrate to this feature.
File to use: cd_release.yml
There are 2 jobs in this workflow, which run in sequence.
First, an update & publish job, which updates the version in the package's root __init__.py file through an Invoke task. The newly created tag (created due to the caller workflow running on.release.types.published) will be updated accordingly, as will the publish branch (defaults to main).
Secondly, a job to update the documentation is run, however, this can be deactivated. The job expects the documentation to be setup with either the mike+MkDocs+GitHub Pages framework or the Sphinx framework.
For more information about the specific changelog inputs, see the related changelog generator actually used, specifically the list of configuration options.
Note
Concerning the changelog generator, the specific input changelog_exclude_labels defaults to a list of different labels if not supplied, hence, if supplied, one might want to include these labels alongside any extra labels. The default value is given here as a help: 'duplicate,question,invalid,wontfix'
The changelog_exclude_tags_regex is also used to remove tags in a list of tags to consider when evaluating the \"previous version\". This is specifically for adding a changelog to the GitHub release body.
If used together with the Update API Reference in Documentation, please align the relative input with the --relative option, when running the hook. See the proper section to understand why and how these options and inputs should be aligned.
PyPI has introduced a feature called Trusted Publisher which allows for a more secure way of publishing packages using OpenID Connect (OIDC). This feature is not yet supported by reusable/callable workflows, but can be used by setting up a GitHub Action workflow in your repository that calls the cd_release.yml workflow in one job, setting the publish_on_pypi input to false and the upload_distribution input to true, and then using the uploaded artifact to publish the package to PyPI in a subsequent job.
In this way you can still benefit from the cd_release.yml dynamically updated workflow, while using PyPI's Trusted Publisher feature.
Info
The artifact name is statically set to dist. If the workflow is run multiple times, the artifact will be overwritten. Retention time for the artifact is kept at the GitHub default (currently 90 days).
Important
The id-token:write permission is required by the PyPI upload action for Trusted Publishers.
The following is an example of how a workflow may look that calls CD - Release and uses the uploaded built distribution artifact to publish the package to PyPI. Note, the non-default dists directory is chosen for the built distribution, and the artifact is downloaded to the my-dists directory.
name: CD - Publish\n\non:\n release:\n types:\n - published\n\njobs:\n build:\n name: Build distribution & publish documentation\n if: github.repository == 'SINTEF/my-python-package' && startsWith(github.ref, 'refs/tags/v')\n uses: SINTEF/ci-cd/.github/workflows/cd_release.yml@v2.8.3\n with:\n # General\n git_username: \"Casper Welzel Andersen\"\n git_email: \"CasperWA@github.com\"\n release_branch: stable\n\n # Build distribution\n python_package: true\n package_dirs: my_python_package\n install_extras: \"[dev,build]\"\n build_libs: build\n build_cmd: \"python -m build -o dists\"\n build_dir: dists\n publish_on_pypi: false\n upload_distribution: true\n\n # Publish documentation\n update_docs: true\n doc_extras: \"[docs]\"\n docs_framework: mkdocs\n\n secrets:\n PAT: ${{ secrets.PAT }}\n\n publish:\n name: Publish to PyPI\n needs: build\n runs-on: ubuntu-latest\n\n # Using environments is recommended by PyPI when using Trusted Publishers\n environment: release\n\n # The id-token:write permission is required by the PyPI upload action for\n # Trusted Publishers\n permissions:\n id-token: write\n\n steps:\n - name: Download distribution\n uses: actions/download-artifact@v4\n with:\n name: dist # The artifact will always be called 'dist'\n path: my-dists\n\n - name: Publish to PyPI\n uses: pypa/gh-action-pypi-publish@release/v1\n with:\n # The path to the distribution to upload\n package-dir: my-dists\nThe content of repository files can be updated to use the new version where necessary. This is done through the version_update_changes (and version_update_changes_separator) inputs.
To see an example of how to use the version_update_changes (and version_update_changes_separator) see for example the workflow used by the SINTEF/ci-cd repository calling the CD Release workflow.
Some notes to consider and respect when using version_update_changes are:
version_update_changes_separator applies to all lines given in version_update_changes, meaning it should be a character, or series of characters, which will not be part of the actual content.Specifically, concerning the 'raw' Python string 'pattern' the following applies:
\"). This is done by prefixing it with a backslash (\\): \\\".`).re library documentation for more information.Concerning the 'replacement string' part, the package_dirs input and full semantic version can be substituted in dynamically by wrapping either package_dir or version in curly braces ({}). Indeed, for the version, one can specify sub-parts of the version to use, e.g., if one desires to only use the major version, this can be done by using the major attribute: {version.major}. The full list of version attributes are: major, minor, patch, pre_release, and build. More can be used, e.g., to only insert the major.minor version: {version.major}.{version.minor}.
For the 'file path' part, package_dir wrapped in curly braces ({}) will also be substituted at run time with each line from the possibly multi-line package_dirs input. E.g., {package_dir}/__init__.py will become ci_cd/__init__.py if the package_dirs input was 'ci_cd'.
This workflow should only be used for releasing a single modern Python package.
The repository contains the following:
__init__.py file with __version__ defined.v, e.g., v1.0.0.The following inputs are general inputs for the workflow as a whole.
Name Description Required Default Typegit_username A git username (used to set the 'user.name' config option). Yes string git_email A git user's email address (used to set the 'user.email' config option). Yes string release_branch The branch name to release/publish from. Yes main string runner The runner to use for the workflow. Note, the callable workflow expects a Linux/Unix system.. No ubuntu-latest string install_extras Any extras to install from the local repository through 'pip'. Must be encapsulated in square parentheses ([]) and be separated by commas (,) without any spaces.Example: '[dev,release]'. No Empty string string relative Whether or not to use install the local Python package(s) as an editable. No false boolean test Whether to use the TestPyPI repository index instead of PyPI as well as output debug statements in both workflow jobs. No false boolean pip_index_url A URL to a PyPI repository index. No https://pypi.org/simple/ string pip_extra_index_urls A space-delimited string of URLs to additional PyPI repository indices. No Empty string string Inputs related to updating the version, building and releasing the Python package to PyPI.
Name Description Required Default Typepublish_on_pypi Whether or not to publish on PyPI.Note: This is only relevant if 'python_package' is 'true', which is the default.Important: The default has changed from true to false to push for the use of PyPI's Trusted Publisher feature.See the Using PyPI's Trusted Publisher section for more information on how to migrate to this feature. Yes (will be non-required in v2.9) false boolean python_package Whether or not this is a Python package, where the version should be updated in the 'package_dir'/__init__.py for the possibly several 'package_dir' lines given in the package_dirs input and a build and release to PyPI should be performed. No true boolean python_version_build The Python version to use for the workflow when building the package. No 3.9 string package_dirs A multi-line string of paths to Python package directories relative to the repository directory to have its __version__ value updated.Example: 'src/my_package'.Important: This is required if 'python_package' is 'true', which is the default.See also Single vs multi-line input. Yes (if 'python_package' is 'true') string version_update_changes A multi-line string of changes to be implemented in the repository files upon updating the version. The string should be made up of three parts: 'file path', 'pattern', and 'replacement string'. These are separated by the 'version_update_changes_separator' value.The 'file path' must always either be relative to the repository root directory or absolute.The 'pattern' should be given as a 'raw' Python string.See also Single vs multi-line input. No Empty string string version_update_changes_separator The separator to use for 'version_update_changes' when splitting the three parts of each string. No , string build_libs A space-separated list of packages to install via PyPI (pip install). No Empty string string build_cmd The package build command, e.g., 'flit build' or 'python -m build'. No python -m build --outdir dist . string build_dir The directory where the built distribution is located. This should reflect the directory used in the build command or by default by the build library. No dist string tag_message_file Relative path to a release tag message file from the root of the repository.Example: '.github/utils/release_tag_msg.txt'. No Empty string string changelog_exclude_tags_regex A regular expression matching any tags that should be excluded from the CHANGELOG.md. No Empty string string changelog_exclude_labels Comma-separated list of labels to exclude from the CHANGELOG.md. No Empty string string upload_distribution Whether or not to upload the built distribution as an artifact.Note: This is only relevant if 'python_package' is 'true', which is the default. No true boolean Inputs related to building and releasing the documentation in general.
Name Description Required Default Typeupdate_docs Whether or not to also run the 'docs' workflow job. No false boolean python_version_docs The Python version to use for the workflow when building the documentation. No 3.9 string doc_extras Any extras to install from the local repository through 'pip'. Must be encapsulated in square parentheses ([]) and be separated by commas (,) without any spaces.Note, if this is empty, 'install_extras' will be used as a fallback.Example: '[docs]'. No Empty string string docs_framework The documentation framework to use. This can only be either 'mkdocs' or 'sphinx'. No mkdocs string system_dependencies A single (space-separated) or multi-line string of Ubuntu APT packages to install prior to building the documentation.See also Single vs multi-line input. No Empty string string Inputs related only to the MkDocs framework.
Name Description Required Default Typemkdocs_update_latest Whether or not to update the 'latest' alias to point to release_branch. No true boolean Finally, inputs related only to the Sphinx framework.
Name Description Required Default Typesphinx-build_options Single (space-separated) or multi-line string of command-line options to use when calling sphinx-build.See also Single vs multi-line input. No Empty string string docs_folder The path to the root documentation folder relative to the repository root. No docs string build_target_folder The path to the target folder for the documentation build relative to the repository root. No site string"},{"location":"workflows/cd_release/#secrets","title":"Secrets","text":"Name Description Required PyPI_token A PyPI token for publishing the built package to PyPI.Important: This is required if both 'python_package' and 'publish_on_pypi' are 'true'. Both are 'true' by default. Yes (if 'python_package' and 'publish_on_pypi' are 'true') PAT A personal access token (PAT) with rights to update the release_branch. This will fallback on GITHUB_TOKEN. No"},{"location":"workflows/cd_release/#usage-example","title":"Usage example","text":"The following is an example of how a workflow may look that calls CD - Release. It is meant to be complete as is.
name: CD - Publish\n\non:\n release:\n types:\n - published\n\njobs:\n publish:\n name: Publish package and documentation\n uses: SINTEF/ci-cd/.github/workflows/cd_release.yml@v2.8.3\n if: github.repository == 'SINTEF/my-python-package' && startsWith(github.ref, 'refs/tags/v')\n with:\n # General\n git_username: \"Casper Welzel Andersen\"\n git_email: \"CasperWA@github.com\"\n release_branch: stable\n\n # Publish distribution\n package_dirs: my_python_package\n install_extras: \"[dev,build]\"\n build_cmd: \"pip install flit && flit build\"\n tag_message_file: \".github/utils/release_tag_msg.txt\"\n changelog_exclude_labels: \"skip_changelog,duplicate\"\n publish_on_pypi: true\n\n # Publish documentation\n update_docs: true\n doc_extras: \"[docs]\"\n secrets:\n PyPI_token: ${{ secrets.PYPI_TOKEN }}\n PAT: ${{ secrets.PAT }}\nFile to use: ci_automerge_prs.yml
Activate auto-merging for a PR.
It is possible to introduce changes to the PR head branch prior to activating the auto-merging, if so desired. This is done by setting perform_changes to 'true' and setting the other inputs accordingly, as they are now required. See Inputs below for a full overview of the available inputs.
The changes input can be both a path to a bash file that should be run, or a multi-line string of bash commands to run. Afterwards any and all changes in the repository will be committed and pushed to the PR head branch.
The motivation for being able to run changes prior to auto-merging, is to update or affect the repository files according to the specific PR being auto-merged. Usually auto-merging is activated for dependabot branches, i.e., when a dependency/requirement is updated. Hence, the changes could include updating this dependency in documentation files or similar, where it will not be updated otherwise.
PR branch name
The generated branch for the PR will be named ci/update-pyproject.
The PAT secret must represent a user with the rights to activate auto-merging.
This workflow can only be called if the triggering event from the caller workflow is pull_request_target.
runner The runner to use for the workflow. Note, the callable workflow expects a Linux/Unix system.. No ubuntu-latest string perform_changes Whether or not to perform and commit changes to the PR branch prior to activating auto-merge. No boolean git_username A git username (used to set the 'user.name' config option).Required if perform_changes is 'true'. No string git_email A git user's email address (used to set the 'user.email' config option).Required if perform_changes is 'true'. No string changes A file to run in the local repository (relative path from the root of the repository) or a multi-line string of bash commands to run.Required if perform_changes is 'true'.See also Single vs multi-line input. No string"},{"location":"workflows/ci_automerge_prs/#secrets","title":"Secrets","text":"Name Description Required PAT A personal access token (PAT) with rights to activate auto-merging. This will fallback on GITHUB_TOKEN. No"},{"location":"workflows/ci_automerge_prs/#usage-example","title":"Usage example","text":"The following is an example of how a workflow may look that calls CI - Activate auto-merging for PRs. It is meant to be complete as is.
name: CI - Activate auto-merging for Dependabot PRs\n\non:\n pull_request_target:\n branches:\n - ci/dependency-updates\n\njobs:\n update-dependency-branch:\n name: Call external workflow\n uses: SINTEF/ci-cd/.github/workflows/ci_automerge_prs.yml@v2.8.3\n if: github.repository_owner == 'SINTEF' && ( ( startsWith(github.event.pull_request.head.ref, 'dependabot/') && github.actor == 'dependabot[bot]' ) || ( github.event.pull_request.head.ref == 'ci/update-pyproject' && github.actor == 'CasperWA' ) )\n secrets:\n PAT: ${{ secrets.RELEASE_PAT }}\nA couple of usage examples when adding changes:
Here, referencing a bash script file for the changes.
name: CI - Activate auto-merging for Dependabot PRs\n\non:\n pull_request_target:\n branches:\n - ci/dependency-updates\n\njobs:\n update-dependency-branch:\n name: Call external workflow\n uses: SINTEF/ci-cd/.github/workflows/ci_automerge_prs.yml@v2.8.3\n if: github.repository_owner == 'SINTEF' && ( ( startsWith(github.event.pull_request.head.ref, 'dependabot/') && github.actor == 'dependabot[bot]' ) || ( github.event.pull_request.head.ref == 'ci/update-pyproject' && github.actor == 'CasperWA' ) )\n with:\n perform_changes: true\n git_username: \"Casper Welzel Andersen\"\n git_email: \"CasperWA@github.com\"\n changes: \".ci/pre_automerge.sh\"\n secrets:\n PAT: ${{ secrets.RELEASE_PAT }}\nHere, writing out the changes explicitly in the job.
name: CI - Activate auto-merging for Dependabot PRs\n\non:\n pull_request_target:\n branches:\n - ci/dependency-updates\n\njobs:\n update-dependency-branch:\n name: Call external workflow\n uses: SINTEF/ci-cd/.github/workflows/ci_automerge_prs.yml@v2.8.3\n if: github.repository_owner == 'SINTEF' && ( ( startsWith(github.event.pull_request.head.ref, 'dependabot/') && github.actor == 'dependabot[bot]' ) || ( github.event.pull_request.head.ref == 'ci/update-pyproject' && github.actor == 'CasperWA' ) )\n with:\n perform_changes: true\n git_username: \"Casper Welzel Andersen\"\n git_email: \"CasperWA@github.com\"\n changes: |\n PYTHON=\"$(python --version || :)\"\n if [ -z \"${PYTHON}\" ]; then\n echo \"Python not detected on the system.\"\n exit 1\n fi\n\n PIP=\"$(python -m pip --version || :)\"\n if [ -z \"${PIP}\" ]; then\n echo \"pip not detected to be installed for ${PYTHON}.\"\n exit 1\n fi\n\n echo \"Python: ${PYTHON}\"\n echo \"pip: ${PIP}\"\n\n python -m pip install -U pip\n pip install -U setuptools wheel\n pip install pre-commit\n\n pre-commit autoupdate\n pre-commit run --all-files || :\n secrets:\n PAT: ${{ secrets.RELEASE_PAT }}\nFile to use: ci_cd_updated_default_branch.yml
Keep your permanent_dependencies_branch branch up-to-date with changes in your main development branch, i.e., the default_repo_branch.
Furthermore, this workflow can optionally update the latest mike+MkDocs+GitHub Pages-framework documentation release alias, which represents the default_repo_branch. The workflow also alternatively supports the Sphinx framework.
Warning
If a PAT is not passed through for the PAT secret and GITHUB_TOKEN is used, beware that any other CI/CD jobs that run for, e.g., pull request events, may not run since GITHUB_TOKEN-generated PRs are designed to not start more workflows to avoid escalation. Hence, if it is important to run CI/CD workflows for pull requests, consider passing a PAT as a secret to this workflow represented by the PAT secret.
Important
If this is to be used together with the CI - Update dependencies PR workflow, the pr_body_file supplied to that workflow (if any) should match the update_depednencies_pr_body_file input in this workflow and be immutable within the first 8 lines, i.e., no check boxes or similar in the first 8 lines. Indeed, it is recommended to not supply pr_body_file to the CI - Update dependencies PR workflow as well as to not supply the update_dependencies_pr_body_file in this workflow in this case.
Note
Concerning the changelog generator, the specific input changelog_exclude_labels defaults to a list of different labels if not supplied, hence, if supplied, one might want to include these labels alongside any extra labels. The default value is given here as a help: 'duplicate,question,invalid,wontfix'
If used together with the Update API Reference in Documentation, please align the relative input with the --relative option, when running the hook. See the proper section to understand why and how these options and inputs should be aligned.
The repository contains the following:
package_dirs input.docs directory.README.md file must exist and desired to be used as the documentation's landing page if the update_docs_landing_page is set to true, which is the default.The following inputs are general inputs for the workflow as a whole.
Name Description Required Default Typegit_username A git username (used to set the 'user.name' config option). Yes string git_email A git user's email address (used to set the 'user.email' config option). Yes string runner The runner to use for the workflow. Note, the callable workflow expects a Linux/Unix system.. No ubuntu-latest string default_repo_branch The branch name of the repository's default branch. More specifically, the branch the PR should target. No main string test Whether to do a \"dry run\", i.e., run the workflow, but avoid pushing to 'permanent_dependencies_branch' branch and deploying documentation (if 'update_docs' is 'true'). No false boolean pip_index_url A URL to a PyPI repository index. No https://pypi.org/simple/ string pip_extra_index_urls A space-delimited string of URLs to additional PyPI repository indices. No Empty string string Inputs related to updating the permanent dependencies branch.
Name Description Required Default Typeupdate_dependencies_branch Whether or not to update the permanent dependencies branch. No true boolean permanent_dependencies_branch The branch name for the permanent dependency updates branch. No ci/dependency-updates string update_dependencies_pr_body_file Relative path to a PR body file from the root of the repository, which is used in the 'CI - Update dependencies PR' workflow, if used.Example: '.github/utils/pr_body_update_deps.txt'. No Empty string string Inputs related to building and releasing the documentation.
Name Description Required Default Typeupdate_docs Whether or not to also run the 'docs' workflow job. No false boolean update_python_api_ref Whether or not to update the Python API documentation reference.Note: If this is 'true', 'package_dirs' is required. No true boolean package_dirs A multi-line string of paths to Python package directories relative to the repository directory to be considered for creating the Python API reference documentation.Example: 'src/my_package'.Important: This is required if 'update_docs' and 'update_python_api_ref' are 'true'.See also Single vs multi-line input. Yes (if 'update_docs' and 'update_python_api_ref' are 'true') string update_docs_landing_page Whether or not to update the documentation landing page. The landing page will be based on the root README.md file. No true boolean python_version The Python version to use for the workflow.Note: This is only relevant if update_pre-commit is true. No 3.9 string doc_extras Any extras to install from the local repository through 'pip'. Must be encapsulated in square parentheses ([]) and be separated by commas (,) without any spaces.Example: '[docs]'. No Empty string string relative Whether or not to use install the local Python package(s) as an editable. No false boolean exclude_dirs A multi-line string of directories to exclude in the Python API reference documentation. Note, only directory names, not paths, may be included. Note, all folders and their contents with these names will be excluded. Defaults to '__pycache__'.Important: When a user value is set, the preset value is overwritten - hence '__pycache__' should be included in the user value if one wants to exclude these directories.See also Single vs multi-line input. No __pycache__ string exclude_files A multi-line string of files to exclude in the Python API reference documentation. Note, only full file names, not paths, may be included, i.e., filename + file extension. Note, all files with these names will be excluded. Defaults to '__init__.py'.Important: When a user value is set, the preset value is overwritten - hence '__init__.py' should be included in the user value if one wants to exclude these files.See also Single vs multi-line input. No __init__.py string full_docs_dirs A multi-line string of directories in which to include everything - even those without documentation strings. This may be useful for a module full of data models or to ensure all class attributes are listed.See also Single vs multi-line input. No Empty string string full_docs_files A multi-line string of relative paths to files in which to include everything - even those without documentation strings. This may be useful for a file full of data models or to ensure all class attributes are listed.See also Single vs multi-line input. No Empty string string special_file_api_ref_options A multi-line string of combinations of a relative path to a Python file and a fully formed mkdocstrings option that should be added to the generated MarkDown file for the Python API reference documentation.Example: my_module/py_file.py,show_bases:false.Encapsulate the value in double quotation marks (\") if including spaces ( ).Important: If multiple package_dirs are supplied, the relative path MUST include/start with the appropriate 'package_dir' value, e.g., \"my_package/my_module/py_file.py,show_bases: false\".See also Single vs multi-line input. No Empty string string landing_page_replacements A multi-line string of replacements (mappings) to be performed on README.md when creating the documentation's landing page (index.md). This list always includes replacing 'docs/' with an empty string to correct relative links, i.e., this cannot be overwritten. By default '(LICENSE)' is replaced by '(LICENSE.md)'.See also Single vs multi-line input. No (LICENSE),(LICENSE.md) string landing_page_replacement_separator String to separate a mapping's 'old' to 'new' parts. Defaults to a comma (,). No , string changelog_exclude_tags_regex A regular expression matching any tags that should be excluded from the CHANGELOG.md. No Empty string string changelog_exclude_labels Comma-separated list of labels to exclude from the CHANGELOG.md. No Empty string string docs_framework The documentation framework to use. This can only be either 'mkdocs' or 'sphinx'. No mkdocs string system_dependencies A single (space-separated) or multi-line string of Ubuntu APT packages to install prior to building the documentation.See also Single vs multi-line input. No Empty string string Finally, inputs related only to the Sphinx framework when building and releasing the documentation.
Name Description Required Default Typesphinx-build_options Single (space-separated) or multi-line string of command-line options to use when calling sphinx-build.See also Single vs multi-line input. No Empty string string docs_folder The path to the root documentation folder relative to the repository root. No docs string build_target_folder The path to the target folder for the documentation build relative to the repository root. No site string"},{"location":"workflows/ci_cd_updated_default_branch/#secrets","title":"Secrets","text":"Name Description Required PAT A personal access token (PAT) with rights to update the permanent_dependencies_branch. This will fallback on GITHUB_TOKEN. No"},{"location":"workflows/ci_cd_updated_default_branch/#usage-example","title":"Usage example","text":"The following is an example of how a workflow may look that calls CI/CD - New updates to default branch. It is meant to be complete as is.
name: CI - Activate auto-merging for Dependabot PRs\n\non:\n push:\n branches:\n - stable\n\njobs:\n updates-to-stable:\n name: Call external workflow\n uses: SINTEF/ci-cd/.github/workflows/ci_cd_updated_default_branch.yml@v2.8.3\n if: github.repository_owner == 'SINTEF'\n with:\n git_username: \"Casper Welzel Andersen\"\n git_email: \"CasperWA@github.com\"\n default_repo_branch: stable\n permanent_dependencies_branch: \"ci/dependency-updates\"\n update_docs: true\n package_dirs: |\n my_python_package\n my_other_python_package\n doc_extras: \"[docs]\"\n exclude_files: __init__.py,config.py\n full_docs_dirs: models\n landing_page_replacements: \"(LICENSE);(LICENSE.md)|(tools);(../tools)\"\n landing_page_replacements_mapping_separator: \";\"\n secrets:\n PAT: ${{ secrets.PAT }}\nFile to use: ci_check_pyproject_dependencies.yml
This workflow runs an Invoke task to check dependencies in a pyproject.toml file.
The reason for having this workflow and not using Dependabot is because it seems to not function properly with this use case.
Warning
If a PAT is not passed through for the PAT secret and GITHUB_TOKEN is used, beware that any other CI/CD jobs that run for, e.g., pull request events, may not run since GITHUB_TOKEN-generated PRs are designed to not start more workflows to avoid escalation. Hence, if it is important to run CI/CD workflows for pull requests, consider passing a PAT as a secret to this workflow represented by the PAT secret.
Info
The generated PR will be created from a new branch named ci/update-pyproject. If you wish to change this value, see the branch_name_extension input option.
To ignore or configure how specific dependencies should be updated, the ignore input option can be utilized. This is done by specifying a line per dependency that contains ellipsis-separated (...) key/value-pairs of:
dependency-name Ignore updates for dependencies with matching names, optionally using * to match zero or more characters. versions Ignore specific versions or ranges of versions. Examples: ~=1.0.5, >= 1.0.5,<2, >=0.1.1. update-types Ignore types of updates, such as SemVer major, minor, patch updates on version updates (for example: version-update:semver-patch will ignore patch updates). This can be combined with dependency-name=* to ignore particular update-types for all dependencies. Supported update-types values
Currently, only version-update:semver-major, version-update:semver-minor, and version-update:semver-patch are supported options for update-types.
The ignore option is essentially similar to the ignore option of Dependabot. If versions and update-types are used together, they will both be respected jointly.
Here is an example of different lines given as value for the ignore option that accomplishes different things:
# ...\njobs:\n check-dependencies:\n uses: SINTEF/ci-cd/.github/workflows/ci_check_pyproject_dependencies.yml@v2.8.3\n with:\n # ...\n # For Sphinx, ignore all updates for/from version 4.5.0 and up / keep the minimum version for Sphinx at 4.5.0.\n # For pydantic, ignore all patch updates\n # For numpy, ignore any and all updates\n ignore: |\n dependency-name=Sphinx...versions=>=4.5.0\n dependency-name=pydantic...update-types=version-update:semver-patch\n dependency-name=numpy\n# ...\nThe repository contains the following:
pyproject.toml file with the Python package's dependencies.git_username A git username (used to set the 'user.name' config option). Yes string git_email A git user's email address (used to set the 'user.email' config option). Yes string runner The runner to use for the workflow. Note, the callable workflow expects a Linux/Unix system.. No ubuntu-latest string target_branch The branch name for the target of the opened PR.Note: If a value is not given for this nor permanent_dependencies_branch, the default value for permanent_dependencies_branch will be used until v2.6.0, whereafter providing an explicit value for target_branch is required. No Empty string string permanent_dependencies_branch DEPRECATED - Will be removed in v2.6.0. Use target_branch instead.The branch name for the permanent dependency updates branch. No ci/dependency-updates string python_version The Python version to use for the workflow. No 3.9 string install_extras Any extras to install from the local repository through 'pip'. Must be encapsulated in square parentheses ([]) and be separated by commas (,) without any spaces.Example: '[dev,release]'. No Empty string string pip_index_url A URL to a PyPI repository index. No https://pypi.org/simple/ string pip_extra_index_urls A space-delimited string of URLs to additional PyPI repository indices. No Empty string string pr_body_file Relative path to PR body file from the root of the repository.Example: '.github/utils/pr_body_deps_check.txt'. No Empty string string fail_fast Whether the task to update dependencies should fail if any error occurs. No false boolean pr_labels A comma separated list of strings of GitHub labels to use for the created PR. No Empty string string ignore Create ignore conditions for certain dependencies. A multi-line string of ignore rules, where each line is an ellipsis-separated (...) string of key/value-pairs. One line per dependency. This option is similar to the ignore option of Dependabot.See also Single vs multi-line input. No Empty string string branch_name_extension A string to append to the branch name of the created PR. Example: '-my-branch'. It will be appended after a forward slash, so the final branch name will be ci/update-pyproject/-my-branch. No Empty string string debug Whether to run the workflow in debug mode, printing extra debug information. No false boolean skip_unnormalized_python_package_names Whether to skip dependencies with unnormalized Python package names. Normalization is outlined here. No false boolean"},{"location":"workflows/ci_check_pyproject_dependencies/#secrets","title":"Secrets","text":"Name Description Required PAT A personal access token (PAT) with rights to create PRs. This will fallback on GITHUB_TOKEN. No"},{"location":"workflows/ci_check_pyproject_dependencies/#usage-example","title":"Usage example","text":"The following is an example of how a workflow may look that calls CI - Check pyproject.toml dependencies. It is meant to be complete as is.
name: CI - Check dependencies\n\non:\n schedule:\n - cron: \"30 5 * * 1\"\n workflow_dispatch:\n\njobs:\n check-dependencies:\n name: Call external workflow\n uses: SINTEF/ci-cd/.github/workflows/ci_check_pyproject_dependencies.yml@v2.8.3\n if: github.repository_owner == 'SINTEF'\n with:\n git_username: \"Casper Welzel Andersen\"\n git_email: \"CasperWA@github.com\"\n target_branch: \"ci/dependency-updates\"\n python_version: \"3.9\"\n install_extras: \"[dev]\"\n pr_labels: \"CI/CD\"\n secrets:\n PAT: ${{ secrets.PAT }}\nFile to use: ci_tests.yml
A basic set of CI tests.
Several different basic test jobs are available in this workflow. By default, they will all run and should be actively \"turned off\".
"},{"location":"workflows/ci_tests/#ci-jobs","title":"CI jobs","text":"The following sections summarizes each job and the individual inputs necessary for it to function or to adjust how it runs. Note, a full list of possible inputs and secrets will be given in a separate table at the end of this page.
"},{"location":"workflows/ci_tests/#globalgeneral-inputs","title":"Global/General inputs","text":"These inputs are general and apply to all jobs in this workflow.
Name Description Required Default Typerunner The runner to use for the workflow. Note, the callable workflow expects a Linux/Unix system.. No ubuntu-latest string install_extras Any extras to install from the local repository through 'pip'. Must be encapsulated in square parentheses ([]) and be separated by commas (,) without any spaces.Example: '[dev,pre-commit]'. No Empty string string"},{"location":"workflows/ci_tests/#run-pre-commit","title":"Run pre-commit","text":"Run the pre-commit tool for all files in the repository according to the repository's configuration file.
pre-commit should be setup for the repository. For more information about pre-commit, please see the tool's website: pre-commit.com.
This job should not be run if the repository does not implement pre-commit.
run_pre-commit Run the pre-commit test job. No true boolean python_version_pre-commit The Python version to use for the pre-commit test job. No 3.9 string pip_index_url_pre-commit A URL to a PyPI repository index. No https://pypi.org/simple/ string pip_extra_index_urls_pre-commit A space-delimited string of URLs to additional PyPI repository indices. No Empty string string skip_pre-commit_hooks A comma-separated list of pre-commit hook IDs to skip when running pre-commit after updating hooks. No Empty string string"},{"location":"workflows/ci_tests/#run-pylint-safety","title":"Run pylint & safety","text":"Run the pylint and/or safety tools.
The pylint tool can be run in different ways. Either it is run once and the pylint_targets is a required input, while pylint_options is a single- or multi-line optional input. Or pylint_runs is used, a single- or multi-line input, to explicitly write out all pylint options and target(s) one line at a time. For each line in pylint_runs, pylint will be executed.
Using pylint_runs is useful if you have a section of your code, which should be run with a custom set of options, otherwise it is recommended to instead simply use the pylint_targets and optionally also pylint_options inputs.
The safety tool checks all installed Python packages, hence the install_extras input should be given as to install all possible dependencies.
There are no expectations or pre-requisites. pylint and safety can be run without a pre-configuration.
run_pylint Run the pylint test job. No true boolean run_safety Run the safety test job. No true boolean python_version_pylint_safety The Python version to use for the pylint and safety test jobs. No 3.9 string pip_index_url_pylint_safety A URL to a PyPI repository index. No https://pypi.org/simple/ string pip_extra_index_urls_pylint_safety A space-delimited string of URLs to additional PyPI repository indices. No Empty string string pylint_targets Space-separated string of pylint file and folder targets.Note: This is only valid if pylint_runs is not defined. Yes, if pylint_runs is not defined Empty string string pylint_options Single (space-separated) or multi-line string of pylint command line options.Note: This is only valid if pylint_runs is not defined.See also Single vs multi-line input. No Empty string string pylint_runs Multi-line string with each line representing a separate pylint run/execution. This should include all desired options and targets.Important: The inputs pylint_options and pylint_targets will be ignored if this is defined.See also Single vs multi-line input. No Empty string string safety_options Single (space-separated) or multi-line string of safety command line options.See also Single vs multi-line input. No Empty string string"},{"location":"workflows/ci_tests/#build-distribution-package","title":"Build distribution package","text":"Test building the Python package.
This job is equivalent to building the package in the CD - Release workflow, but will not publish anything.
"},{"location":"workflows/ci_tests/#expectations_2","title":"Expectations","text":"The repository should be a \"buildable\" Python package.
"},{"location":"workflows/ci_tests/#inputs_2","title":"Inputs","text":"Name Description Required Default Typerun_build_package Run the build package test job. No true boolean python_version_package The Python version to use for the build package test job. No 3.9 string pip_index_url_package A URL to a PyPI repository index. No https://pypi.org/simple/ string pip_extra_index_urls_package A space-delimited string of URLs to additional PyPI repository indices. No Empty string string build_libs A space-separated list of packages to install via PyPI (pip install). No Empty string string build_cmd The package build command, e.g., 'flit build' or 'python -m build' (default). No python -m build string"},{"location":"workflows/ci_tests/#build-documentation","title":"Build Documentation","text":"Test building the documentation.
Two frameworks are supported: MkDocs and Sphinx.
By default the MkDocs framework is used. To use the Sphinx framework set the input use_sphinx to true. The input use_mkdocs can also explicitly be set to true for more transparent documentation in your workflow.
Note, if both use_sphinx and use_mkdocs are false (as is the default value for both), the workflow will fallback to using MkDocs, i.e., it is equivalent to setting use_mkdocs to true.
For MkDocs users
If using mike, note that this will not be tested, as this would be equivalent to testing mike itself and whether it can build a MkDocs documentation, which should never be part of a repository that uses these tools.
If used together with the Update API Reference in Documentation, please align the relative input with the --relative option, when running the hook. See the proper section to understand why and how these options and inputs should be aligned.
Is is expected that documentation exists, which is using either the MkDocs framework or the Sphinx framework. For MkDocs, this requires at minimum a mkdocs.yml configuration file. For Sphinx, it requires at minimum the files created from running sphinx-quickstart.
General inputs for building the documentation:
Name Description Required Default Typerun_build_docs Run the build package test job. No true boolean python_version_docs The Python version to use for the build documentation test job. No 3.9 string pip_index_url_docs A URL to a PyPI repository index. No https://pypi.org/simple/ string pip_extra_index_urls_docs A space-delimited string of URLs to additional PyPI repository indices. No Empty string string relative Whether or not to use the locally installed Python package(s), and install it as an editable. No false boolean system_dependencies A single (space-separated) or multi-line string of Ubuntu APT packages to install prior to building the documentation.See also Single vs multi-line input. No Empty string string warnings_as_errors Build the documentation in 'strict' mode, treating warnings as errors.Important: If this is set to false, beware that the documentation may not be rendered or built as one may have intended.Default: true. No true boolean use_mkdocs Whether or not to build the documentation using the MkDocs framework. Mutually exclusive with use_sphinx. No false boolean use_sphinx Whether or not to build the documentation using the Sphinx framework. Mutually exclusive with use_mkdocs. No false boolean MkDocs-specific inputs:
Name Description Required Default Typeupdate_python_api_ref Whether or not to update the Python API documentation reference.Note: If this is true, package_dirs is required. No true boolean update_docs_landing_page Whether or not to update the documentation landing page. The landing page will be based on the root README.md file. No true boolean package_dirs A multi-line string of path to Python package directories relative to the repository directory to be considered for creating the Python API reference documentation.Example: 'src/my_package'.See also Single vs multi-line input. Yes, if update_python_api_ref is true (default) Empty string string exclude_dirs A multi-line string of directories to exclude in the Python API reference documentation. Note, only directory names, not paths, may be included. Note, all folders and their contents with these names will be excluded. Defaults to '__pycache__'.Important: When a user value is set, the preset value is overwritten - hence '__pycache__' should be included in the user value if one wants to exclude these directories.See also Single vs multi-line input. No __pycache__ string exclude_files A multi-line string of files to exclude in the Python API reference documentation. Note, only full file names, not paths, may be included, i.e., filename + file extension. Note, all files with these names will be excluded. Defaults to '__init__.py'.Important: When a user value is set, the preset value is overwritten - hence '__init__.py' should be included in the user value if one wants to exclude these files.See also Single vs multi-line input. No __init__.py string full_docs_dirs A multi-line string of directories in which to include everything - even those without documentation strings. This may be useful for a module full of data models or to ensure all class attributes are listed.See also Single vs multi-line input. No Empty string string full_docs_files A multi-line string of relative paths to files in which to include everything - even those without documentation strings. This may be useful for a file full of data models or to ensure all class attributes are listed.See also Single vs multi-line input. No Empty string string special_file_api_ref_options A multi-line string of combinations of a relative path to a Python file and a fully formed mkdocstrings option that should be added to the generated MarkDown file for the Python API reference documentation.Example: my_module/py_file.py,show_bases:false.Encapsulate the value in double quotation marks (\") if including spaces ( ).Important: If multiple package_dirs are supplied, the relative path MUST include/start with the appropriate 'package_dir' value, e.g., \"my_package/my_module/py_file.py,show_bases: false\".See also Single vs multi-line input. No Empty string string landing_page_replacements A multi-line string of replacements (mappings) to be performed on README.md when creating the documentation's landing page (index.md). This list always includes replacing 'docs/' with an empty string to correct relative links, i.e., this cannot be overwritten. By default '(LICENSE)' is replaced by '(LICENSE.md)'.See also Single vs multi-line input. No (LICENSE),(LICENSE.md) string landing_page_replacement_separator String to separate a mapping's 'old' to 'new' parts. Defaults to a comma (,). No , string debug Whether to do print extra debug statements. No false boolean Sphinx-specific inputs:
Name Description Required Default Typesphinx-build_options Single (space-separated) or multi-line string of command-line options to use when calling sphinx-build.Note: The -W option will be added if warnings_as_errors is true (default).See also Single vs multi-line input. No Empty string string docs_folder The path to the root documentation folder relative to the repository root. No docs string build_target_folder The path to the target folder for the documentation build relative to the repository root. No site string"},{"location":"workflows/ci_tests/#usage-example","title":"Usage example","text":"The following is an example of how a workflow may look that calls CI - Tests. It is meant to be complete as is.
name: CI - Tests\n\non:\n pull_request:\n pull:\n branches:\n - 'main'\n\njobs:\n tests:\n name: Run basic tests\n uses: SINTEF/ci-cd/.github/workflows/ci_tests.yml@v2.8.3\n with:\n python_version_pylint_safety: \"3.8\"\n python_version_docs: \"3.7\"\n install_extras: \"[dev,docs]\"\n skip_pre-commit_hooks: pylint\n pylint_options: --rcfile=pyproject.toml\n pylint_targets: my_python_package\n build_libs: flit\n build_cmd: flit build\n update_python_api_ref: false\n update_docs_landing_page: false\nHere is another example using pylint_runs instead of pylint_targets and pylint_options.
name: CI - Tests\n\non:\n pull_request:\n pull:\n branches:\n - 'main'\n\njobs:\n tests:\n name: Run basic tests\n uses: SINTEF/ci-cd/.github/workflows/ci_tests.yml@v2.8.3\n with:\n python_version_pylint_safety: \"3.8\"\n python_version_docs: \"3.7\"\n install_extras: \"[dev,docs]\"\n skip_pre-commit_hooks: pylint\n pylint_runs: |\n --rcfile=pyproject.toml --ignore-paths=tests/ my_python_package\n --rcfile=pyproject.toml --disable=import-outside-toplevel,redefined-outer-name tests\n build_libs: flit\n build_cmd: flit build\n update_python_api_ref: false\n update_docs_landing_page: false\nHere follows the full list of inputs available for this workflow. However, it is recommended to instead refer to the job-specific tables of inputs when considering which inputs to provide.
See also General information.
Name Description Required Default Typerunner The runner to use for the workflow. Note, the callable workflow expects a Linux/Unix system.. No ubuntu-latest string install_extras Any extras to install from the local repository through 'pip'. Must be encapsulated in square parentheses ([]) and be separated by commas (,) without any spaces.Example: '[dev,pre-commit]'. No Empty string string run_pre-commit Run the pre-commit test job. No true boolean python_version_pre-commit The Python version to use for the pre-commit test job. No 3.9 string pip_index_url_pre-commit A URL to a PyPI repository index. No https://pypi.org/simple/ string pip_extra_index_urls_pre-commit A space-delimited string of URLs to additional PyPI repository indices. No Empty string string skip_pre-commit_hooks A comma-separated list of pre-commit hook IDs to skip when running pre-commit after updating hooks. No Empty string string run_pylint Run the pylint test job. No true boolean run_safety Run the safety test job. No true boolean python_version_pylint_safety The Python version to use for the pylint and safety test jobs. No 3.9 string pip_index_url_pylint_safety A URL to a PyPI repository index. No https://pypi.org/simple/ string pip_extra_index_urls_pylint_safety A space-delimited string of URLs to additional PyPI repository indices. No Empty string string pylint_targets Space-separated string of pylint file and folder targets.Note: This is only valid if pylint_runs is not defined. Yes, if pylint_runs is not defined Empty string string pylint_options Single (space-separated) or multi-line string of pylint command line options.Note: This is only valid if pylint_runs is not defined. No Empty string string pylint_runs Single or multi-line string with each line representing a separate pylint run/execution. This should include all desired options and targets.Important: The inputs pylint_options and pylint_targets will be ignored if this is defined. No Empty string string safety_options Single (space-separated) or multi-line string of safety command line options. No Empty string string run_build_package Run the build package test job. No true boolean python_version_package The Python version to use for the build package test job. No 3.9 string pip_index_url_package A URL to a PyPI repository index. No https://pypi.org/simple/ string pip_extra_index_urls_package A space-delimited string of URLs to additional PyPI repository indices. No Empty string string build_libs A space-separated list of packages to install via PyPI (pip install). No Empty string string build_cmd The package build command, e.g., 'flit build' or 'python -m build' (default). No python -m build string run_build_docs Run the build package test job. No true boolean python_version_docs The Python version to use for the build documentation test job. No 3.9 string pip_index_url_docs A URL to a PyPI repository index. No https://pypi.org/simple/ string pip_extra_index_urls_docs A space-delimited string of URLs to additional PyPI repository indices. No Empty string string relative Whether or not to use the locally installed Python package(s), and install it as an editable. No false boolean system_dependencies A single (space-separated) or multi-line string of Ubuntu APT packages to install prior to building the documentation. No Empty string string warnings_as_errors Build the documentation in 'strict' mode, treating warnings as errors.Important: If this is set to false, beware that the documentation may not be rendered or built as one may have intended.Default: true. No true boolean use_mkdocs Whether or not to build the documentation using the MkDocs framework. Mutually exclusive with use_sphinx. No false boolean use_sphinx Whether or not to build the documentation using the Sphinx framework. Mutually exclusive with use_mkdocs. No false boolean update_python_api_ref Whether or not to update the Python API documentation reference.Note: If this is true, package_dirs is required. No true boolean update_docs_landing_page Whether or not to update the documentation landing page. The landing page will be based on the root README.md file. No true boolean package_dirs A multi-line string of path to Python package directories relative to the repository directory to be considered for creating the Python API reference documentation.Example: 'src/my_package'. Yes, if update_python_api_ref is true (default) Empty string string exclude_dirs A multi-line string of directories to exclude in the Python API reference documentation. Note, only directory names, not paths, may be included. Note, all folders and their contents with these names will be excluded. Defaults to '__pycache__'.Important: When a user value is set, the preset value is overwritten - hence '__pycache__' should be included in the user value if one wants to exclude these directories. No __pycache__ string exclude_files A multi-line string of files to exclude in the Python API reference documentation. Note, only full file names, not paths, may be included, i.e., filename + file extension. Note, all files with these names will be excluded. Defaults to '__init__.py'.Important: When a user value is set, the preset value is overwritten - hence '__init__.py' should be included in the user value if one wants to exclude these files. No __init__.py string full_docs_dirs A multi-line string of directories in which to include everything - even those without documentation strings. This may be useful for a module full of data models or to ensure all class attributes are listed. No Empty string string full_docs_files A multi-line string of relative paths to files in which to include everything - even those without documentation strings. This may be useful for a file full of data models or to ensure all class attributes are listed. No Empty string string special_file_api_ref_options A multi-line string of combinations of a relative path to a Python file and a fully formed mkdocstrings option that should be added to the generated MarkDown file for the Python API reference documentation.Example: my_module/py_file.py,show_bases:false.Encapsulate the value in double quotation marks (\") if including spaces ( ).Important: If multiple package_dirs are supplied, the relative path MUST include/start with the appropriate 'package_dir' value, e.g., \"my_package/my_module/py_file.py,show_bases: false\". No Empty string string landing_page_replacements A multi-line string of replacements (mappings) to be performed on README.md when creating the documentation's landing page (index.md). This list always includes replacing 'docs/' with an empty string to correct relative links, i.e., this cannot be overwritten. By default '(LICENSE)' is replaced by '(LICENSE.md)'. No (LICENSE),(LICENSE.md) string landing_page_replacement_separator String to separate a mapping's 'old' to 'new' parts. Defaults to a comma (,). No , string debug Whether to do print extra debug statements. No false boolean sphinx-build_options Single or multi-line string of command-line options to use when calling sphinx-build.Note: The -W option will be added if warnings_as_errors is true (default). No Empty string string docs_folder The path to the root documentation folder relative to the repository root. No docs string build_target_folder The path to the target folder for the documentation build relative to the repository root. No site string"},{"location":"workflows/ci_update_dependencies/","title":"CI - Update dependencies PR","text":"File to use: ci_update_dependencies.yml
This workflow creates a PR if there are any updates in the permanent_dependencies_branch branch that have not been included in the default_repo_branch branch.
This workflow works nicely together with the CI - Check pyproject.toml dependencies workflow, and the same value for permanent_dependencies_branch should be used. In this way, this workflow can be called on a schedule to update the dependencies that have been merged into the permanent_dependencies_branch branch into the default_repo_branch branch.
The main point of having this workflow is to have a single PR, which can be squash merged, to merge several dependency updates performed by Dependabot or similar.
As a \"bonus\" this workflow supports updating pre-commit hooks.
PR branch name
The generated branch for the PR will be named ci/update-dependencies.
Warning
If a PAT is not passed through for the PAT secret and GITHUB_TOKEN is used, beware that any other CI/CD jobs that run for, e.g., pull request events, may not run since GITHUB_TOKEN-generated PRs are designed to not start more workflows to avoid escalation. Hence, if it is important to run CI/CD workflows for pull requests, consider passing a PAT as a secret to this workflow represented by the PAT secret.
Important
If this is to be used together with the CI/CD - New updates to default branch workflow, the pr_body_file supplied (if any) should be immutable within the first 8 lines, i.e., no check boxes or similar in the first 8 lines. Indeed, it is recommended to not supply a pr_body_file in this case.
There are no expectations of the repo when using this workflow.
"},{"location":"workflows/ci_update_dependencies/#inputs","title":"Inputs","text":"Name Description Required Default Typegit_username A git username (used to set the 'user.name' config option). Yes string git_email A git user's email address (used to set the 'user.email' config option). Yes string runner The runner to use for the workflow. Note, the callable workflow expects a Linux/Unix system.. No ubuntu-latest string permanent_dependencies_branch The branch name for the permanent dependency updates branch. No ci/dependency-updates string default_repo_branch The branch name of the repository's default branch. More specifically, the branch the PR should target. No main string pr_body_file Relative path to PR body file from the root of the repository.Example: '.github/utils/pr_body_update_deps.txt'. No Empty string string pr_labels A comma separated list of strings of GitHub labels to use for the created PR. No Empty string string extra_to_dos A multi-line string (insert \\n to create line breaks) with extra 'to do' checks. Should start with - [ ].See also Single vs multi-line input. No Empty string string update_pre-commit Whether or not to update pre-commit hooks as part of creating the PR. No false boolean python_version The Python version to use for the workflow.Note: This is only relevant if update_pre-commit is true. No 3.9 string install_extras Any extras to install from the local repository through 'pip'. Must be encapsulated in square parentheses ([]) and be separated by commas (,) without any spaces.Example: '[dev,pre-commit]'.Note: This is only relevant if update_pre-commit is true. No Empty string string pip_index_url A URL to a PyPI repository index.Note: This is only relevant if update_pre-commit is true. No https://pypi.org/simple/ string pip_extra_index_urls A space-delimited string of URLs to additional PyPI repository indices.Note: This is only relevant if update_pre-commit is true. No Empty string string skip_pre-commit_hooks A comma-separated list of pre-commit hook IDs to skip when running pre-commit after updating hooks.Note: This is only relevant if update_pre-commit is true. No Empty string string"},{"location":"workflows/ci_update_dependencies/#secrets","title":"Secrets","text":"Name Description Required PAT A personal access token (PAT) with rights to create PRs. This will fallback on GITHUB_TOKEN. No"},{"location":"workflows/ci_update_dependencies/#usage-example","title":"Usage example","text":"The following is an example of how a workflow may look that calls CI - Update dependencies PR. It is meant to be complete as is.
name: CI - Update dependencies\n\non:\n schedule:\n - cron: \"30 6 * * 3\"\n workflow_dispatch:\n\njobs:\n check-dependencies:\n name: Call external workflow\n uses: SINTEF/ci-cd/.github/workflows/ci_update_dependencies.yml@v2.8.3\n if: github.repository_owner == 'SINTEF'\n with:\n git_username: \"Casper Welzel Andersen\"\n git_email: \"CasperWA@github.com\"\n permanent_dependencies_branch: \"ci/dependency-updates\"\n default_repo_branch: stable\n pr_labels: \"CI/CD\"\n extra_to_dos: \"- [ ] Make sure the PR is **squash** merged, with a sensible commit message.\\n- [ ] Check related `requirements*.txt` files are updated accordingly.\"\n update_pre-commit: true\n python_version: \"3.9\"\n install_extras: \"[pre-commit]\"\n skip_pre-commit_hooks: \"pylint,pylint-models\"\n secrets:\n PAT: ${{ secrets.PAT }}\nCurrent version to use: v2.8.3
Important
The default for publish_on_pypi in the CD - Release workflow has changed from true to false in version 2.8.0.
To keep using the previous behaviour, set publish_on_pypi: true in the workflow file.
This change has been introduced to push for the use of PyPI's Trusted Publisher feature, which is not yet supported by reusable/callable workflows.
See the Using PyPI's Trusted Publisher section for more information on how to migrate to this feature.
Use tried and tested continuous integration (CI) and continuous deployment (CD) tools from this repository.
Currently, the repository offers GitHub Actions callable/reusable workflows and pre-commit hooks.
"},{"location":"#github-actions-callablereusable-workflows","title":"GitHub Actions callable/reusable Workflows","text":"This repository contains reusable workflows for GitHub Actions.
They are mainly for usage with modern Python package repositories.
"},{"location":"#available-workflows","title":"Available workflows","text":"The callable, reusable workflows available from this repository are described in detail in this documentation under the Workflows section.
"},{"location":"#general-usage","title":"General usage","text":"See the GitHub Docs on the topic of calling a reusable workflow to understand how one can incoporate one of these workflows in your workflow.
Note
Workflow-level set env context variables cannot be used when setting input values for the called workflow. See the GitHub documentation for more information on the env context.
Under the Workflows section for each available workflow, a usage example will be given.
"},{"location":"#pre-commit-hooks","title":"pre-commit hooks","text":"This repository contains hooks for keeping the documentation up-to-date, making available a few invoke tasks used in the reusable workflows.
By implementing and using these hooks together with the workflows, one may ensure no extra commits are created during the workflow run to update the documentation.
"},{"location":"#available-hooks","title":"Available hooks","text":"The pre-commit hooks available from this repository are described in detail in this documentation under the Hooks section.
"},{"location":"#general-usage_1","title":"General usage","text":"Add the hooks to your .pre-commit-config.yaml file. See the pre-commit webpage for more information about how to use pre-commit.
Under the Hooks section for each available hook, a usage example will be given.
"},{"location":"#license-copyright","title":"License & copyright","text":"This repository licensed under the MIT LICENSE with copyright \u00a9 2022 Casper Welzel Andersen (CasperWA) & SINTEF (on GitHub).
"},{"location":"#funding-support","title":"Funding support","text":"This repository has been supported by the following projects:
OntoTrans (2020-2024) that receives funding from the European Union\u2019s Horizon 2020 Research and Innovation Programme, under Grant Agreement n. 862136.
OpenModel (2021-2025) receives funding from the European Union\u2019s Horizon 2020 Research and Innovation Programme - DT-NMBP-11-2020 Open Innovation Platform for Materials Modelling, under Grant Agreement no: 953167.
Full Changelog
Closed issues:
Merged pull requests:
Full Changelog
"},{"location":"CHANGELOG/#update-github-actions","title":"Update GitHub Actions","text":"Update the used GitHub Actions in the callable workflows.
"},{"location":"CHANGELOG/#dx","title":"DX","text":"Update development tools and dependencies for an improved developer experience.
Merged pull requests:
Full Changelog
"},{"location":"CHANGELOG/#support-self-hosted-runners","title":"Support self-hosted runners","text":"The runs-on key can not be specified via the runner input, which is available for all callable workflows. This means one can use the callable workflows with self-hosted runners, for example.
It is worth noting that the workflows are built with Linux/Unix systems in mind, hence specifying windows-latest may lead to issues with certain workflows. This is also true if the self-hosted runner is not Linux/Unix-based.
Implemented enhancements:
Merged pull requests:
runner input for all callable workflows #280 (CasperWA)Full Changelog
"},{"location":"CHANGELOG/#support-custom-pypi-indices","title":"Support custom PyPI indices","text":"All callable workflows now have support for setting the PIP_INDEX_URL and PIP_EXTRA_INDEX_URL environment variable whenever pip install is being invoked. Note, the PIP_EXTRA_INDEX_URL allows for multiple URLs to be provided, given they are space-delimited.
For more information on the specific workflow, see the documentation.
Implemented enhancements:
Merged pull requests:
Full Changelog
"},{"location":"CHANGELOG/#support-trusted-publishers-from-pypi","title":"Support Trusted Publishers from PyPI","text":"Trusted Publishers from PyPI is now supported via uploading the distribution(s) as artifacts (for more information about GitHub Actions artifacts, see the GitHub Docs).
Breaking change: This is not a \"true\" breaking change - but it may cause certain workflows to fail that uses the callable workflow CD - Release: The parameter publish_on_pypi has become required, meaning one must provide it in the with section of the calling workflow. For more information, see the documentation page for the CD - Release workflow.
Several fixes from the development tools have been implemented into the code base.
Implemented enhancements:
Merged pull requests:
Full Changelog
Implemented enhancements:
setver #243Merged pull requests:
Full Changelog
Fixed bugs:
git add -- . instead of git commit -a #236Merged pull requests:
Full Changelog
Fixed bugs:
Closed issues:
Merged pull requests:
Full Changelog
"},{"location":"CHANGELOG/#v270-2023-12-07","title":"v2.7.0 (2023-12-07)","text":"Full Changelog
Implemented enhancements:
Fixed bugs:
packaging.version.Version #220 (CasperWA)Closed issues:
Merged pull requests:
Full Changelog
Implemented enhancements:
update_deps.py further #148Fixed bugs:
Merged pull requests:
pyproject.toml) #213 (TEAM4-0)pyproject.toml) #206 (TEAM4-0)Full Changelog
Implemented enhancements:
Fixed bugs:
Merged pull requests:
already_handled_packages #202 (CasperWA)Full Changelog
Implemented enhancements:
latest alias MkDocs release #187Merged pull requests:
mkdocs_update_latest bool input #188 (CasperWA)Full Changelog
Fixed bugs:
--full-docs-dir input #174Merged pull requests:
Full Changelog
Fixed bugs:
pylint_options not working as intended #169Merged pull requests:
pylint_options depending on newlines #170 (CasperWA)Full Changelog
Implemented enhancements:
update-deps task #24Fixed bugs:
ci-cd update-deps #130Closed issues:
Merged pull requests:
update-pyproject pre-commit hook #128 (CasperWA)Full Changelog
Merged pull requests:
Full Changelog
Implemented enhancements:
Fixed bugs:
Merged pull requests:
Full Changelog
Fixed bugs:
Merged pull requests:
Full Changelog
Implemented enhancements:
Fixed bugs:
fail_fast should still make update-deps task fail #112Merged pull requests:
ignore option for update-deps task #111 (CasperWA)ci-cd #110 (CasperWA)Full Changelog
Implemented enhancements:
Closed issues:
Merged pull requests:
PAT prior to GITHUB_TOKEN #105 (CasperWA)Full Changelog
Implemented enhancements:
test: true actually work for \"CD - Release\" #83Fixed bugs:
Closed issues:
vMAJOR dynamic tag #81Merged pull requests:
Full Changelog
Fixed bugs:
Merged pull requests:
--strict toggleable for mkdocs build #78 (CasperWA)Full Changelog
Implemented enhancements:
Merged pull requests:
Full Changelog
Implemented enhancements:
Merged pull requests:
Full Changelog
Fixed bugs:
.pages in API ref hook #66Merged pull requests:
.pages does not get mkdocstrings option #67 (CasperWA)Full Changelog
Fixed bugs:
Merged pull requests:
Full Changelog
Implemented enhancements:
Fixed bugs:
Closed issues:
Merged pull requests:
Full Changelog
Fixed bugs:
Merged pull requests:
Full Changelog
Fixed bugs:
Merged pull requests:
Full Changelog
Implemented enhancements:
Fixed bugs:
CasperWA to SINTEF #37Closed issues:
Merged pull requests:
Full Changelog
Implemented enhancements:
Fixed bugs:
args for docs-landing-page doesn't work #27Merged pull requests:
args fix for docs-landing-page hook #31 (CasperWA)Full Changelog
Fixed bugs:
Merged pull requests:
Full Changelog
Implemented enhancements:
Merged pull requests:
Full Changelog
Fixed bugs:
Closed issues:
@v1 #15Full Changelog
Implemented enhancements:
Fixed bugs:
Closed issues:
Merged pull requests:
Full Changelog
Merged pull requests:
* This Changelog was automatically generated by github_changelog_generator
"},{"location":"LICENSE/","title":"License","text":"MIT License
Copyright (c) 2022 Casper Welzel Andersen & SINTEF
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the \"Software\"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
"},{"location":"api_reference/exceptions/","title":"exceptions","text":"CI/CD-specific exceptions.
"},{"location":"api_reference/exceptions/#ci_cd.exceptions.CICDException","title":" CICDException (Exception) ","text":"Top-level package exception class.
Source code inci_cd/exceptions.py class CICDException(Exception):\n \"\"\"Top-level package exception class.\"\"\"\n InputError (ValueError, CICDException) ","text":"There is an error with the input given to a task.
Source code inci_cd/exceptions.py class InputError(ValueError, CICDException):\n \"\"\"There is an error with the input given to a task.\"\"\"\n InputParserError (InputError) ","text":"The input could not be parsed, it may be wrongly formatted.
Source code inci_cd/exceptions.py class InputParserError(InputError):\n \"\"\"The input could not be parsed, it may be wrongly formatted.\"\"\"\n UnableToResolve (CICDException) ","text":"Unable to resolve a task or sub-task.
Source code inci_cd/exceptions.py class UnableToResolve(CICDException):\n \"\"\"Unable to resolve a task or sub-task.\"\"\"\nMain invoke Program.
See invoke documentation for more information.
"},{"location":"api_reference/tasks/api_reference_docs/","title":"api_reference_docs","text":"create_api_reference_docs task.
Create Python API reference in the documentation. This is specifically to be used with the MkDocs and mkdocstrings framework.
"},{"location":"api_reference/tasks/api_reference_docs/#ci_cd.tasks.api_reference_docs.create_api_reference_docs","title":"create_api_reference_docs(context, package_dir, pre_clean=False, pre_commit=False, root_repo_path='.', docs_folder='docs', unwanted_folder=None, unwanted_file=None, full_docs_folder=None, full_docs_file=None, special_option=None, relative=False, debug=False)","text":"Create the Python API Reference in the documentation.
Source code inci_cd/tasks/api_reference_docs.py @task(\n help={\n \"package-dir\": (\n \"Relative path to a package dir from the repository root, \"\n \"e.g., 'src/my_package'. This input option can be supplied multiple times.\"\n ),\n \"pre-clean\": \"Remove the 'api_reference' sub directory prior to (re)creation.\",\n \"pre-commit\": (\n \"Whether or not this task is run as a pre-commit hook. Will return a \"\n \"non-zero error code if changes were made.\"\n ),\n \"root-repo-path\": (\n \"A resolvable path to the root directory of the repository folder.\"\n ),\n \"docs-folder\": (\n \"The folder name for the documentation root folder. \"\n \"This defaults to 'docs'.\"\n ),\n \"unwanted-folder\": (\n \"A folder to avoid including into the Python API reference documentation. \"\n \"Note, only folder names, not paths, may be included. Note, all folders \"\n \"and their contents with this name will be excluded. Defaults to \"\n \"'__pycache__'. This input option can be supplied multiple times.\"\n ),\n \"unwanted-file\": (\n \"A file to avoid including into the Python API reference documentation. \"\n \"Note, only full file names, not paths, may be included, i.e., filename + \"\n \"file extension. Note, all files with this names will be excluded. \"\n \"Defaults to '__init__.py'. This input option can be supplied multiple \"\n \"times.\"\n ),\n \"full-docs-folder\": (\n \"A folder in which to include everything - even those without \"\n \"documentation strings. This may be useful for a module full of data \"\n \"models or to ensure all class attributes are listed. This input option \"\n \"can be supplied multiple times.\"\n ),\n \"full-docs-file\": (\n \"A full relative path to a file in which to include everything - even \"\n \"those without documentation strings. This may be useful for a file full \"\n \"of data models or to ensure all class attributes are listed. This input \"\n \"option can be supplied multiple times.\"\n ),\n \"special-option\": (\n \"A combination of a relative path to a file and a fully formed \"\n \"mkdocstrings option that should be added to the generated MarkDown file. \"\n \"The combination should be comma-separated. Example: \"\n \"'my_module/py_file.py,show_bases:false'. Encapsulate the value in double \"\n 'quotation marks (\") if including spaces ( ). Important: If multiple '\n \"package-dir options are supplied, the relative path MUST include/start \"\n \"with the package-dir value, e.g., \"\n \"'\\\"my_package/my_module/py_file.py,show_bases: false\\\"'. This input \"\n \"option can be supplied multiple times. The options will be accumulated \"\n \"for the same file, if given several times.\"\n ),\n \"relative\": (\n \"Whether or not to use relative Python import links in the API reference \"\n \"markdown files.\"\n ),\n \"debug\": \"Whether or not to print debug statements.\",\n },\n iterable=[\n \"package_dir\",\n \"unwanted_folder\",\n \"unwanted_file\",\n \"full_docs_folder\",\n \"full_docs_file\",\n \"special_option\",\n ],\n)\ndef create_api_reference_docs(\n context,\n package_dir,\n pre_clean=False,\n pre_commit=False,\n root_repo_path=\".\",\n docs_folder=\"docs\",\n unwanted_folder=None,\n unwanted_file=None,\n full_docs_folder=None,\n full_docs_file=None,\n special_option=None,\n relative=False,\n debug=False,\n):\n \"\"\"Create the Python API Reference in the documentation.\"\"\"\n if TYPE_CHECKING: # pragma: no cover\n context: Context = context # type: ignore[no-redef]\n pre_clean: bool = pre_clean # type: ignore[no-redef]\n pre_commit: bool = pre_commit # type: ignore[no-redef]\n root_repo_path: str = root_repo_path # type: ignore[no-redef]\n docs_folder: str = docs_folder # type: ignore[no-redef]\n relative: bool = relative # type: ignore[no-redef]\n debug: bool = debug # type: ignore[no-redef]\n\n if not unwanted_folder:\n unwanted_folder: list[str] = [\"__pycache__\"] # type: ignore[no-redef]\n if not unwanted_file:\n unwanted_file: list[str] = [\"__init__.py\"] # type: ignore[no-redef]\n if not full_docs_folder:\n full_docs_folder: list[str] = [] # type: ignore[no-redef]\n if not full_docs_file:\n full_docs_file: list[str] = [] # type: ignore[no-redef]\n if not special_option:\n special_option: list[str] = [] # type: ignore[no-redef]\n\n # Initialize user-given paths as pure POSIX paths\n package_dir: list[PurePosixPath] = [PurePosixPath(_) for _ in package_dir]\n root_repo_path = str(PurePosixPath(root_repo_path))\n docs_folder: PurePosixPath = PurePosixPath(docs_folder) # type: ignore[no-redef]\n full_docs_folder = [Path(PurePosixPath(_)) for _ in full_docs_folder]\n\n def write_file(full_path: Path, content: str) -> None:\n \"\"\"Write file with `content` to `full_path`\"\"\"\n if full_path.exists():\n cached_content = full_path.read_text(encoding=\"utf8\")\n if content == cached_content:\n del cached_content\n return\n del cached_content\n full_path.write_text(content, encoding=\"utf8\")\n\n if pre_commit:\n # Ensure git is installed\n result: Result = context.run(\"git --version\", hide=True)\n if result.exited != 0:\n sys.exit(\n \"Git is not installed. Please install it before running this task.\"\n )\n\n if pre_commit and root_repo_path == \".\":\n # Use git to determine repo root\n result = context.run(\"git rev-parse --show-toplevel\", hide=True)\n root_repo_path = result.stdout.strip(\"\\n\") # type: ignore[no-redef]\n\n root_repo_path: Path = Path(root_repo_path).resolve() # type: ignore[no-redef]\n package_dirs: list[Path] = [Path(root_repo_path / _) for _ in package_dir]\n docs_api_ref_dir = Path(root_repo_path / docs_folder / \"api_reference\")\n\n LOGGER.debug(\n \"\"\"package_dirs: %s\ndocs_api_ref_dir: %s\nunwanted_folder: %s\nunwanted_file: %s\nfull_docs_folder: %s\nfull_docs_file: %s\nspecial_option: %s\"\"\",\n package_dirs,\n docs_api_ref_dir,\n unwanted_folder,\n unwanted_file,\n full_docs_folder,\n full_docs_file,\n special_option,\n )\n if debug:\n print(\"package_dirs:\", package_dirs, flush=True)\n print(\"docs_api_ref_dir:\", docs_api_ref_dir, flush=True)\n print(\"unwanted_folder:\", unwanted_folder, flush=True)\n print(\"unwanted_file:\", unwanted_file, flush=True)\n print(\"full_docs_folder:\", full_docs_folder, flush=True)\n print(\"full_docs_file:\", full_docs_file, flush=True)\n print(\"special_option:\", special_option, flush=True)\n\n special_options_files = defaultdict(list)\n for special_file, option in [_.split(\",\", maxsplit=1) for _ in special_option]:\n if any(\",\" in _ for _ in (special_file, option)):\n LOGGER.error(\n \"Failing for special-option: %s\", \",\".join([special_file, option])\n )\n if debug:\n print(\n \"Failing for special-option:\",\n \",\".join([special_file, option]),\n flush=True,\n )\n sys.exit(\n \"special-option values may only include a single comma (,) to \"\n \"separate the relative file path and the mkdocstsrings option.\"\n )\n special_options_files[special_file].append(option)\n\n LOGGER.debug(\"special_options_files: %s\", special_options_files)\n if debug:\n print(\"special_options_files:\", special_options_files, flush=True)\n\n if any(os.sep in _ or \"/\" in _ for _ in unwanted_folder + unwanted_file):\n sys.exit(\"Unwanted folders and files may NOT be paths.\")\n\n pages_template = 'title: \"{name}\"\\n'\n md_template = \"# {name}\\n\\n::: {py_path}\\n\"\n no_docstring_template_addition = (\n f\"{' ' * 4}options:\\n{' ' * 6}show_if_no_docstring: true\\n\"\n )\n\n if docs_api_ref_dir.exists() and pre_clean:\n LOGGER.debug(\"Removing %s\", docs_api_ref_dir)\n if debug:\n print(f\"Removing {docs_api_ref_dir}\", flush=True)\n shutil.rmtree(docs_api_ref_dir, ignore_errors=True)\n if docs_api_ref_dir.exists():\n sys.exit(f\"{docs_api_ref_dir} should have been removed!\")\n docs_api_ref_dir.mkdir(exist_ok=True)\n\n LOGGER.debug(\"Writing file: %s\", docs_api_ref_dir / \".pages\")\n if debug:\n print(f\"Writing file: {docs_api_ref_dir / '.pages'}\", flush=True)\n write_file(\n full_path=docs_api_ref_dir / \".pages\",\n content=pages_template.format(name=\"API Reference\"),\n )\n\n single_package = len(package_dirs) == 1\n for package in package_dirs:\n for dirpath, dirnames, filenames in os.walk(package):\n for unwanted in unwanted_folder:\n LOGGER.debug(\"unwanted: %s\\ndirnames: %s\", unwanted, dirnames)\n if debug:\n print(\"unwanted:\", unwanted, flush=True)\n print(\"dirnames:\", dirnames, flush=True)\n if unwanted in dirnames:\n # Avoid walking into or through unwanted directories\n dirnames.remove(unwanted)\n\n relpath = Path(dirpath).relative_to(\n package if single_package else package.parent\n )\n abspath = (\n package / relpath if single_package else package.parent / relpath\n ).resolve()\n LOGGER.debug(\"relpath: %s\\nabspath: %s\", relpath, abspath)\n if debug:\n print(\"relpath:\", relpath, flush=True)\n print(\"abspath:\", abspath, flush=True)\n\n if not (abspath / \"__init__.py\").exists():\n # Avoid paths that are not included in the public Python API\n LOGGER.debug(\"does not exist: %s\", abspath / \"__init__.py\")\n print(\"does not exist:\", abspath / \"__init__.py\", flush=True)\n continue\n\n # Create `.pages`\n docs_sub_dir = docs_api_ref_dir / relpath\n docs_sub_dir.mkdir(exist_ok=True)\n LOGGER.debug(\"docs_sub_dir: %s\", docs_sub_dir)\n if debug:\n print(\"docs_sub_dir:\", docs_sub_dir, flush=True)\n if str(relpath) != \".\":\n LOGGER.debug(\"Writing file: %s\", docs_sub_dir / \".pages\")\n if debug:\n print(f\"Writing file: {docs_sub_dir / '.pages'}\", flush=True)\n write_file(\n full_path=docs_sub_dir / \".pages\",\n content=pages_template.format(name=relpath.name),\n )\n\n # Create markdown files\n for filename in (Path(_) for _ in filenames):\n if (\n re.match(r\".*\\.py$\", str(filename)) is None\n or str(filename) in unwanted_file\n ):\n # Not a Python file: We don't care about it!\n # Or filename is in the list of unwanted files:\n # We don't want it!\n LOGGER.debug(\n \"%s is not a Python file or is an unwanted file (through user \"\n \"input). Skipping it.\",\n filename,\n )\n if debug:\n print(\n f\"{filename} is not a Python file or is an unwanted file \"\n \"(through user input). Skipping it.\",\n flush=True,\n )\n continue\n\n py_path_root = (\n package.relative_to(root_repo_path) if relative else package.name\n )\n py_path = (\n f\"{py_path_root}/{filename.stem}\"\n if str(relpath) == \".\"\n or (str(relpath) == package.name and not single_package)\n else (\n f\"{py_path_root}/\"\n f\"{relpath if single_package else relpath.relative_to(package.name)}/\" # noqa: E501\n f\"{filename.stem}\"\n )\n )\n\n # Replace OS specific path separators with forward slashes before\n # replacing that with dots (for Python import paths).\n py_path = py_path.replace(os.sep, \"/\").replace(\"/\", \".\")\n\n LOGGER.debug(\"filename: %s\\npy_path: %s\", filename, py_path)\n if debug:\n print(\"filename:\", filename, flush=True)\n print(\"py_path:\", py_path, flush=True)\n\n relative_file_path = Path(\n str(filename) if str(relpath) == \".\" else str(relpath / filename)\n ).as_posix()\n\n # For special files we want to include EVERYTHING, even if it doesn't\n # have a doc-string\n template = md_template + (\n no_docstring_template_addition\n if relative_file_path in full_docs_file\n or relpath in full_docs_folder\n else \"\"\n )\n\n # Include special options, if any, for certain files.\n if relative_file_path in special_options_files:\n template += (\n f\"{' ' * 4}options:\\n\" if \"options:\\n\" not in template else \"\"\n )\n template += \"\\n\".join(\n f\"{' ' * 6}{option}\"\n for option in special_options_files[relative_file_path]\n )\n template += \"\\n\"\n\n LOGGER.debug(\n \"template: %s\\nWriting file: %s\",\n template,\n docs_sub_dir / filename.with_suffix(\".md\"),\n )\n if debug:\n print(\"template:\", template, flush=True)\n print(\n f\"Writing file: {docs_sub_dir / filename.with_suffix('.md')}\",\n flush=True,\n )\n\n write_file(\n full_path=docs_sub_dir / filename.with_suffix(\".md\"),\n content=template.format(name=filename.stem, py_path=py_path),\n )\n\n if pre_commit:\n # Check if there have been any changes.\n # List changes if yes.\n\n # NOTE: Concerning the weird regular expression, see:\n # http://manpages.ubuntu.com/manpages/precise/en/man1/git-status.1.html\n result = context.run(\n f'git -C \"{root_repo_path}\" status --porcelain '\n f\"{docs_api_ref_dir.relative_to(root_repo_path)}\",\n hide=True,\n )\n if result.stdout:\n for line in result.stdout.splitlines():\n if re.match(r\"^[? MARC][?MD]\", line):\n sys.exit(\n f\"{Emoji.CURLY_LOOP.value} The following files have been \"\n f\"changed/added/removed:\\n\\n{result.stdout}\\n\"\n \"Please stage them:\\n\\n\"\n f\" git add {docs_api_ref_dir.relative_to(root_repo_path)}\"\n )\n print(\n f\"{Emoji.CHECK_MARK.value} No changes - your API reference documentation \"\n \"is up-to-date !\"\n )\ncreate_docs_index task.
Create the documentation index (home) page from README.md.
create_docs_index(context, pre_commit=False, root_repo_path='.', docs_folder='docs', replacement=None, replacement_separator=',')","text":"Create the documentation index page from README.md.
Source code inci_cd/tasks/docs_index.py @task(\n help={\n \"pre-commit\": \"Whether or not this task is run as a pre-commit hook.\",\n \"root-repo-path\": (\n \"A resolvable path to the root directory of the repository folder.\"\n ),\n \"docs-folder\": (\n \"The folder name for the documentation root folder. \"\n \"This defaults to 'docs'.\"\n ),\n \"replacement\": (\n \"A replacement (mapping) to be performed on README.md when creating the \"\n \"documentation's landing page (index.md). This list ALWAYS includes \"\n \"replacing '{docs-folder}/' with an empty string, in order to correct \"\n \"relative links. This input option can be supplied multiple times.\"\n ),\n \"replacement-separator\": (\n \"String to separate a replacement's 'old' to 'new' parts.\"\n \"Defaults to a comma (,).\"\n ),\n },\n iterable=[\"replacement\"],\n)\ndef create_docs_index(\n context,\n pre_commit=False,\n root_repo_path=\".\",\n docs_folder=\"docs\",\n replacement=None,\n replacement_separator=\",\",\n):\n \"\"\"Create the documentation index page from README.md.\"\"\"\n if TYPE_CHECKING: # pragma: no cover\n context: Context = context # type: ignore[no-redef]\n pre_commit: bool = pre_commit # type: ignore[no-redef]\n root_repo_path: str = root_repo_path # type: ignore[no-redef]\n replacement_separator: str = replacement_separator # type: ignore[no-redef]\n\n docs_folder: Path = Path(docs_folder)\n\n if not replacement:\n replacement: list[str] = [] # type: ignore[no-redef]\n replacement.append(f\"{docs_folder.name}/{replacement_separator}\")\n\n if pre_commit and root_repo_path == \".\":\n # Use git to determine repo root\n result: Result = context.run(\"git rev-parse --show-toplevel\", hide=True)\n root_repo_path = result.stdout.strip(\"\\n\")\n\n root_repo_path: Path = Path(root_repo_path).resolve()\n readme = root_repo_path / \"README.md\"\n docs_index = root_repo_path / docs_folder / \"index.md\"\n\n content = readme.read_text(encoding=\"utf8\")\n\n for mapping in replacement:\n try:\n old, new = mapping.split(replacement_separator)\n except ValueError:\n sys.exit(\n \"A replacement must only include an 'old' and 'new' part, i.e., be of \"\n \"exactly length 2 when split by the '--replacement-separator'. The \"\n \"following replacement did not fulfill this requirement: \"\n f\"{mapping!r}\\n --replacement-separator={replacement_separator!r}\"\n )\n content = content.replace(old, new)\n\n docs_index.write_text(content, encoding=\"utf8\")\n\n if pre_commit:\n # Check if there have been any changes.\n # List changes if yes.\n\n # NOTE: Concerning the weird regular expression, see:\n # http://manpages.ubuntu.com/manpages/precise/en/man1/git-status.1.html\n result: Result = context.run( # type: ignore[no-redef]\n f'git -C \"{root_repo_path}\" status --porcelain '\n f\"{docs_index.relative_to(root_repo_path)}\",\n hide=True,\n )\n if result.stdout:\n for line in result.stdout.splitlines():\n if re.match(r\"^[? MARC][?MD]\", line):\n sys.exit(\n f\"{Emoji.CURLY_LOOP.value} The landing page has been updated.\"\n \"\\n\\nPlease stage it:\\n\\n\"\n f\" git add {docs_index.relative_to(root_repo_path)}\"\n )\n print(\n f\"{Emoji.CHECK_MARK.value} No changes - your landing page is up-to-date !\"\n )\nsetver task.
Set the specified version.
"},{"location":"api_reference/tasks/setver/#ci_cd.tasks.setver.setver","title":"setver(_, package_dir, version, root_repo_path='.', code_base_update=None, code_base_update_separator=',', test=False, fail_fast=False)","text":"Sets the specified version of specified Python package.
Source code inci_cd/tasks/setver.py @task(\n help={\n \"version\": \"Version to set. Must be either a SemVer or a PEP 440 version.\",\n \"package-dir\": (\n \"Relative path to package dir from the repository root, \"\n \"e.g. 'src/my_package'.\"\n ),\n \"root-repo-path\": (\n \"A resolvable path to the root directory of the repository folder.\"\n ),\n \"code-base-update\": (\n \"'--code-base-update-separator'-separated string defining 'file path', \"\n \"'pattern', 'replacement string', in that order, for something to update \"\n \"in the code base. E.g., '{package_dir}/__init__.py,__version__ *= \"\n \"*('|\\\").*('|\\\"),__version__ = \\\"{version}\\\"', where '{package_dir}' \"\n \"and {version} will be exchanged with the given '--package-dir' value and \"\n \"given '--version' value, respectively. The 'file path' must always \"\n \"either be relative to the repository root directory or absolute. The \"\n \"'pattern' should be given as a 'raw' Python string. This input option \"\n \"can be supplied multiple times.\"\n ),\n \"code-base-update-separator\": (\n \"The string separator to use for '--code-base-update' values.\"\n ),\n \"fail_fast\": (\n \"Whether to exit the task immediately upon failure or wait until the end. \"\n \"Note, no code changes will happen if an error occurs.\"\n ),\n \"test\": (\n \"Whether to do a dry run or not. If set, the task will not make any \"\n \"changes to the code base.\"\n ),\n },\n iterable=[\"code_base_update\"],\n)\ndef setver(\n _,\n package_dir,\n version,\n root_repo_path=\".\",\n code_base_update=None,\n code_base_update_separator=\",\",\n test=False,\n fail_fast=False,\n):\n \"\"\"Sets the specified version of specified Python package.\"\"\"\n if TYPE_CHECKING: # pragma: no cover\n package_dir: str = package_dir # type: ignore[no-redef]\n version: str = version # type: ignore[no-redef]\n root_repo_path: str = root_repo_path # type: ignore[no-redef]\n code_base_update: list[str] = code_base_update # type: ignore[no-redef]\n code_base_update_separator: str = code_base_update_separator # type: ignore[no-redef]\n test: bool = test # type: ignore[no-redef]\n fail_fast: bool = fail_fast # type: ignore[no-redef]\n\n # Validate inputs\n # Version\n try:\n semantic_version = SemanticVersion(version)\n except ValueError:\n msg = (\n \"Please specify version as a semantic version (SemVer) or PEP 440 version. \"\n \"The version may be prepended by a 'v'.\"\n )\n sys.exit(f\"{Emoji.CROSS_MARK.value} {error_msg(msg)}\")\n\n # Root repo path\n root_repo = Path(root_repo_path).resolve()\n if not root_repo.exists():\n msg = (\n f\"Could not find the repository root at: {root_repo} (user provided: \"\n f\"{root_repo_path!r})\"\n )\n sys.exit(f\"{Emoji.CROSS_MARK.value} {error_msg(msg)}\")\n\n # Run the task with defaults\n if not code_base_update:\n init_file = root_repo / package_dir / \"__init__.py\"\n if not init_file.exists():\n msg = (\n \"Could not find the Python package's root '__init__.py' file at: \"\n f\"{init_file}\"\n )\n sys.exit(f\"{Emoji.CROSS_MARK.value} {error_msg(msg)}\")\n\n update_file(\n init_file,\n (\n r'__version__ *= *(?:\\'|\").*(?:\\'|\")',\n f'__version__ = \"{semantic_version}\"',\n ),\n )\n\n # Success, done\n print(\n f\"{Emoji.PARTY_POPPER.value} Bumped version for {package_dir} to \"\n f\"{semantic_version}.\"\n )\n return\n\n # Code base updates were provided\n # First, validate the inputs\n validated_code_base_updates: list[tuple[Path, str, str, str]] = []\n error: bool = False\n for code_update in code_base_update:\n try:\n filepath, pattern, replacement = code_update.split(\n code_base_update_separator\n )\n except ValueError as exc:\n msg = (\n \"Could not properly extract 'file path', 'pattern', \"\n f\"'replacement string' from the '--code-base-update'={code_update}:\"\n f\"\\n{exc}\"\n )\n LOGGER.error(msg)\n LOGGER.debug(\"Traceback: %s\", traceback.format_exc())\n if fail_fast:\n sys.exit(f\"{Emoji.CROSS_MARK.value} {error_msg(msg)}\")\n print(error_msg(msg), file=sys.stderr, flush=True)\n error = True\n continue\n\n # Resolve file path\n filepath = Path(\n filepath.format(package_dir=package_dir, version=semantic_version)\n )\n\n if not filepath.is_absolute():\n filepath = root_repo / filepath\n\n if not filepath.exists():\n msg = f\"Could not find the user-provided file at: {filepath}\"\n LOGGER.error(msg)\n if fail_fast:\n sys.exit(f\"{Emoji.CROSS_MARK.value} {error_msg(msg)}\")\n print(error_msg(msg), file=sys.stderr, flush=True)\n error = True\n continue\n\n LOGGER.debug(\n \"\"\"filepath: %s\npattern: %r\nreplacement (input): %s\nreplacement (handled): %s\n\"\"\",\n filepath,\n pattern,\n replacement,\n replacement.format(package_dir=package_dir, version=semantic_version),\n )\n\n validated_code_base_updates.append(\n (\n filepath,\n pattern,\n replacement.format(package_dir=package_dir, version=semantic_version),\n replacement,\n )\n )\n\n if error:\n sys.exit(\n f\"{Emoji.CROSS_MARK.value} Errors occurred! See printed statements above.\"\n )\n\n for (\n filepath,\n pattern,\n replacement,\n input_replacement,\n ) in validated_code_base_updates:\n if test:\n print(\n f\"filepath: {filepath}\\npattern: {pattern!r}\\n\"\n f\"replacement (input): {input_replacement}\\n\"\n f\"replacement (handled): {replacement}\"\n )\n continue\n\n try:\n update_file(filepath, (pattern, replacement))\n except re.error as exc:\n if validated_code_base_updates[0] != (\n filepath,\n pattern,\n replacement,\n input_replacement,\n ):\n msg = \"Some files have already been updated !\\n\\n \"\n\n msg += (\n f\"Could not update file {filepath} according to the given input:\\n\\n \"\n f\"pattern: {pattern}\\n replacement: {replacement}\\n\\nException: \"\n f\"{exc}\"\n )\n LOGGER.error(msg)\n LOGGER.debug(\"Traceback: %s\", traceback.format_exc())\n sys.exit(f\"{Emoji.CROSS_MARK.value} {error_msg(msg)}\")\n\n # Success, done\n print(\n f\"{Emoji.PARTY_POPPER.value} Bumped version for {package_dir} to \"\n f\"{semantic_version}.\"\n )\nupdate_deps task.
Update dependencies in a pyproject.toml file.
VALID_PACKAGE_NAME_PATTERN","text":"Pattern to validate package names.
This is a valid non-normalized name, i.e., it can contain capital letters and underscores, periods, and multiples of these, including minus characters.
See PEP 508 for more information, as well as the packaging documentation: https://packaging.python.org/en/latest/specifications/name-normalization/
"},{"location":"api_reference/tasks/update_deps/#ci_cd.tasks.update_deps.update_deps","title":"update_deps(context, root_repo_path='.', fail_fast=False, pre_commit=False, ignore=None, ignore_separator='...', verbose=False, skip_unnormalized_python_package_names=False)","text":"Update dependencies in specified Python package's pyproject.toml.
ci_cd/tasks/update_deps.py @task(\n help={\n \"fail-fast\": (\n \"Fail immediately if an error occurs. Otherwise, print and ignore all \"\n \"non-critical errors.\"\n ),\n \"root-repo-path\": (\n \"A resolvable path to the root directory of the repository folder.\"\n ),\n \"pre-commit\": \"Whether or not this task is run as a pre-commit hook.\",\n \"ignore\": (\n \"Ignore-rules based on the `ignore` config option of Dependabot. It \"\n \"should be of the format: key=value...key=value, i.e., an ellipsis \"\n \"(`...`) separator and then equal-sign-separated key/value-pairs. \"\n \"Alternatively, the `--ignore-separator` can be set to something else to \"\n \"overwrite the ellipsis. The only supported keys are: `dependency-name`, \"\n \"`versions`, and `update-types`. Can be supplied multiple times per \"\n \"`dependency-name`.\"\n ),\n \"ignore-separator\": (\n \"Value to use instead of ellipsis (`...`) as a separator in `--ignore` \"\n \"key/value-pairs.\"\n ),\n \"verbose\": \"Whether or not to print debug statements.\",\n \"skip-unnormalized-python-package-names\": (\n \"Whether to skip dependencies with unnormalized Python package names. \"\n \"Normalization is outlined here: \"\n \"https://packaging.python.org/en/latest/specifications/name-normalization.\"\n ),\n },\n iterable=[\"ignore\"],\n)\ndef update_deps(\n context,\n root_repo_path=\".\",\n fail_fast=False,\n pre_commit=False,\n ignore=None,\n ignore_separator=\"...\",\n verbose=False,\n skip_unnormalized_python_package_names=False,\n):\n \"\"\"Update dependencies in specified Python package's `pyproject.toml`.\"\"\"\n if TYPE_CHECKING: # pragma: no cover\n context: Context = context # type: ignore[no-redef]\n root_repo_path: str = root_repo_path # type: ignore[no-redef]\n fail_fast: bool = fail_fast # type: ignore[no-redef]\n pre_commit: bool = pre_commit # type: ignore[no-redef]\n ignore_separator: str = ignore_separator # type: ignore[no-redef]\n verbose: bool = verbose # type: ignore[no-redef]\n skip_unnormalized_python_package_names: bool = ( # type: ignore[no-redef]\n skip_unnormalized_python_package_names\n )\n\n if not ignore:\n ignore: list[str] = [] # type: ignore[no-redef]\n\n if verbose:\n LOGGER.addHandler(logging.StreamHandler(sys.stdout))\n LOGGER.debug(\"Verbose logging enabled.\")\n\n try:\n ignore_rules = parse_ignore_entries(ignore, ignore_separator)\n except InputError as exc:\n sys.exit(\n f\"{Emoji.CROSS_MARK.value} Error: Could not parse ignore options.\\n\"\n f\"Exception: {exc}\"\n )\n LOGGER.debug(\"Parsed ignore rules: %s\", ignore_rules)\n\n if pre_commit and root_repo_path == \".\":\n # Use git to determine repo root\n result: Result = context.run(\"git rev-parse --show-toplevel\", hide=True)\n root_repo_path = result.stdout.strip(\"\\n\")\n\n pyproject_path = Path(root_repo_path).resolve() / \"pyproject.toml\"\n if not pyproject_path.exists():\n sys.exit(\n f\"{Emoji.CROSS_MARK.value} Error: Could not find the Python package \"\n f\"repository's 'pyproject.toml' file at: {pyproject_path}\"\n )\n\n # Parse pyproject.toml\n try:\n pyproject = tomlkit.parse(pyproject_path.read_bytes())\n except TOMLKitError as exc:\n sys.exit(\n f\"{Emoji.CROSS_MARK.value} Error: Could not parse the 'pyproject.toml' \"\n f\"file at: {pyproject_path}\\nException: {exc}\"\n )\n\n # Retrieve the minimum required Python version\n try:\n py_version = get_min_max_py_version(\n pyproject.get(\"project\", {}).get(\"requires-python\", \"\")\n )\n except UnableToResolve as exc:\n sys.exit(\n f\"{Emoji.CROSS_MARK.value} Error: Cannot determine minimum Python version.\"\n f\"\\nException: {exc}\"\n )\n LOGGER.debug(\"Minimum required Python version: %s\", py_version)\n\n # Retrieve the Python project's package name\n project_name: str = pyproject.get(\"project\", {}).get(\"name\", \"\")\n if not project_name:\n sys.exit(\n f\"{Emoji.CROSS_MARK.value} Error: Could not find the Python project's name\"\n \" in 'pyproject.toml'.\"\n )\n\n # Build the list of dependencies listed in pyproject.toml\n dependencies: list[str] = pyproject.get(\"project\", {}).get(\"dependencies\", [])\n for optional_deps in (\n pyproject.get(\"project\", {}).get(\"optional-dependencies\", {}).values()\n ):\n dependencies.extend(optional_deps)\n\n # Placeholder and default variables\n already_handled_packages: set[Requirement] = set()\n updated_packages: dict[str, str] = {}\n error: bool = False\n\n for dependency in dependencies:\n try:\n parsed_requirement = Requirement(dependency)\n except InvalidRequirement as exc:\n if skip_unnormalized_python_package_names:\n msg = (\n f\"Skipping requirement {dependency!r}, as unnormalized Python \"\n \"package naming is allowed by user. Note, the requirements could \"\n f\"not be parsed: {exc}\"\n )\n LOGGER.info(msg)\n print(info_msg(msg), flush=True)\n continue\n\n msg = (\n f\"Could not parse requirement {dependency!r} from pyproject.toml: \"\n f\"{exc}\"\n )\n LOGGER.error(msg)\n if fail_fast:\n sys.exit(f\"{Emoji.CROSS_MARK.value} {error_msg(msg)}\")\n print(error_msg(msg), file=sys.stderr, flush=True)\n error = True\n continue\n LOGGER.debug(\"Parsed requirement: %r\", parsed_requirement)\n\n # Skip package if already handled\n if parsed_requirement in already_handled_packages:\n continue\n\n # Skip package if it is this project (this can happen for inter-relative extra\n # dependencies)\n if parsed_requirement.name == project_name:\n msg = (\n f\"Dependency {parsed_requirement.name!r} is detected as being this \"\n \"project and will be skipped.\"\n )\n LOGGER.info(msg)\n print(info_msg(msg), flush=True)\n\n _format_and_update_dependency(\n parsed_requirement, dependency, pyproject_path\n )\n already_handled_packages.add(parsed_requirement)\n continue\n\n # Skip URL versioned dependencies\n # BUT do regenerate the dependency in order to have a consistent formatting\n if parsed_requirement.url:\n msg = (\n f\"Dependency {parsed_requirement.name!r} is pinned to a URL and \"\n \"will be skipped.\"\n )\n LOGGER.info(msg)\n print(info_msg(msg), flush=True)\n\n _format_and_update_dependency(\n parsed_requirement, dependency, pyproject_path\n )\n already_handled_packages.add(parsed_requirement)\n continue\n\n # Skip and warn if package is not version-restricted\n # BUT do regenerate the dependency in order to have a consistent formatting\n if not parsed_requirement.specifier:\n # Only warn if package name does not match project name\n if parsed_requirement.name != project_name:\n msg = (\n f\"Dependency {parsed_requirement.name!r} is not version \"\n \"restricted and will be skipped. Consider adding version \"\n \"restrictions.\"\n )\n LOGGER.warning(msg)\n print(warning_msg(msg), flush=True)\n\n _format_and_update_dependency(\n parsed_requirement, dependency, pyproject_path\n )\n already_handled_packages.add(parsed_requirement)\n continue\n\n # Examine markers for a custom set of Python version specifiers\n marker_py_version = \"\"\n if parsed_requirement.marker:\n environment_keys = default_environment().keys()\n empty_environment = {key: \"\" for key in environment_keys}\n python_version_centric_environment = empty_environment\n python_version_centric_environment.update({\"python_version\": py_version})\n\n if not parsed_requirement.marker.evaluate(\n environment=python_version_centric_environment\n ):\n # Current (minimum) Python version does NOT satisfy the marker\n marker_py_version = find_minimum_py_version(\n marker=parsed_requirement.marker,\n project_py_version=py_version,\n )\n else:\n marker_py_version = get_min_max_py_version(parsed_requirement.marker)\n\n LOGGER.debug(\"Min/max Python version from marker: %s\", marker_py_version)\n\n # Check version from PyPI's online package index\n out: Result = context.run(\n \"pip index versions \"\n f\"--python-version {marker_py_version or py_version} \"\n f\"{parsed_requirement.name}\",\n hide=True,\n )\n package_latest_version_line = out.stdout.split(sep=\"\\n\", maxsplit=1)[0]\n match = re.match(\n r\"(?P<package>\\S+) \\((?P<version>\\S+)\\)\", package_latest_version_line\n )\n if match is None:\n msg = (\n \"Could not parse package and version from 'pip index versions' output \"\n f\"for line:\\n {package_latest_version_line}\"\n )\n LOGGER.error(msg)\n if fail_fast:\n sys.exit(f\"{Emoji.CROSS_MARK.value} {error_msg(msg)}\")\n print(error_msg(msg), file=sys.stderr, flush=True)\n already_handled_packages.add(parsed_requirement)\n error = True\n continue\n\n try:\n latest_version = Version(match.group(\"version\"))\n except InvalidVersion as exc:\n msg = (\n f\"Could not parse version {match.group('version')!r} from 'pip index \"\n f\"versions' output for line:\\n {package_latest_version_line}.\\n\"\n f\"Exception: {exc}\"\n )\n LOGGER.error(msg)\n if fail_fast:\n sys.exit(f\"{Emoji.CROSS_MARK.value} {error_msg(msg)}\")\n print(error_msg(msg), file=sys.stderr, flush=True)\n error = True\n continue\n LOGGER.debug(\"Retrieved latest version: %r\", latest_version)\n\n # Here used to be a sanity check to ensure that the package name parsed from\n # pyproject.toml matches the name returned from 'pip index versions'.\n # But I cannot think of a reason why they would not match, so it has been\n # removed.\n # When checking 'pip index versions' output, it seems that the package name\n # returned is always the same as is used in the command call, e.g., if\n # 'pip index versions reQUEsts' is called, then the output will always be\n # 'reQUEsts (<latest version here>)'.\n\n # Check whether pyproject.toml already uses the latest version\n # This is expected if the latest version equals a specifier with any of the\n # operators: ==, >=, or ~=.\n split_latest_version = latest_version.base_version.split(\".\")\n _continue = False\n for specifier in parsed_requirement.specifier:\n if specifier.operator in [\"==\", \">=\", \"~=\"]:\n split_specifier_version = specifier.version.split(\".\")\n equal_length_latest_version = split_latest_version[\n : len(split_specifier_version)\n ]\n if equal_length_latest_version == split_specifier_version:\n LOGGER.debug(\n \"Package %r is already up-to-date. Specifiers: %s. \"\n \"Latest version: %s\",\n parsed_requirement.name,\n parsed_requirement.specifier,\n latest_version,\n )\n already_handled_packages.add(parsed_requirement)\n _continue = True\n if _continue:\n continue\n\n # Create ignore rules based on specifier set\n requirement_ignore_rules = create_ignore_rules(parsed_requirement.specifier)\n if requirement_ignore_rules[\"versions\"]:\n if parsed_requirement.name in ignore_rules:\n # Only \"versions\" key exists in requirement_ignore_rules\n if \"versions\" in ignore_rules[parsed_requirement.name]:\n ignore_rules[parsed_requirement.name][\"versions\"].extend(\n requirement_ignore_rules[\"versions\"]\n )\n else:\n ignore_rules[parsed_requirement.name].update(\n requirement_ignore_rules\n )\n else:\n ignore_rules[parsed_requirement.name] = requirement_ignore_rules\n LOGGER.debug(\n \"Created ignore rules (from specifier set): %s\",\n requirement_ignore_rules,\n )\n\n # Apply ignore rules\n if parsed_requirement.name in ignore_rules or \"*\" in ignore_rules:\n versions: IgnoreVersions = []\n update_types: IgnoreUpdateTypes = {}\n\n if \"*\" in ignore_rules:\n versions, update_types = parse_ignore_rules(ignore_rules[\"*\"])\n\n if parsed_requirement.name in ignore_rules:\n parsed_rules = parse_ignore_rules(ignore_rules[parsed_requirement.name])\n\n versions.extend(parsed_rules[0])\n update_types.update(parsed_rules[1])\n\n LOGGER.debug(\n \"Ignore rules:\\nversions: %s\\nupdate_types: %s\", versions, update_types\n )\n\n # Get \"current\" version from specifier set, i.e., the lowest allowed version\n # If a minimum version is not explicitly specified, use '0.0.0'\n for specifier in parsed_requirement.specifier:\n if specifier.operator in [\"==\", \">=\", \"~=\"]:\n current_version = specifier.version.split(\".\")\n break\n else:\n if latest_version.epoch == 0:\n current_version = \"0.0.0\".split(\".\")\n else:\n current_version = f\"{latest_version.epoch}!0.0.0\".split(\".\")\n\n if ignore_version(\n current=current_version,\n latest=split_latest_version,\n version_rules=versions,\n semver_rules=update_types,\n ):\n already_handled_packages.add(parsed_requirement)\n continue\n\n # Update specifier set to include the latest version.\n try:\n updated_specifier_set = update_specifier_set(\n latest_version=latest_version,\n current_specifier_set=parsed_requirement.specifier,\n )\n except UnableToResolve as exc:\n msg = (\n \"Could not determine how to update to the latest version using the \"\n f\"version range specifier set: {parsed_requirement.specifier}. \"\n f\"Package: {parsed_requirement.name}. Latest version: {latest_version}\"\n )\n LOGGER.error(\"%s. Exception: %s\", msg, exc)\n if fail_fast:\n sys.exit(f\"{Emoji.CROSS_MARK.value} {error_msg(msg)}\")\n print(error_msg(msg), file=sys.stderr, flush=True)\n already_handled_packages.add(parsed_requirement)\n error = True\n continue\n\n if not error:\n # Regenerate the full requirement string with the updated specifiers\n # Note: If any white space is present after the name (possibly incl.\n # extras) is reduced to a single space.\n match = re.search(rf\"{parsed_requirement.name}(?:\\[.*\\])?\\s+\", dependency)\n updated_dependency = regenerate_requirement(\n parsed_requirement,\n specifier=updated_specifier_set,\n post_name_space=bool(match),\n )\n LOGGER.debug(\"Updated dependency: %r\", updated_dependency)\n\n pattern_sub_line = re.escape(dependency)\n replacement_sub_line = updated_dependency.replace('\"', \"'\")\n\n LOGGER.debug(\"pattern_sub_line: %s\", pattern_sub_line)\n LOGGER.debug(\"replacement_sub_line: %s\", replacement_sub_line)\n\n # Update pyproject.toml\n update_file(pyproject_path, (pattern_sub_line, replacement_sub_line))\n already_handled_packages.add(parsed_requirement)\n updated_packages[parsed_requirement.name] = \",\".join(\n str(_)\n for _ in sorted(\n updated_specifier_set,\n key=lambda spec: spec.operator,\n reverse=True,\n )\n ) + (f\" ; {parsed_requirement.marker}\" if parsed_requirement.marker else \"\")\n\n if error:\n sys.exit(\n f\"{Emoji.CROSS_MARK.value} Errors occurred! See printed statements above.\"\n )\n\n if updated_packages:\n print(\n f\"{Emoji.PARTY_POPPER.value} Successfully updated the following \"\n \"dependencies:\\n\"\n + \"\\n\".join(\n f\" {package} ({version})\"\n for package, version in updated_packages.items()\n )\n + \"\\n\"\n )\n else:\n print(f\"{Emoji.CHECK_MARK.value} No dependency updates available.\")\nRelevant tools for printing to the console.
"},{"location":"api_reference/utils/console_printing/#ci_cd.utils.console_printing.Color","title":" Color (str, Enum) ","text":"ANSI escape sequences for colors.
Source code inci_cd/utils/console_printing.py class Color(str, Enum):\n \"\"\"ANSI escape sequences for colors.\"\"\"\n\n def __new__(cls, value: str) -> Self:\n obj = str.__new__(cls, value)\n obj._value_ = value\n return obj\n\n RED = \"\\033[91m\"\n GREEN = \"\\033[92m\"\n YELLOW = \"\\033[93m\"\n BLUE = \"\\033[94m\"\n MAGENTA = \"\\033[95m\"\n CYAN = \"\\033[96m\"\n WHITE = \"\\033[97m\"\n RESET = \"\\033[0m\"\n\n def write(self, text: str) -> str:\n \"\"\"Write the text with the color.\"\"\"\n return f\"{self.value}{text}{Color.RESET.value}\"\n Emoji (str, Enum) ","text":"Unicode strings for certain emojis.
Source code inci_cd/utils/console_printing.py class Emoji(str, Enum):\n \"\"\"Unicode strings for certain emojis.\"\"\"\n\n def __new__(cls, value: str) -> Self:\n obj = str.__new__(cls, value)\n if platform.system() == \"Windows\":\n # Windows does not support unicode emojis, so we replace them with\n # their corresponding unicode escape sequences\n obj._value_ = value.encode(\"unicode_escape\").decode(\"utf-8\")\n else:\n obj._value_ = value\n return obj\n\n PARTY_POPPER = \"\\U0001f389\"\n CHECK_MARK = \"\\u2714\"\n CROSS_MARK = \"\\u274c\"\n CURLY_LOOP = \"\\u27b0\"\n Formatting (str, Enum) ","text":"ANSI escape sequences for formatting.
Source code inci_cd/utils/console_printing.py class Formatting(str, Enum):\n \"\"\"ANSI escape sequences for formatting.\"\"\"\n\n def __new__(cls, value: str) -> Self:\n obj = str.__new__(cls, value)\n obj._value_ = value\n return obj\n\n BOLD = \"\\033[1m\"\n UNDERLINE = \"\\033[4m\"\n INVERT = \"\\033[7m\"\n RESET = \"\\033[0m\"\n\n def write(self, text: str) -> str:\n \"\"\"Write the text with the formatting.\"\"\"\n return f\"{self.value}{text}{Formatting.RESET.value}\"\nerror_msg(text)","text":"Write the text as an error message.
Source code inci_cd/utils/console_printing.py def error_msg(text: str) -> str:\n \"\"\"Write the text as an error message.\"\"\"\n return (\n f\"{Color.RED.write(Formatting.BOLD.write('ERROR'))}\"\n f\"{Color.RED.write(' - ' + text)}\"\n )\ninfo_msg(text)","text":"Write the text as an info message.
Source code inci_cd/utils/console_printing.py def info_msg(text: str) -> str:\n \"\"\"Write the text as an info message.\"\"\"\n return (\n f\"{Color.BLUE.write(Formatting.BOLD.write('INFO'))}\"\n f\"{Color.BLUE.write(' - ' + text)}\"\n )\nwarning_msg(text)","text":"Write the text as a warning message.
Source code inci_cd/utils/console_printing.py def warning_msg(text: str) -> str:\n \"\"\"Write the text as a warning message.\"\"\"\n return (\n f\"{Color.YELLOW.write(Formatting.BOLD.write('WARNING'))}\"\n f\"{Color.YELLOW.write(' - ' + text)}\"\n )\nUtilities for handling IO operations.
"},{"location":"api_reference/utils/file_io/#ci_cd.utils.file_io.update_file","title":"update_file(filename, sub_line, strip=None)","text":"Utility function for tasks to read, update, and write files
Source code inci_cd/utils/file_io.py def update_file(\n filename: Path, sub_line: tuple[str, str], strip: str | None = None\n) -> None:\n \"\"\"Utility function for tasks to read, update, and write files\"\"\"\n if strip is None and filename.suffix == \".md\":\n # Keep special white space endings for markdown files\n strip = \"\\n\"\n lines = [\n re.sub(sub_line[0], sub_line[1], line.rstrip(strip))\n for line in filename.read_text(encoding=\"utf8\").splitlines()\n ]\n filename.write_text(\"\\n\".join(lines) + \"\\n\", encoding=\"utf8\")\nHandle versions.
"},{"location":"api_reference/utils/versions/#ci_cd.utils.versions.PART_TO_LENGTH_MAPPING","title":"PART_TO_LENGTH_MAPPING","text":"Mapping of version-style name to their number of version parts.
E.g., a minor version has two parts, so the length is 2.
IgnoreEntryPair (tuple) ","text":"A key/value-pair within an ignore entry.
Source code inci_cd/utils/versions.py class IgnoreEntryPair(NamedTuple):\n \"\"\"A key/value-pair within an ignore entry.\"\"\"\n\n key: Literal[\"dependency-name\", \"versions\", \"update-types\"]\n value: str\n__getnewargs__(self) special","text":"Return self as a plain tuple. Used by copy and pickle.
Source code inci_cd/utils/versions.py def __getnewargs__(self):\n 'Return self as a plain tuple. Used by copy and pickle.'\n return _tuple(self)\n__new__(_cls, key, value) special staticmethod","text":"Create new instance of IgnoreEntryPair(key, value)
"},{"location":"api_reference/utils/versions/#ci_cd.utils.versions.IgnoreEntryPair.__repr__","title":"__repr__(self) special","text":"Return a nicely formatted representation string
Source code inci_cd/utils/versions.py def __repr__(self):\n 'Return a nicely formatted representation string'\n return self.__class__.__name__ + repr_fmt % self\n SemanticVersion (str) ","text":"A semantic version.
See SemVer.org for more information about semantic versioning.
The semantic version is in this invocation considered to build up in the following way:
<major>.<minor>.<patch>-<pre_release>+<build>\nWhere the names in carets are callable attributes for the instance.
When casting instances of SemanticVersion to str, the full version will be returned, i.e., as shown above, with a minimum of major.minor.patch.
For example, for the version 1.5, i.e., major=1, minor=5, the returned str representation will be the full major.minor.patch version: 1.5.0. The patch attribute will default to 0 while pre_release and build will be None, when asked for explicitly.
Precedence for comparing versions is done according to the rules outlined in point 11 of the specification found at SemVer.org.
Parameters:
Name Type Description Defaultmajor Union[str, int] The major version.
'' minor Optional[Union[str, int]] The minor version.
None patch Optional[Union[str, int]] The patch version.
None pre_release Optional[str] The pre-release part of the version, i.e., the part supplied after a minus (-), but before a plus (+).
None build Optional[str] The build metadata part of the version, i.e., the part supplied at the end of the version, after a plus (+).
None Attributes:
Name Type Descriptionmajor int The major version.
minor int The minor version.
patch int The patch version.
pre_release str The pre-release part of the version, i.e., the part supplied after a minus (-), but before a plus (+).
build str The build metadata part of the version, i.e., the part supplied at the end of the version, after a plus (+).
ci_cd/utils/versions.py class SemanticVersion(str):\n \"\"\"A semantic version.\n\n See [SemVer.org](https://semver.org) for more information about semantic\n versioning.\n\n The semantic version is in this invocation considered to build up in the following\n way:\n\n <major>.<minor>.<patch>-<pre_release>+<build>\n\n Where the names in carets are callable attributes for the instance.\n\n When casting instances of `SemanticVersion` to `str`, the full version will be\n returned, i.e., as shown above, with a minimum of major.minor.patch.\n\n For example, for the version `1.5`, i.e., `major=1, minor=5`, the returned `str`\n representation will be the full major.minor.patch version: `1.5.0`.\n The `patch` attribute will default to `0` while `pre_release` and `build` will be\n `None`, when asked for explicitly.\n\n Precedence for comparing versions is done according to the rules outlined in point\n 11 of the specification found at [SemVer.org](https://semver.org/#spec-item-11).\n\n Parameters:\n major (Union[str, int]): The major version.\n minor (Optional[Union[str, int]]): The minor version.\n patch (Optional[Union[str, int]]): The patch version.\n pre_release (Optional[str]): The pre-release part of the version, i.e., the\n part supplied after a minus (`-`), but before a plus (`+`).\n build (Optional[str]): The build metadata part of the version, i.e., the part\n supplied at the end of the version, after a plus (`+`).\n\n Attributes:\n major (int): The major version.\n minor (int): The minor version.\n patch (int): The patch version.\n pre_release (str): The pre-release part of the version, i.e., the part\n supplied after a minus (`-`), but before a plus (`+`).\n build (str): The build metadata part of the version, i.e., the part supplied at\n the end of the version, after a plus (`+`).\n\n \"\"\"\n\n _semver_regex = (\n r\"^(?P<major>0|[1-9]\\d*)(?:\\.(?P<minor>0|[1-9]\\d*))?(?:\\.(?P<patch>0|[1-9]\\d*))?\"\n r\"(?:-(?P<pre_release>(?:0|[1-9]\\d*|\\d*[a-zA-Z-][0-9a-zA-Z-]*)\"\n r\"(?:\\.(?:0|[1-9]\\d*|\\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?\"\n r\"(?:\\+(?P<build>[0-9a-zA-Z-]+(?:\\.[0-9a-zA-Z-]+)*))?$\"\n )\n \"\"\"The regular expression for a semantic version.\n See\n https://semver.org/#is-there-a-suggested-regular-expression-regex-to-check-a-semver-string.\"\"\"\n\n @no_type_check\n def __new__(cls, version: str | Version | None = None, **kwargs: str | int) -> Self:\n return super().__new__(\n cls, str(version) if version else cls._build_version(**kwargs)\n )\n\n def __init__(\n self,\n version: str | Version | None = None,\n *,\n major: str | int = \"\",\n minor: str | int | None = None,\n patch: str | int | None = None,\n pre_release: str | None = None,\n build: str | None = None,\n ) -> None:\n self._python_version: Version | None = None\n\n if version is not None:\n if major or minor or patch or pre_release or build:\n raise ValueError(\n \"version cannot be specified along with other parameters\"\n )\n\n if isinstance(version, Version):\n self._python_version = version\n version = \".\".join(str(_) for _ in version.release)\n\n match = re.match(self._semver_regex, version)\n if match is None:\n # Try to parse it as a Python version and try again\n try:\n _python_version = Version(version)\n except InvalidVersion as exc:\n raise ValueError(\n f\"version ({version}) cannot be parsed as a semantic version \"\n \"according to the SemVer.org regular expression\"\n ) from exc\n\n # Success. Now let's redo the SemVer.org regular expression match\n self._python_version = _python_version\n match = re.match(\n self._semver_regex,\n \".\".join(str(_) for _ in _python_version.release),\n )\n if match is None: # pragma: no cover\n # This should not really be possible at this point, as the\n # Version.releasethis is a guaranteed match.\n # But we keep it here for sanity's sake.\n raise ValueError(\n f\"version ({version}) cannot be parsed as a semantic version \"\n \"according to the SemVer.org regular expression\"\n )\n\n major, minor, patch, pre_release, build = match.groups()\n\n self._major = int(major)\n self._minor = int(minor) if minor else 0\n self._patch = int(patch) if patch else 0\n self._pre_release = pre_release if pre_release else None\n self._build = build if build else None\n\n @classmethod\n def _build_version(\n cls,\n major: str | int | None = None,\n minor: str | int | None = None,\n patch: str | int | None = None,\n pre_release: str | None = None,\n build: str | None = None,\n ) -> str:\n \"\"\"Build a version from the given parameters.\"\"\"\n if major is None:\n raise ValueError(\"At least major must be given\")\n version = str(major)\n if minor is not None:\n version += f\".{minor}\"\n if patch is not None:\n if minor is None:\n raise ValueError(\"Minor must be given if patch is given\")\n version += f\".{patch}\"\n if pre_release is not None:\n # semver spec #9: A pre-release version MAY be denoted by appending a\n # hyphen and a series of dot separated identifiers immediately following\n # the patch version.\n # https://semver.org/#spec-item-9\n if patch is None:\n raise ValueError(\"Patch must be given if pre_release is given\")\n version += f\"-{pre_release}\"\n if build is not None:\n # semver spec #10: Build metadata MAY be denoted by appending a plus sign\n # and a series of dot separated identifiers immediately following the patch\n # or pre-release version.\n # https://semver.org/#spec-item-10\n if patch is None:\n raise ValueError(\"Patch must be given if build is given\")\n version += f\"+{build}\"\n return version\n\n @property\n def major(self) -> int:\n \"\"\"The major version.\"\"\"\n return self._major\n\n @property\n def minor(self) -> int:\n \"\"\"The minor version.\"\"\"\n return self._minor\n\n @property\n def patch(self) -> int:\n \"\"\"The patch version.\"\"\"\n return self._patch\n\n @property\n def pre_release(self) -> str | None:\n \"\"\"The pre-release part of the version\n\n This is the part supplied after a minus (`-`), but before a plus (`+`).\n \"\"\"\n return self._pre_release\n\n @property\n def build(self) -> str | None:\n \"\"\"The build metadata part of the version.\n\n This is the part supplied at the end of the version, after a plus (`+`).\n \"\"\"\n return self._build\n\n @property\n def python_version(self) -> Version | None:\n \"\"\"The Python version as defined by `packaging.version.Version`.\"\"\"\n return self._python_version\n\n def as_python_version(self, shortened: bool = True) -> Version:\n \"\"\"Return the Python version as defined by `packaging.version.Version`.\"\"\"\n if not self.python_version:\n return Version(\n self.shortened()\n if shortened\n else \".\".join(str(_) for _ in (self.major, self.minor, self.patch))\n )\n\n # The SemanticVersion was generated from a Version. Return the original\n # epoch (and the rest, if the release equals the current version).\n\n # epoch\n redone_version = (\n f\"{self.python_version.epoch}!\" if self.python_version.epoch != 0 else \"\"\n )\n\n # release\n if shortened:\n redone_version += self.shortened()\n else:\n redone_version += \".\".join(\n str(_) for _ in (self.major, self.minor, self.patch)\n )\n\n if (self.major, self.minor, self.patch)[\n : len(self.python_version.release)\n ] == self.python_version.release:\n # The release is the same as the current version. Add the pre, post, dev,\n # and local parts, if any.\n if self.python_version.pre is not None:\n redone_version += \"\".join(str(_) for _ in self.python_version.pre)\n\n if self.python_version.post is not None:\n redone_version += f\".post{self.python_version.post}\"\n\n if self.python_version.dev is not None:\n redone_version += f\".dev{self.python_version.dev}\"\n\n if self.python_version.local is not None:\n redone_version += f\"+{self.python_version.local}\"\n\n return Version(redone_version)\n\n def __str__(self) -> str:\n \"\"\"Return the full version.\"\"\"\n if self.python_version:\n return str(self.as_python_version(shortened=False))\n return (\n f\"{self.major}.{self.minor}.{self.patch}\"\n f\"{f'-{self.pre_release}' if self.pre_release else ''}\"\n f\"{f'+{self.build}' if self.build else ''}\"\n )\n\n def __repr__(self) -> str:\n \"\"\"Return the string representation of the object.\"\"\"\n return f\"{self.__class__.__name__}({self.__str__()!r})\"\n\n def __getattribute__(self, name: str) -> Any:\n \"\"\"Return the attribute value.\"\"\"\n accepted_python_attributes = (\n \"epoch\",\n \"release\",\n \"pre\",\n \"post\",\n \"dev\",\n \"local\",\n \"public\",\n \"base_version\",\n \"micro\",\n )\n\n try:\n return object.__getattribute__(self, name)\n except AttributeError as exc:\n # Try returning the attribute from the Python version, if it is in a list\n # of accepted attributes\n if name not in accepted_python_attributes:\n raise AttributeError(\n f\"{self.__class__.__name__} object has no attribute {name!r}\"\n ) from exc\n\n python_version = object.__getattribute__(self, \"as_python_version\")(\n shortened=False\n )\n try:\n return getattr(python_version, name)\n except AttributeError as exc:\n raise AttributeError(\n f\"{self.__class__.__name__} object has no attribute {name!r}\"\n ) from exc\n\n def _validate_other_type(self, other: Any) -> SemanticVersion:\n \"\"\"Initial check/validation of `other` before rich comparisons.\"\"\"\n not_implemented_exc = NotImplementedError(\n f\"Rich comparison not implemented between {self.__class__.__name__} and \"\n f\"{type(other)}\"\n )\n\n if isinstance(other, self.__class__):\n return other\n\n if isinstance(other, (Version, str)):\n try:\n return self.__class__(other)\n except (TypeError, ValueError) as exc:\n raise not_implemented_exc from exc\n\n raise not_implemented_exc\n\n def __lt__(self, other: Any) -> bool:\n \"\"\"Less than (`<`) rich comparison.\"\"\"\n other_semver = self._validate_other_type(other)\n\n if self.major < other_semver.major:\n return True\n if self.major == other_semver.major:\n if self.minor < other_semver.minor:\n return True\n if self.minor == other_semver.minor:\n if self.patch < other_semver.patch:\n return True\n if self.patch == other_semver.patch:\n if self.pre_release is None:\n return False\n if other_semver.pre_release is None:\n return True\n return self.pre_release < other_semver.pre_release\n return False\n\n def __le__(self, other: Any) -> bool:\n \"\"\"Less than or equal to (`<=`) rich comparison.\"\"\"\n return self.__lt__(other) or self.__eq__(other)\n\n def __eq__(self, other: object) -> bool:\n \"\"\"Equal to (`==`) rich comparison.\"\"\"\n other_semver = self._validate_other_type(other)\n\n return (\n self.major == other_semver.major\n and self.minor == other_semver.minor\n and self.patch == other_semver.patch\n and self.pre_release == other_semver.pre_release\n )\n\n def __ne__(self, other: object) -> bool:\n \"\"\"Not equal to (`!=`) rich comparison.\"\"\"\n return not self.__eq__(other)\n\n def __ge__(self, other: Any) -> bool:\n \"\"\"Greater than or equal to (`>=`) rich comparison.\"\"\"\n return not self.__lt__(other)\n\n def __gt__(self, other: Any) -> bool:\n \"\"\"Greater than (`>`) rich comparison.\"\"\"\n return not self.__le__(other)\n\n def next_version(self, version_part: str) -> SemanticVersion:\n \"\"\"Return the next version for the specified version part.\n\n Parameters:\n version_part: The version part to increment.\n\n Returns:\n The next version.\n\n Raises:\n ValueError: If the version part is not one of `major`, `minor`, or `patch`.\n\n \"\"\"\n if version_part not in (\"major\", \"minor\", \"patch\"):\n raise ValueError(\n \"version_part must be one of 'major', 'minor', or 'patch', not \"\n f\"{version_part!r}\"\n )\n\n if version_part == \"major\":\n next_version = f\"{self.major + 1}.0.0\"\n elif version_part == \"minor\":\n next_version = f\"{self.major}.{self.minor + 1}.0\"\n else:\n next_version = f\"{self.major}.{self.minor}.{self.patch + 1}\"\n\n return self.__class__(next_version)\n\n def previous_version(\n self, version_part: str, max_filler: str | int | None = None\n ) -> SemanticVersion:\n \"\"\"Return the previous version for the specified version part.\n\n Parameters:\n version_part: The version part to decrement.\n max_filler: The maximum value for the version part to decrement.\n\n Returns:\n The previous version.\n\n Raises:\n ValueError: If the version part is not one of `major`, `minor`, or `patch`.\n\n \"\"\"\n if version_part not in (\"major\", \"minor\", \"patch\"):\n raise ValueError(\n \"version_part must be one of 'major', 'minor', or 'patch', not \"\n f\"{version_part!r}\"\n )\n\n if max_filler is None:\n max_filler = 99\n elif isinstance(max_filler, str):\n max_filler = int(max_filler)\n\n if not isinstance(max_filler, int):\n raise TypeError(\"max_filler must be an integer, string or None\")\n\n if version_part == \"major\":\n prev_version = f\"{self.major - 1}.{max_filler}.{max_filler}\"\n\n elif version_part == \"minor\" or self.patch == 0:\n prev_version = (\n f\"{self.major - 1}.{max_filler}.{max_filler}\"\n if self.minor == 0\n else f\"{self.major}.{self.minor - 1}.{max_filler}\"\n )\n\n else:\n prev_version = f\"{self.major}.{self.minor}.{self.patch - 1}\"\n\n return self.__class__(prev_version)\n\n def shortened(self) -> str:\n \"\"\"Return a shortened version of the version.\n\n The shortened version is the full version, but without the patch and/or minor\n version if they are `0`, and without the pre-release and build metadata parts.\n\n Returns:\n The shortened version.\n\n \"\"\"\n if self.patch == 0:\n if self.minor == 0:\n return str(self.major)\n return f\"{self.major}.{self.minor}\"\n return f\"{self.major}.{self.minor}.{self.patch}\"\nbuild: str | None property readonly","text":"The build metadata part of the version.
This is the part supplied at the end of the version, after a plus (+).
major: int property readonly","text":"The major version.
"},{"location":"api_reference/utils/versions/#ci_cd.utils.versions.SemanticVersion.minor","title":"minor: int property readonly","text":"The minor version.
"},{"location":"api_reference/utils/versions/#ci_cd.utils.versions.SemanticVersion.patch","title":"patch: int property readonly","text":"The patch version.
"},{"location":"api_reference/utils/versions/#ci_cd.utils.versions.SemanticVersion.pre_release","title":"pre_release: str | None property readonly","text":"The pre-release part of the version
This is the part supplied after a minus (-), but before a plus (+).
python_version: Version | None property readonly","text":"The Python version as defined by packaging.version.Version.
__eq__(self, other) special","text":"Equal to (==) rich comparison.
ci_cd/utils/versions.py def __eq__(self, other: object) -> bool:\n \"\"\"Equal to (`==`) rich comparison.\"\"\"\n other_semver = self._validate_other_type(other)\n\n return (\n self.major == other_semver.major\n and self.minor == other_semver.minor\n and self.patch == other_semver.patch\n and self.pre_release == other_semver.pre_release\n )\n__ge__(self, other) special","text":"Greater than or equal to (>=) rich comparison.
ci_cd/utils/versions.py def __ge__(self, other: Any) -> bool:\n \"\"\"Greater than or equal to (`>=`) rich comparison.\"\"\"\n return not self.__lt__(other)\n__getattribute__(self, name) special","text":"Return the attribute value.
Source code inci_cd/utils/versions.py def __getattribute__(self, name: str) -> Any:\n \"\"\"Return the attribute value.\"\"\"\n accepted_python_attributes = (\n \"epoch\",\n \"release\",\n \"pre\",\n \"post\",\n \"dev\",\n \"local\",\n \"public\",\n \"base_version\",\n \"micro\",\n )\n\n try:\n return object.__getattribute__(self, name)\n except AttributeError as exc:\n # Try returning the attribute from the Python version, if it is in a list\n # of accepted attributes\n if name not in accepted_python_attributes:\n raise AttributeError(\n f\"{self.__class__.__name__} object has no attribute {name!r}\"\n ) from exc\n\n python_version = object.__getattribute__(self, \"as_python_version\")(\n shortened=False\n )\n try:\n return getattr(python_version, name)\n except AttributeError as exc:\n raise AttributeError(\n f\"{self.__class__.__name__} object has no attribute {name!r}\"\n ) from exc\n__gt__(self, other) special","text":"Greater than (>) rich comparison.
ci_cd/utils/versions.py def __gt__(self, other: Any) -> bool:\n \"\"\"Greater than (`>`) rich comparison.\"\"\"\n return not self.__le__(other)\n__le__(self, other) special","text":"Less than or equal to (<=) rich comparison.
ci_cd/utils/versions.py def __le__(self, other: Any) -> bool:\n \"\"\"Less than or equal to (`<=`) rich comparison.\"\"\"\n return self.__lt__(other) or self.__eq__(other)\n__lt__(self, other) special","text":"Less than (<) rich comparison.
ci_cd/utils/versions.py def __lt__(self, other: Any) -> bool:\n \"\"\"Less than (`<`) rich comparison.\"\"\"\n other_semver = self._validate_other_type(other)\n\n if self.major < other_semver.major:\n return True\n if self.major == other_semver.major:\n if self.minor < other_semver.minor:\n return True\n if self.minor == other_semver.minor:\n if self.patch < other_semver.patch:\n return True\n if self.patch == other_semver.patch:\n if self.pre_release is None:\n return False\n if other_semver.pre_release is None:\n return True\n return self.pre_release < other_semver.pre_release\n return False\n__ne__(self, other) special","text":"Not equal to (!=) rich comparison.
ci_cd/utils/versions.py def __ne__(self, other: object) -> bool:\n \"\"\"Not equal to (`!=`) rich comparison.\"\"\"\n return not self.__eq__(other)\n__new__(cls, version=None, **kwargs) special staticmethod","text":"Create and return a new object. See help(type) for accurate signature.
Source code inci_cd/utils/versions.py @no_type_check\ndef __new__(cls, version: str | Version | None = None, **kwargs: str | int) -> Self:\n return super().__new__(\n cls, str(version) if version else cls._build_version(**kwargs)\n )\n__repr__(self) special","text":"Return the string representation of the object.
Source code inci_cd/utils/versions.py def __repr__(self) -> str:\n \"\"\"Return the string representation of the object.\"\"\"\n return f\"{self.__class__.__name__}({self.__str__()!r})\"\n__str__(self) special","text":"Return the full version.
Source code inci_cd/utils/versions.py def __str__(self) -> str:\n \"\"\"Return the full version.\"\"\"\n if self.python_version:\n return str(self.as_python_version(shortened=False))\n return (\n f\"{self.major}.{self.minor}.{self.patch}\"\n f\"{f'-{self.pre_release}' if self.pre_release else ''}\"\n f\"{f'+{self.build}' if self.build else ''}\"\n )\nas_python_version(self, shortened=True)","text":"Return the Python version as defined by packaging.version.Version.
ci_cd/utils/versions.py def as_python_version(self, shortened: bool = True) -> Version:\n \"\"\"Return the Python version as defined by `packaging.version.Version`.\"\"\"\n if not self.python_version:\n return Version(\n self.shortened()\n if shortened\n else \".\".join(str(_) for _ in (self.major, self.minor, self.patch))\n )\n\n # The SemanticVersion was generated from a Version. Return the original\n # epoch (and the rest, if the release equals the current version).\n\n # epoch\n redone_version = (\n f\"{self.python_version.epoch}!\" if self.python_version.epoch != 0 else \"\"\n )\n\n # release\n if shortened:\n redone_version += self.shortened()\n else:\n redone_version += \".\".join(\n str(_) for _ in (self.major, self.minor, self.patch)\n )\n\n if (self.major, self.minor, self.patch)[\n : len(self.python_version.release)\n ] == self.python_version.release:\n # The release is the same as the current version. Add the pre, post, dev,\n # and local parts, if any.\n if self.python_version.pre is not None:\n redone_version += \"\".join(str(_) for _ in self.python_version.pre)\n\n if self.python_version.post is not None:\n redone_version += f\".post{self.python_version.post}\"\n\n if self.python_version.dev is not None:\n redone_version += f\".dev{self.python_version.dev}\"\n\n if self.python_version.local is not None:\n redone_version += f\"+{self.python_version.local}\"\n\n return Version(redone_version)\nnext_version(self, version_part)","text":"Return the next version for the specified version part.
Parameters:
Name Type Description Defaultversion_part str The version part to increment.
requiredReturns:
Type DescriptionSemanticVersion The next version.
Exceptions:
Type DescriptionValueError If the version part is not one of major, minor, or patch.
ci_cd/utils/versions.py def next_version(self, version_part: str) -> SemanticVersion:\n \"\"\"Return the next version for the specified version part.\n\n Parameters:\n version_part: The version part to increment.\n\n Returns:\n The next version.\n\n Raises:\n ValueError: If the version part is not one of `major`, `minor`, or `patch`.\n\n \"\"\"\n if version_part not in (\"major\", \"minor\", \"patch\"):\n raise ValueError(\n \"version_part must be one of 'major', 'minor', or 'patch', not \"\n f\"{version_part!r}\"\n )\n\n if version_part == \"major\":\n next_version = f\"{self.major + 1}.0.0\"\n elif version_part == \"minor\":\n next_version = f\"{self.major}.{self.minor + 1}.0\"\n else:\n next_version = f\"{self.major}.{self.minor}.{self.patch + 1}\"\n\n return self.__class__(next_version)\nprevious_version(self, version_part, max_filler=None)","text":"Return the previous version for the specified version part.
Parameters:
Name Type Description Defaultversion_part str The version part to decrement.
requiredmax_filler str | int | None The maximum value for the version part to decrement.
None Returns:
Type DescriptionSemanticVersion The previous version.
Exceptions:
Type DescriptionValueError If the version part is not one of major, minor, or patch.
ci_cd/utils/versions.py def previous_version(\n self, version_part: str, max_filler: str | int | None = None\n) -> SemanticVersion:\n \"\"\"Return the previous version for the specified version part.\n\n Parameters:\n version_part: The version part to decrement.\n max_filler: The maximum value for the version part to decrement.\n\n Returns:\n The previous version.\n\n Raises:\n ValueError: If the version part is not one of `major`, `minor`, or `patch`.\n\n \"\"\"\n if version_part not in (\"major\", \"minor\", \"patch\"):\n raise ValueError(\n \"version_part must be one of 'major', 'minor', or 'patch', not \"\n f\"{version_part!r}\"\n )\n\n if max_filler is None:\n max_filler = 99\n elif isinstance(max_filler, str):\n max_filler = int(max_filler)\n\n if not isinstance(max_filler, int):\n raise TypeError(\"max_filler must be an integer, string or None\")\n\n if version_part == \"major\":\n prev_version = f\"{self.major - 1}.{max_filler}.{max_filler}\"\n\n elif version_part == \"minor\" or self.patch == 0:\n prev_version = (\n f\"{self.major - 1}.{max_filler}.{max_filler}\"\n if self.minor == 0\n else f\"{self.major}.{self.minor - 1}.{max_filler}\"\n )\n\n else:\n prev_version = f\"{self.major}.{self.minor}.{self.patch - 1}\"\n\n return self.__class__(prev_version)\nshortened(self)","text":"Return a shortened version of the version.
The shortened version is the full version, but without the patch and/or minor version if they are 0, and without the pre-release and build metadata parts.
Returns:
Type Descriptionstr The shortened version.
Source code inci_cd/utils/versions.py def shortened(self) -> str:\n \"\"\"Return a shortened version of the version.\n\n The shortened version is the full version, but without the patch and/or minor\n version if they are `0`, and without the pre-release and build metadata parts.\n\n Returns:\n The shortened version.\n\n \"\"\"\n if self.patch == 0:\n if self.minor == 0:\n return str(self.major)\n return f\"{self.major}.{self.minor}\"\n return f\"{self.major}.{self.minor}.{self.patch}\"\ncreate_ignore_rules(specifier_set)","text":"Create ignore rules based on version specifier set.
The only ignore rules needed are related to versions that should be explicitly avoided, i.e., the != operator. All other specifiers should require an explicit ignore rule by the user, if no update should be suggested.
ci_cd/utils/versions.py def create_ignore_rules(specifier_set: SpecifierSet) -> IgnoreRules:\n \"\"\"Create ignore rules based on version specifier set.\n\n The only ignore rules needed are related to versions that should be explicitly\n avoided, i.e., the `!=` operator. All other specifiers should require an explicit\n ignore rule by the user, if no update should be suggested.\n \"\"\"\n return {\n \"versions\": [\n f\"=={specifier.version}\"\n for specifier in specifier_set\n if specifier.operator == \"!=\"\n ]\n }\nfind_minimum_py_version(marker, project_py_version)","text":"Find the minimum Python version from a marker.
Source code inci_cd/utils/versions.py def find_minimum_py_version(marker: Marker, project_py_version: str) -> str:\n \"\"\"Find the minimum Python version from a marker.\"\"\"\n split_py_version = project_py_version.split(\".\")\n\n def _next_version(_version: SemanticVersion) -> SemanticVersion:\n if len(split_py_version) == PART_TO_LENGTH_MAPPING[\"major\"]:\n return _version.next_version(\"major\")\n if len(split_py_version) == PART_TO_LENGTH_MAPPING[\"minor\"]:\n return _version.next_version(\"minor\")\n return _version.next_version(\"patch\")\n\n min_py_version = SemanticVersion(project_py_version)\n\n environment_keys = default_environment().keys()\n empty_environment = {key: \"\" for key in environment_keys}\n python_version_centric_environment = empty_environment\n python_version_centric_environment.update({\"python_version\": min_py_version})\n\n while not _semi_valid_python_version(min_py_version) or not marker.evaluate(\n environment=python_version_centric_environment\n ):\n min_py_version = _next_version(min_py_version)\n python_version_centric_environment.update({\"python_version\": min_py_version})\n\n return min_py_version.shortened()\nget_min_max_py_version(requires_python)","text":"Get minimum or maximum Python version from requires_python.
This also works for parsing requirement markers specifying validity for a specific python_version.
A minimum version will be preferred.
This means all the minimum operators will be checked first, and if none of them match, then all the maximum operators will be checked. See below to understand which operators are minimum and which are maximum.
Note
The first operator to match will be used, so if a minimum operator matches, then the maximum operators will not be checked.
Whether it will minimum or maximum will depend on the operator: Minimum: >=, ==, ~=, > Maximum: <=, <
Returns:
Type Descriptionstr The minimum or maximum Python version. E.g., if requires_python is \">=3.6\", then \"3.6\" (min) will be returned, and if requires_python is Marker(\"python_version < '3.6'\"), then \"3.5.99\" (max) will be returned.
ci_cd/utils/versions.py def get_min_max_py_version(\n requires_python: str | Marker,\n) -> str:\n \"\"\"Get minimum or maximum Python version from `requires_python`.\n\n This also works for parsing requirement markers specifying validity for a specific\n `python_version`.\n\n _A minimum version will be preferred._\n\n This means all the minimum operators will be checked first, and if none of them\n match, then all the maximum operators will be checked.\n See below to understand which operators are minimum and which are maximum.\n\n Note:\n The first operator to match will be used, so if a minimum operator matches,\n then the maximum operators will not be checked.\n\n Whether it will minimum or maximum will depend on the operator:\n Minimum: `>=`, `==`, `~=`, `>`\n Maximum: `<=`, `<`\n\n Returns:\n The minimum or maximum Python version.\n E.g., if `requires_python` is `\">=3.6\"`, then `\"3.6\"` (min) will be returned,\n and if `requires_python` is `Marker(\"python_version < '3.6'\")`, then `\"3.5.99\"`\n (max) will be returned.\n\n \"\"\"\n if isinstance(requires_python, Marker):\n match = re.search(\n r\"python_version\\s*\"\n r\"(?P<operator>==|!=|<=|>=|<|>|~=)\\s*\"\n r\"('|\\\")(?P<version>[0-9]+(?:\\.[0-9]+)*)('|\\\")\",\n str(requires_python),\n )\n\n if match is None:\n raise UnableToResolve(\"Could not retrieve 'python_version' marker.\")\n\n requires_python = f\"{match.group('operator')}{match.group('version')}\"\n\n try:\n specifier_set = SpecifierSet(requires_python)\n except InvalidSpecifier as exc:\n raise UnableToResolve(\n \"Cannot parse 'requires_python' as a specifier set.\"\n ) from exc\n\n py_version = \"\"\n min_or_max = \"min\"\n\n length_to_part_mapping = {\n 1: \"major\",\n 2: \"minor\",\n 3: \"patch\",\n }\n\n # Minimum\n for specifier in specifier_set:\n if specifier.operator in [\">=\", \"==\", \"~=\"]:\n py_version = specifier.version\n break\n\n if specifier.operator == \">\":\n split_version = specifier.version.split(\".\")\n parsed_version = SemanticVersion(specifier.version)\n\n if len(split_version) == PART_TO_LENGTH_MAPPING[\"major\"]:\n py_version = str(parsed_version.next_version(\"major\").major)\n elif len(split_version) == PART_TO_LENGTH_MAPPING[\"minor\"]:\n py_version = \".\".join(\n parsed_version.next_version(\"minor\").split(\".\")[:2]\n )\n elif len(split_version) == PART_TO_LENGTH_MAPPING[\"patch\"]:\n py_version = str(parsed_version.next_version(\"patch\"))\n\n break\n else:\n # Maximum\n min_or_max = \"max\"\n\n for specifier in specifier_set:\n if specifier.operator == \"<=\":\n py_version = specifier.version\n break\n\n if specifier.operator == \"<\":\n split_version = specifier.version.split(\".\")\n parsed_version = SemanticVersion(specifier.version)\n\n if parsed_version == SemanticVersion(\"0\"):\n raise UnableToResolve(\n f\"{specifier} is not a valid Python version specifier.\"\n )\n\n if len(split_version) not in length_to_part_mapping:\n raise UnableToResolve(\n f\"{specifier} is not a valid Python version specifier. It was \"\n \"expected to be a major, minor, or patch version specifier.\"\n )\n\n # Fill with 0's and shorten. This is an attempt to return a full range\n # of previous versions.\n py_version = str(\n parsed_version.previous_version(\n length_to_part_mapping[len(split_version)],\n max_filler=0,\n ).shortened()\n )\n\n break\n else:\n raise UnableToResolve(\n \"Cannot determine min/max Python version from version specifier(s): \"\n f\"{specifier_set}\"\n )\n\n if py_version not in specifier_set:\n split_py_version = py_version.split(\".\")\n parsed_py_version = SemanticVersion(py_version)\n\n # See the _semi_valid_python_version() function for these values\n largest_value_for_a_patch_part = 18\n largest_value_for_a_minor_part = 12\n largest_value_for_a_major_part = 3\n largest_value_for_any_part = max(\n largest_value_for_a_patch_part,\n largest_value_for_a_minor_part,\n largest_value_for_a_major_part,\n )\n\n while (\n not _semi_valid_python_version(parsed_py_version)\n or py_version not in specifier_set\n ):\n if min_or_max == \"min\":\n if parsed_py_version.patch >= largest_value_for_a_patch_part:\n parsed_py_version = parsed_py_version.next_version(\"minor\")\n elif parsed_py_version.minor >= largest_value_for_a_minor_part:\n parsed_py_version = parsed_py_version.next_version(\"major\")\n else:\n parsed_py_version = parsed_py_version.next_version(\"patch\")\n else:\n parsed_py_version = parsed_py_version.previous_version(\n length_to_part_mapping[len(split_py_version)],\n max_filler=largest_value_for_any_part,\n )\n\n py_version = parsed_py_version.shortened()\n split_py_version = py_version.split(\".\")\n\n return py_version\nignore_version(current, latest, version_rules, semver_rules)","text":"Determine whether the latest version can be ignored.
Parameters:
Name Type Description Defaultcurrent list[str] The current version as a list of version parts. It's expected, but not required, the version is a semantic version.
requiredlatest list[str] The latest version as a list of version parts. It's expected, but not required, the version is a semantic version.
requiredversion_rules IgnoreVersions Version ignore rules.
requiredsemver_rules IgnoreUpdateTypes Semantic version ignore rules.
requiredReturns:
Type Descriptionbool Whether or not the latest version can be ignored based on the version and semantic version ignore rules.
Source code inci_cd/utils/versions.py def ignore_version(\n current: list[str],\n latest: list[str],\n version_rules: IgnoreVersions,\n semver_rules: IgnoreUpdateTypes,\n) -> bool:\n \"\"\"Determine whether the latest version can be ignored.\n\n Parameters:\n current: The current version as a list of version parts. It's expected, but not\n required, the version is a semantic version.\n latest: The latest version as a list of version parts. It's expected, but not\n required, the version is a semantic version.\n version_rules: Version ignore rules.\n semver_rules: Semantic version ignore rules.\n\n Returns:\n Whether or not the latest version can be ignored based on the version and\n semantic version ignore rules.\n\n \"\"\"\n # ignore all updates\n if not version_rules and not semver_rules:\n # A package name has been specified without specific rules, ignore all updates\n # for package.\n return True\n\n # version rules\n if _ignore_version_rules_specifier_set(latest, version_rules):\n return True\n\n # semver rules\n return bool(\n \"version-update\" in semver_rules\n and _ignore_semver_rules(current, latest, semver_rules)\n )\nparse_ignore_entries(entries, separator)","text":"Parser for the --ignore option.
The --ignore option values are given as key/value-pairs in the form: key=value...key=value. Here ... is the separator value supplied by --ignore-separator.
Parameters:
Name Type Description Defaultentries list[str] The list of supplied --ignore options.
separator str The supplied --ignore-separator value.
Returns:
Type DescriptionIgnoreRulesCollection A parsed mapping of dependencies to ignore rules.
Source code inci_cd/utils/versions.py def parse_ignore_entries(entries: list[str], separator: str) -> IgnoreRulesCollection:\n \"\"\"Parser for the `--ignore` option.\n\n The `--ignore` option values are given as key/value-pairs in the form:\n `key=value...key=value`. Here `...` is the separator value supplied by\n `--ignore-separator`.\n\n Parameters:\n entries: The list of supplied `--ignore` options.\n separator: The supplied `--ignore-separator` value.\n\n Returns:\n A parsed mapping of dependencies to ignore rules.\n\n \"\"\"\n ignore_entries: IgnoreRulesCollection = {}\n\n for entry in entries:\n pairs = entry.split(separator, maxsplit=2)\n for pair in pairs:\n if separator in pair:\n raise InputParserError(\n \"More than three key/value-pairs were given for an `--ignore` \"\n \"option, while there are only three allowed key names. Input \"\n f\"value: --ignore={entry!r}\"\n )\n\n ignore_entry: IgnoreEntry = {}\n for pair in pairs:\n match = re.match(\n r\"^(?P<key>dependency-name|versions|update-types)=(?P<value>.*)$\",\n pair,\n )\n if match is None:\n raise InputParserError(\n f\"Could not parse ignore configuration: {pair!r} (part of the \"\n f\"ignore option: {entry!r})\"\n )\n\n parsed_pair = IgnoreEntryPair(**match.groupdict()) # type: ignore[arg-type]\n\n if parsed_pair.key in ignore_entry:\n raise InputParserError(\n \"An ignore configuration can only be given once per option. The \"\n f\"configuration key {parsed_pair.key!r} was found multiple \"\n f\"times in the option {entry!r}\"\n )\n\n ignore_entry[parsed_pair.key] = parsed_pair.value.strip()\n\n if \"dependency-name\" not in ignore_entry:\n raise InputError(\n \"Ignore option entry missing required 'dependency-name' \"\n f\"configuration. Ignore option entry: {entry}\"\n )\n\n dependency_name = ignore_entry[\"dependency-name\"]\n if dependency_name not in ignore_entries:\n ignore_entries[dependency_name] = {\n key: [value]\n for key, value in ignore_entry.items()\n if key != \"dependency-name\"\n }\n else:\n for key, value in ignore_entry.items():\n if key != \"dependency-name\":\n ignore_entries[dependency_name][key].append(value)\n\n return ignore_entries\nparse_ignore_rules(rules)","text":"Parser for a specific set of ignore rules.
Parameters:
Name Type Description Defaultrules IgnoreRules A set of ignore rules for one or more packages.
requiredReturns:
Type Descriptiontuple[IgnoreVersions, IgnoreUpdateTypes] A tuple of the parsed 'versions' and 'update-types' entries as dictionaries.
Source code inci_cd/utils/versions.py def parse_ignore_rules(\n rules: IgnoreRules,\n) -> tuple[IgnoreVersions, IgnoreUpdateTypes]:\n \"\"\"Parser for a specific set of ignore rules.\n\n Parameters:\n rules: A set of ignore rules for one or more packages.\n\n Returns:\n A tuple of the parsed 'versions' and 'update-types' entries as dictionaries.\n\n \"\"\"\n if not rules:\n # Ignore package altogether\n return [{\"operator\": \">=\", \"version\": \"0\"}], {}\n\n versions: IgnoreVersions = []\n update_types: IgnoreUpdateTypes = {}\n\n if \"versions\" in rules:\n for versions_entry in rules[\"versions\"]:\n match = re.match(\n r\"^(?P<operator>>|<|<=|>=|==|!=|~=)\\s*(?P<version>\\S+)$\",\n versions_entry,\n )\n if match is None:\n raise InputParserError(\n \"Ignore option's 'versions' value cannot be parsed. It \"\n \"must be a single operator followed by a version number.\\n\"\n f\"Unparseable 'versions' value: {versions_entry!r}\"\n )\n versions.append(match.groupdict()) # type: ignore[arg-type]\n\n if \"update-types\" in rules:\n update_types[\"version-update\"] = []\n for update_type_entry in rules[\"update-types\"]:\n match = re.match(\n r\"^version-update:semver-(?P<semver_part>major|minor|patch)$\",\n update_type_entry,\n )\n if match is None:\n raise InputParserError(\n \"Ignore option's 'update-types' value cannot be parsed.\"\n \" It must be either: 'version-update:semver-major', \"\n \"'version-update:semver-minor' or \"\n \"'version-update:semver-patch'.\\nUnparseable 'update-types' \"\n f\"value: {update_type_entry!r}\"\n )\n update_types[\"version-update\"].append(match.group(\"semver_part\")) # type: ignore[arg-type]\n\n return versions, update_types\nregenerate_requirement(requirement, *, name=None, extras=None, specifier=None, url=None, marker=None, post_name_space=False)","text":"Regenerate a requirement string including the given parameters.
Parameters:
Name Type Description Defaultrequirement Requirement The requirement to regenerate and fallback to.
requiredname str | None A new name to use for the requirement.
None extras set[str] | None New extras to use for the requirement.
None specifier SpecifierSet | str | None A new specifier set to use for the requirement.
None url str | None A new URL to use for the requirement.
None marker Marker | str | None A new marker to use for the requirement.
None post_name_space bool Whether or not to add a single space after the name (possibly including extras), but before the specifier.
False Returns:
Type Descriptionstr The regenerated requirement string.
Source code inci_cd/utils/versions.py def regenerate_requirement(\n requirement: Requirement,\n *,\n name: str | None = None,\n extras: set[str] | None = None,\n specifier: SpecifierSet | str | None = None,\n url: str | None = None,\n marker: Marker | str | None = None,\n post_name_space: bool = False,\n) -> str:\n \"\"\"Regenerate a requirement string including the given parameters.\n\n Parameters:\n requirement: The requirement to regenerate and fallback to.\n name: A new name to use for the requirement.\n extras: New extras to use for the requirement.\n specifier: A new specifier set to use for the requirement.\n url: A new URL to use for the requirement.\n marker: A new marker to use for the requirement.\n post_name_space: Whether or not to add a single space after the name (possibly\n including extras), but before the specifier.\n\n Returns:\n The regenerated requirement string.\n\n \"\"\"\n updated_dependency = name or requirement.name\n\n if extras or requirement.extras:\n formatted_extras = \",\".join(sorted(extras or requirement.extras))\n updated_dependency += f\"[{formatted_extras}]\"\n\n if post_name_space:\n updated_dependency += \" \"\n\n if specifier or requirement.specifier:\n if specifier and not isinstance(specifier, SpecifierSet):\n specifier = SpecifierSet(specifier)\n updated_dependency += \",\".join(\n str(_)\n for _ in sorted(\n specifier or requirement.specifier,\n key=lambda spec: spec.operator,\n reverse=True,\n )\n )\n\n if url or requirement.url:\n updated_dependency += f\"@ {url or requirement.url}\"\n if marker or requirement.marker:\n updated_dependency += \" \"\n\n if marker or requirement.marker:\n updated_dependency += f\"; {marker or requirement.marker}\"\n\n return updated_dependency\nupdate_specifier_set(latest_version, current_specifier_set)","text":"Update the specifier set to include the latest version.
Source code inci_cd/utils/versions.py def update_specifier_set(\n latest_version: SemanticVersion | Version | str, current_specifier_set: SpecifierSet\n) -> SpecifierSet:\n \"\"\"Update the specifier set to include the latest version.\"\"\"\n logger = logging.getLogger(__name__)\n\n latest_version = SemanticVersion(latest_version)\n\n new_specifier_set = set(current_specifier_set)\n updated_specifiers = []\n split_latest_version = (\n latest_version.as_python_version(shortened=False).base_version.split(\".\")\n if latest_version.python_version\n else latest_version.split(\".\")\n )\n current_version_epochs = {Version(_.version).epoch for _ in current_specifier_set}\n\n logger.debug(\n \"Received latest version: %s and current specifier set: %s\",\n latest_version,\n current_specifier_set,\n )\n\n if latest_version in current_specifier_set:\n # The latest version is already included in the specifier set.\n # Update specifier set if the latest version is included via a `~=` or a `==`\n # operator.\n for specifier in current_specifier_set:\n if specifier.operator in [\"~=\", \"==\"]:\n split_specifier_version = specifier.version.split(\".\")\n updated_version = \".\".join(\n split_latest_version[: len(split_specifier_version)]\n )\n updated_specifiers.append(f\"{specifier.operator}{updated_version}\")\n new_specifier_set.remove(specifier)\n break\n else:\n # The latest version is already included in the specifier set, and the set\n # does not need updating. To communicate this, make updated_specifiers\n # non-empty, but include only an empty string.\n updated_specifiers.append(\"\")\n\n elif (\n latest_version.python_version\n and latest_version.as_python_version().epoch not in current_version_epochs\n ):\n # The latest version is *not* included in the specifier set.\n # And the latest version is NOT in the same epoch as the current version range.\n\n # Sanity check that the latest version's epoch is larger than the largest\n # epoch in the specifier set.\n if current_version_epochs and latest_version.as_python_version().epoch < max(\n current_version_epochs\n ):\n raise UnableToResolve(\n \"The latest version's epoch is smaller than the largest epoch in \"\n \"the specifier set.\"\n )\n\n # Simply add the latest version as a specifier.\n updated_specifiers.append(f\"=={latest_version}\")\n\n else:\n # The latest version is *not* included in the specifier set.\n # But we're in the right epoch.\n\n # Expect the latest version to be greater than the current version range.\n for specifier in current_specifier_set:\n # Simply expand the range if the version range is capped through a specifier\n # using either of the `<` or `<=` operators.\n if specifier.operator == \"<=\":\n split_specifier_version = specifier.version.split(\".\")\n updated_version = \".\".join(\n split_latest_version[: len(split_specifier_version)]\n )\n\n updated_specifiers.append(f\"{specifier.operator}{updated_version}\")\n new_specifier_set.remove(specifier)\n break\n\n if specifier.operator == \"<\":\n # Update to include latest version by upping to the next\n # version up from the latest version\n split_specifier_version = specifier.version.split(\".\")\n\n updated_version = \"\"\n\n # Add epoch if present\n if (\n latest_version.python_version\n and latest_version.as_python_version().epoch != 0\n ):\n updated_version += f\"{latest_version.as_python_version().epoch}!\"\n\n # Up only the last version segment of the latest version according to\n # what version segments are defined in the specifier version.\n if len(split_specifier_version) == PART_TO_LENGTH_MAPPING[\"major\"]:\n updated_version += str(latest_version.next_version(\"major\").major)\n elif len(split_specifier_version) == PART_TO_LENGTH_MAPPING[\"minor\"]:\n updated_version += \".\".join(\n latest_version.next_version(\"minor\").split(\".\")[:2]\n )\n elif len(split_specifier_version) == PART_TO_LENGTH_MAPPING[\"patch\"]:\n updated_version += latest_version.next_version(\"patch\")\n else:\n raise UnableToResolve(\n \"Invalid/unable to handle number of version parts: \"\n f\"{len(split_specifier_version)}\"\n )\n\n updated_specifiers.append(f\"{specifier.operator}{updated_version}\")\n new_specifier_set.remove(specifier)\n break\n\n if specifier.operator == \"~=\":\n # Expand and change ~= to >= and < operators if the latest version\n # changes major version. Otherwise, update to include latest version as\n # the minimum version\n current_version = SemanticVersion(specifier.version)\n\n # Add epoch if present\n epoch = \"\"\n if (\n latest_version.python_version\n and latest_version.as_python_version().epoch != 0\n ):\n epoch += f\"{latest_version.as_python_version().epoch}!\"\n\n if latest_version.major > current_version.major:\n # Expand and change ~= to >= and < operators\n\n # >= current_version (fully padded)\n updated_specifiers.append(f\">={current_version}\")\n\n # < next major version up from latest_version\n updated_specifiers.append(\n f\"<{epoch}{latest_version.next_version('major').major!s}\"\n )\n else:\n # Keep the ~= operator, but update to include the latest version as\n # the minimum version\n split_specifier_version = specifier.version.split(\".\")\n updated_version = \".\".join(\n split_latest_version[: len(split_specifier_version)]\n )\n updated_specifiers.append(f\"{specifier.operator}{updated_version}\")\n\n new_specifier_set.remove(specifier)\n break\n\n # Finally, add updated specifier(s) to new specifier set or raise.\n if updated_specifiers:\n # If updated_specifiers includes only an empty string, it means that the\n # current specifier set is valid as is and already includes the latest version\n if updated_specifiers != [\"\"]:\n # Otherwise, add updated specifier(s) to new specifier set\n new_specifier_set |= {Specifier(_) for _ in updated_specifiers}\n else:\n raise UnableToResolve(\n \"Cannot resolve how to update specifier set to include latest version.\"\n )\n\n return SpecifierSet(\",\".join(str(_) for _ in new_specifier_set))\npre-commit is an excellent tool for running CI tasks prior to committing new changes. The tasks are called \"hooks\" and are run in a separate virtual environment. The hooks usually change files in-place, meaning after pre-commit is run during a git commit command, the changed files can be reviewed and re-staged and committed.
Through SINTEF/ci-cd several hooks are available, mainly related to the GitHub Actions callable/reusable workflows that are also available in this repository.
This section contains all the available pre-commit hooks:
pyproject.tomlpre-commit hook id: docs-api-reference
Run this hook to update the API Reference section of your documentation.
The hook walks through a package directory, finding all Python files and creating a markdown file matching it along with recreating the Python API tree under the docs/api_reference/ folder.
The hook will run when any Python file is changed in the repository.
The hook expects the documentation to be setup with the MkDocs framework, including the mkdocstrings plugin for parsing in the Python class/function and method doc-strings, as well as the awesome-pages plugin for providing proper titles in the table-of-contents navigation.
"},{"location":"hooks/docs_api_reference/#using-it-together-with-cicd-workflows","title":"Using it together with CI/CD workflows","text":"If this hook is being used together with the workflow CI - Tests, to test if the documentation can be built, or CD - Release and/or CI/CD - New updates to default branch, to build and publish the documentation upon a release or push to the default branch, it is necessary to understand the way the Python API modules are referenced in the markdown files under docs/api_reference/.
By default, the references refer to the Python import path of a module. However, should a package be installed as an editable installation, i.e., using pip install -e, then the relative path from the repository root will be used.
This differentiation is only relevant for repositories, where these two cases are not aligned, such as when the Python package folder is in a nested folder, e.g., src/my_package/.
In order to remedy this, there are a single configuration in each workflow and this hooks that needs to be set to the same value. For this hook, the option name is --relative and the value for using the relative path, i.e., an editable installation, is to simply include this toggle option. If the option is not included, then a non-editable installation is assumed, i.e., the -e option is not used when installing the package, and a proper resolvable Python import statement is used as a link in the API reference markdown files. The latter is the default.
For the workflows, one should set the configuration option relative to true to use the relative path, i.e., an editable installation. And likewise set relative to false if a proper resolvable Python import statement is to be used, without forcing the -e option. The latter is the default.
It is required to specify the --package-dir argument at least once through the args key.
Otherwise, as noted above, without the proper framework, the created markdown files will not bring about the desired result in a built documentation.
"},{"location":"hooks/docs_api_reference/#options","title":"Options","text":"Any of these options can be given through the args key when defining the hook.
--package-dir Relative path to a package dir from the repository root, e.g., 'src/my_package'.This input option can be supplied multiple times. Yes string --docs-folder The folder name for the documentation root folder. No string docs --unwanted-folder A folder to avoid including into the Python API reference documentation. If this is not supplied, it will default to __pycache__.Note: Only folder names, not paths, may be included.Note: All folders and their contents with these names will be excluded.This input option can be supplied multiple times. No string __pycache__ --unwanted-file A file to avoid including into the Python API reference documentation. If this is not supplied, it will default to __init__.pyNote: Only full file names, not paths, may be included, i.e., filename + file extension.Note: All files with these names will be excluded.This input option can be supplied multiple times. No string __init__.py --full-docs-folder A folder in which to include everything - even those without documentation strings. This may be useful for a module full of data models or to ensure all class attributes are listed.This input option can be supplied multiple times. No string --full-docs-file A full relative path to a file in which to include everything - even those without documentation strings. This may be useful for a file full of data models or to ensure all class attributes are listed.This input option can be supplied multiple times. No string --special-option A combination of a relative path to a file and a fully formed mkdocstrings option that should be added to the generated MarkDown file. The combination should be comma-separated.Example: my_module/py_file.py,show_bases:false.Encapsulate the value in double quotation marks (\") if including spaces ( ).Important: If multiple package-dir options are supplied, the relative path MUST include/start with the package-dir value, e.g., \"my_package/my_module/py_file.py,show_bases: false\".This input option can be supplied multiple times. The options will be accumulated for the same file, if given several times. No string --relative Whether or not to use relative Python import links in the API reference markdown files. See section Using it together with CI/CD workflows above. No flag --debug Whether or not to print debug statements. No flag"},{"location":"hooks/docs_api_reference/#usage-example","title":"Usage example","text":"The following is an example of how an addition of the Update API Reference in Documentation hook into a .pre-commit-config.yaml file may look. It is meant to be complete as is.
repos:\n - repo: https://github.com/SINTEF/ci-cd\n rev: v2.8.3\n hooks:\n - id: docs-api-reference\n args:\n - --package-dir\n - src/my_python_package\n - --package-dir\n - src/my_other_python_package\n - --full-docs-folder\n - models\n - --full-docs-folder\n - data\npre-commit hook id: docs-landing-page
Run this hook to update the landing page (root index.md file) for your documentation.
The hook copies the root README.md file into the root of your documentation folder, renaming it to index.md and implementing any replacements specified.
The hook will run when the root README.md file is changed in the repository.
The hook expects the documentation to be a framework that can build markdown files for deploying a documentation site.
"},{"location":"hooks/docs_landing_page/#expectations","title":"Expectations","text":"It is required that the root README.md exists and the documentation's landing page is named index.md and can be found in the root of the documentation folder.
Any of these options can be given through the args key when defining the hook.
--docs-folder The folder name for the documentation root folder. No string docs --replacement A replacement (mapping) to be performed on README.md when creating the documentation's landing page (index.md). This list always includes replacing '--docs-folder/' with an empty string, in order to correct relative links. By default the value (LICENSE),(LICENSE.md) is set, but this will be overwritten if args is set.This input option can be supplied multiple times. No string (LICENSE),(LICENSE.md) --replacement-separator String to separate a replacement's 'old' to 'new' parts. Defaults to a comma (,). No string ,"},{"location":"hooks/docs_landing_page/#usage-example","title":"Usage example","text":"The following is an example of how an addition of the Update Landing Page (index.md) for Documentation hook into a .pre-commit-config.yaml file may look. It is meant to be complete as is.
repos:\n - repo: https://github.com/SINTEF/ci-cd\n rev: v2.8.3\n hooks:\n - id: docs-landing-page\n args:\n # Replace `(LICENSE)` with `(LICENSE.md)` (i.e., don't overwrite the default)\n - '--replacement'\n - '(LICENSE);(LICENSE.md)'\n # Replace `(tools/` with `(`\n - '--replacement'\n - '(tools/;('\n - '--replacement-separator'\n - ';'\npyproject.toml","text":"pre-commit hook id: update-pyproject
Run this hook to update the dependencies in your pyproject.toml file.
The hook utilizes pip index versions to determine the latest version available for all required and optional dependencies listed in your pyproject.toml file. It checks this based on the Python version listed as the minimum supported Python version by the package (defined through the requires-python key in your pyproject.toml file).
To ignore or configure how specific dependencies should be updated, the --ignore argument option can be utilized. This is done by specifying a line per dependency that contains --ignore-separator-separated (defaults to ellipsis (...)) key/value-pairs of:
dependency-name Ignore updates for dependencies with matching names, optionally using * to match zero or more characters. versions Ignore specific versions or ranges of versions. Examples: ~=1.0.5, >= 1.0.5,<2, >=0.1.1. update-types Ignore types of updates, such as SemVer major, minor, patch updates on version updates (for example: version-update:semver-patch will ignore patch updates). This can be combined with dependency-name=* to ignore particular update-types for all dependencies. Supported update-types values
Currently, only version-update:semver-major, version-update:semver-minor, and version-update:semver-patch are supported options for update-types.
The --ignore option is essentially similar to the ignore option of Dependabot. If versions and update-types are used together, they will both be respected jointly.
Here are some examples of different values that may be given for the --ignore option that accomplishes different things:
Value: dependency-name=Sphinx...versions=>=4.5.0 Accomplishes: For Sphinx, ignore all updates for/from version 4.5.0 and up / keep the minimum version for Sphinx at 4.5.0.
Value: dependency-name=pydantic...update-types=version-update:semver-patch Accomplishes: For pydantic, ignore all patch updates.
Value: dependency-name=numpy Accomplishes: For NumPy, ignore any and all updates.
Below is a usage example, where some of the example values above are implemented.
"},{"location":"hooks/update_pyproject/#expectations","title":"Expectations","text":"It is required that the root pyproject.toml exists.
A minimum Python version for the Python package should be specified in the pyproject.toml file through the requires-python key.
An active internet connection and for PyPI not to be down.
"},{"location":"hooks/update_pyproject/#options","title":"Options","text":"Any of these options can be given through the args key when defining the hook.
--root-repo-path A resolvable path to the root directory of the repository folder, where the pyproject.toml file can be found. No string . --fail-fast Fail immediately if an error occurs. Otherwise, print and ignore all non-critical errors. No flag --ignore Ignore-rules based on the ignore config option of Dependabot.It should be of the format: key=value...key=value, i.e., an ellipsis (...) separator and then equal-sign-separated key/value-pairs.Alternatively, the --ignore-separator can be set to something else to overwrite the ellipsis.The only supported keys are: dependency-name, versions, and update-types.Can be supplied multiple times per dependency-name. No string --ignore-separator Value to use instead of ellipsis (...) as a separator in --ignore key/value-pairs. No string --verbose Whether or not to print debug statements. No flag --skip-unnormalized-python-package-names Whether to skip dependencies with unnormalized Python package names. Normalization is outlined here. No flag"},{"location":"hooks/update_pyproject/#usage-example","title":"Usage example","text":"The following is an example of how an addition of the Update dependencies in pyproject.toml hook into a .pre-commit-config.yaml file may look. It is meant to be complete as is.
repos:\n - repo: https://github.com/SINTEF/ci-cd\n rev: v2.8.3\n hooks:\n - id: update-pyproject\n args:\n - --fail-fast\n - --ignore-separator=//\n - --ignore\n - dependency-name=Sphinx//versions=>=4.5.0\n - --ignore\n - dependency-name=numpy\nThis section contains all the available callable/reusable workflows:
cd_release.yml)ci_automerge_prs.yml)ci_cd_updated_default_branch.yml))ci_check_pyproject_dependencies.yml)ci_tests.yml)ci_update_dependencies.yml)For inputs specifying single or multi-line input values, the following rules apply:
If only \"single\" is mentioned, it means that the input value must be a single line (the workflow might fail if it is not).
Note
There is currently no input parameter that is explicitly single line only. Instead, one should consider the input parameter to be single line only if it is not explicitly mentioned as multi-line.
If both \"single\" and \"multi-line\" is mentioned, it means that multiple values can be specified, but they must be separated either over several, separate lines or within a single line by a space.
Here are some examples:
"},{"location":"workflows/#multi-line-input","title":"Multi-line input","text":"Accepted input styles:
# Two separate version update changes:\nversion_update_changes: |\n \"file/path,pattern,replacement string\"\n \"another/file/path,pattern,replacement string\"\n\n# A single version update change\nversion_update_changes: |\n \"file/path,pattern,replacement string\"\n\n# A single version update change, different formatting for input\nversion_update_changes: \"file/path,pattern,replacement string\"\nDisallowed input styles:
# Two separate version update changes:\nversion_update_changes: \"file/path,pattern,replacement string another/file/path,pattern,replacement string\"\nAccepted input styles:
# A single git username:\ngit_username: \"Casper Welzel Andersen\"\n\n# A single git username, different formatting for input\ngit_username: |\n \"Casper Welzel Andersen\"\nDisallowed input styles:
# Two separate git usernames:\ngit_username: |\n \"Casper Welzel Andersen\"\n \"Francesca L. Bleken\"\n\n# Two separate git usernames, different formatting for input\ngit_username: \"Casper Welzel Andersen Francesca L. Bleken\"\nWarning
It is important to note that the disallowed examples will work without fault in this case (might not always be true for other parameters). But the git username will be a single string, combining the names in succession, instead of being two separate values.
"},{"location":"workflows/#single-or-multi-line-input","title":"Single or multi-line input","text":"Accepted input styles:
# A single system dependency:\nsystem_dependencies: \"graphviz\"\n\n# A single system dependency, different formatting for input\nsystem_dependencies: |\n \"graphviz\"\n\n# Two separate system dependencies:\nsystem_dependencies: |\n \"graphviz\"\n \"Sphinx\"\n\n# Two separate system dependencies, different formatting for input\nsystem_dependencies: \"graphviz Sphinx\"\nDisallowed input styles:
# Use of custom separator:\nsystem_dependencies: \"graphviz,Sphinx\"\nImportant
The default for publish_on_pypi has changed from true to false in version 2.8.0.
To keep using the previous behaviour, set publish_on_pypi: true in the workflow file.
This change has been introduced to push for the use of PyPI's Trusted Publisher feature, which is not yet supported by reusable/callable workflows.
See the Using PyPI's Trusted Publisher section for more information on how to migrate to this feature.
File to use: cd_release.yml
There are 2 jobs in this workflow, which run in sequence.
First, an update & publish job, which updates the version in the package's root __init__.py file through an Invoke task. The newly created tag (created due to the caller workflow running on.release.types.published) will be updated accordingly, as will the publish branch (defaults to main).
Secondly, a job to update the documentation is run, however, this can be deactivated. The job expects the documentation to be setup with either the mike+MkDocs+GitHub Pages framework or the Sphinx framework.
For more information about the specific changelog inputs, see the related changelog generator actually used, specifically the list of configuration options.
Note
Concerning the changelog generator, the specific input changelog_exclude_labels defaults to a list of different labels if not supplied, hence, if supplied, one might want to include these labels alongside any extra labels. The default value is given here as a help: 'duplicate,question,invalid,wontfix'
The changelog_exclude_tags_regex is also used to remove tags in a list of tags to consider when evaluating the \"previous version\". This is specifically for adding a changelog to the GitHub release body.
If used together with the Update API Reference in Documentation, please align the relative input with the --relative option, when running the hook. See the proper section to understand why and how these options and inputs should be aligned.
PyPI has introduced a feature called Trusted Publisher which allows for a more secure way of publishing packages using OpenID Connect (OIDC). This feature is not yet supported by reusable/callable workflows, but can be used by setting up a GitHub Action workflow in your repository that calls the cd_release.yml workflow in one job, setting the publish_on_pypi input to false and the upload_distribution input to true, and then using the uploaded artifact to publish the package to PyPI in a subsequent job.
In this way you can still benefit from the cd_release.yml dynamically updated workflow, while using PyPI's Trusted Publisher feature.
Info
The artifact name is statically set to dist. If the workflow is run multiple times, the artifact will be overwritten. Retention time for the artifact is kept at the GitHub default (currently 90 days).
Important
The id-token:write permission is required by the PyPI upload action for Trusted Publishers.
The following is an example of how a workflow may look that calls CD - Release and uses the uploaded built distribution artifact to publish the package to PyPI. Note, the non-default dists directory is chosen for the built distribution, and the artifact is downloaded to the my-dists directory.
name: CD - Publish\n\non:\n release:\n types:\n - published\n\njobs:\n build:\n name: Build distribution & publish documentation\n if: github.repository == 'SINTEF/my-python-package' && startsWith(github.ref, 'refs/tags/v')\n uses: SINTEF/ci-cd/.github/workflows/cd_release.yml@v2.8.3\n with:\n # General\n git_username: \"Casper Welzel Andersen\"\n git_email: \"CasperWA@github.com\"\n release_branch: stable\n\n # Build distribution\n python_package: true\n package_dirs: my_python_package\n install_extras: \"[dev,build]\"\n build_libs: build\n build_cmd: \"python -m build -o dists\"\n build_dir: dists\n publish_on_pypi: false\n upload_distribution: true\n\n # Publish documentation\n update_docs: true\n doc_extras: \"[docs]\"\n docs_framework: mkdocs\n\n secrets:\n PAT: ${{ secrets.PAT }}\n\n publish:\n name: Publish to PyPI\n needs: build\n runs-on: ubuntu-latest\n\n # Using environments is recommended by PyPI when using Trusted Publishers\n environment: release\n\n # The id-token:write permission is required by the PyPI upload action for\n # Trusted Publishers\n permissions:\n id-token: write\n\n steps:\n - name: Download distribution\n uses: actions/download-artifact@v4\n with:\n name: dist # The artifact will always be called 'dist'\n path: my-dists\n\n - name: Publish to PyPI\n uses: pypa/gh-action-pypi-publish@release/v1\n with:\n # The path to the distribution to upload\n packages-dir: my-dists/\nThe content of repository files can be updated to use the new version where necessary. This is done through the version_update_changes (and version_update_changes_separator) inputs.
To see an example of how to use the version_update_changes (and version_update_changes_separator) see for example the workflow used by the SINTEF/ci-cd repository calling the CD Release workflow.
Some notes to consider and respect when using version_update_changes are:
version_update_changes_separator applies to all lines given in version_update_changes, meaning it should be a character, or series of characters, which will not be part of the actual content.Specifically, concerning the 'raw' Python string 'pattern' the following applies:
\"). This is done by prefixing it with a backslash (\\): \\\".`).re library documentation for more information.Concerning the 'replacement string' part, the package_dirs input and full semantic version can be substituted in dynamically by wrapping either package_dir or version in curly braces ({}). Indeed, for the version, one can specify sub-parts of the version to use, e.g., if one desires to only use the major version, this can be done by using the major attribute: {version.major}. The full list of version attributes are: major, minor, patch, pre_release, and build. More can be used, e.g., to only insert the major.minor version: {version.major}.{version.minor}.
For the 'file path' part, package_dir wrapped in curly braces ({}) will also be substituted at run time with each line from the possibly multi-line package_dirs input. E.g., {package_dir}/__init__.py will become ci_cd/__init__.py if the package_dirs input was 'ci_cd'.
This workflow should only be used for releasing a single modern Python package.
The repository contains the following:
__init__.py file with __version__ defined.v, e.g., v1.0.0.The following inputs are general inputs for the workflow as a whole.
Name Description Required Default Typegit_username A git username (used to set the 'user.name' config option). Yes string git_email A git user's email address (used to set the 'user.email' config option). Yes string release_branch The branch name to release/publish from. Yes main string runner The runner to use for the workflow. Note, the callable workflow expects a Linux/Unix system.. No ubuntu-latest string install_extras Any extras to install from the local repository through 'pip'. Must be encapsulated in square parentheses ([]) and be separated by commas (,) without any spaces.Example: '[dev,release]'. No Empty string string relative Whether or not to use install the local Python package(s) as an editable. No false boolean test Whether to use the TestPyPI repository index instead of PyPI as well as output debug statements in both workflow jobs. No false boolean pip_index_url A URL to a PyPI repository index. No https://pypi.org/simple/ string pip_extra_index_urls A space-delimited string of URLs to additional PyPI repository indices. No Empty string string Inputs related to updating the version, building and releasing the Python package to PyPI.
Name Description Required Default Typepublish_on_pypi Whether or not to publish on PyPI.Note: This is only relevant if 'python_package' is 'true', which is the default.Important: The default has changed from true to false to push for the use of PyPI's Trusted Publisher feature.See the Using PyPI's Trusted Publisher section for more information on how to migrate to this feature. Yes (will be non-required in v2.9) false boolean python_package Whether or not this is a Python package, where the version should be updated in the 'package_dir'/__init__.py for the possibly several 'package_dir' lines given in the package_dirs input and a build and release to PyPI should be performed. No true boolean python_version_build The Python version to use for the workflow when building the package. No 3.9 string package_dirs A multi-line string of paths to Python package directories relative to the repository directory to have its __version__ value updated.Example: 'src/my_package'.Important: This is required if 'python_package' is 'true', which is the default.See also Single vs multi-line input. Yes (if 'python_package' is 'true') string version_update_changes A multi-line string of changes to be implemented in the repository files upon updating the version. The string should be made up of three parts: 'file path', 'pattern', and 'replacement string'. These are separated by the 'version_update_changes_separator' value.The 'file path' must always either be relative to the repository root directory or absolute.The 'pattern' should be given as a 'raw' Python string.See also Single vs multi-line input. No Empty string string version_update_changes_separator The separator to use for 'version_update_changes' when splitting the three parts of each string. No , string build_libs A space-separated list of packages to install via PyPI (pip install). No Empty string string build_cmd The package build command, e.g., 'flit build' or 'python -m build'. No python -m build --outdir dist . string build_dir The directory where the built distribution is located. This should reflect the directory used in the build command or by default by the build library. No dist string tag_message_file Relative path to a release tag message file from the root of the repository.Example: '.github/utils/release_tag_msg.txt'. No Empty string string changelog_exclude_tags_regex A regular expression matching any tags that should be excluded from the CHANGELOG.md. No Empty string string changelog_exclude_labels Comma-separated list of labels to exclude from the CHANGELOG.md. No Empty string string upload_distribution Whether or not to upload the built distribution as an artifact.Note: This is only relevant if 'python_package' is 'true', which is the default. No true boolean Inputs related to building and releasing the documentation in general.
Name Description Required Default Typeupdate_docs Whether or not to also run the 'docs' workflow job. No false boolean python_version_docs The Python version to use for the workflow when building the documentation. No 3.9 string doc_extras Any extras to install from the local repository through 'pip'. Must be encapsulated in square parentheses ([]) and be separated by commas (,) without any spaces.Note, if this is empty, 'install_extras' will be used as a fallback.Example: '[docs]'. No Empty string string docs_framework The documentation framework to use. This can only be either 'mkdocs' or 'sphinx'. No mkdocs string system_dependencies A single (space-separated) or multi-line string of Ubuntu APT packages to install prior to building the documentation.See also Single vs multi-line input. No Empty string string Inputs related only to the MkDocs framework.
Name Description Required Default Typemkdocs_update_latest Whether or not to update the 'latest' alias to point to release_branch. No true boolean Finally, inputs related only to the Sphinx framework.
Name Description Required Default Typesphinx-build_options Single (space-separated) or multi-line string of command-line options to use when calling sphinx-build.See also Single vs multi-line input. No Empty string string docs_folder The path to the root documentation folder relative to the repository root. No docs string build_target_folder The path to the target folder for the documentation build relative to the repository root. No site string"},{"location":"workflows/cd_release/#secrets","title":"Secrets","text":"Name Description Required PyPI_token A PyPI token for publishing the built package to PyPI.Important: This is required if both 'python_package' and 'publish_on_pypi' are 'true'. Both are 'true' by default. Yes (if 'python_package' and 'publish_on_pypi' are 'true') PAT A personal access token (PAT) with rights to update the release_branch. This will fallback on GITHUB_TOKEN. No"},{"location":"workflows/cd_release/#usage-example","title":"Usage example","text":"The following is an example of how a workflow may look that calls CD - Release. It is meant to be complete as is.
name: CD - Publish\n\non:\n release:\n types:\n - published\n\njobs:\n publish:\n name: Publish package and documentation\n uses: SINTEF/ci-cd/.github/workflows/cd_release.yml@v2.8.3\n if: github.repository == 'SINTEF/my-python-package' && startsWith(github.ref, 'refs/tags/v')\n with:\n # General\n git_username: \"Casper Welzel Andersen\"\n git_email: \"CasperWA@github.com\"\n release_branch: stable\n\n # Publish distribution\n package_dirs: my_python_package\n install_extras: \"[dev,build]\"\n build_cmd: \"pip install flit && flit build\"\n tag_message_file: \".github/utils/release_tag_msg.txt\"\n changelog_exclude_labels: \"skip_changelog,duplicate\"\n publish_on_pypi: true\n\n # Publish documentation\n update_docs: true\n doc_extras: \"[docs]\"\n secrets:\n PyPI_token: ${{ secrets.PYPI_TOKEN }}\n PAT: ${{ secrets.PAT }}\nFile to use: ci_automerge_prs.yml
Activate auto-merging for a PR.
It is possible to introduce changes to the PR head branch prior to activating the auto-merging, if so desired. This is done by setting perform_changes to 'true' and setting the other inputs accordingly, as they are now required. See Inputs below for a full overview of the available inputs.
The changes input can be both a path to a bash file that should be run, or a multi-line string of bash commands to run. Afterwards any and all changes in the repository will be committed and pushed to the PR head branch.
The motivation for being able to run changes prior to auto-merging, is to update or affect the repository files according to the specific PR being auto-merged. Usually auto-merging is activated for dependabot branches, i.e., when a dependency/requirement is updated. Hence, the changes could include updating this dependency in documentation files or similar, where it will not be updated otherwise.
PR branch name
The generated branch for the PR will be named ci/update-pyproject.
The PAT secret must represent a user with the rights to activate auto-merging.
This workflow can only be called if the triggering event from the caller workflow is pull_request_target.
runner The runner to use for the workflow. Note, the callable workflow expects a Linux/Unix system.. No ubuntu-latest string perform_changes Whether or not to perform and commit changes to the PR branch prior to activating auto-merge. No boolean git_username A git username (used to set the 'user.name' config option).Required if perform_changes is 'true'. No string git_email A git user's email address (used to set the 'user.email' config option).Required if perform_changes is 'true'. No string changes A file to run in the local repository (relative path from the root of the repository) or a multi-line string of bash commands to run.Required if perform_changes is 'true'.See also Single vs multi-line input. No string"},{"location":"workflows/ci_automerge_prs/#secrets","title":"Secrets","text":"Name Description Required PAT A personal access token (PAT) with rights to activate auto-merging. This will fallback on GITHUB_TOKEN. No"},{"location":"workflows/ci_automerge_prs/#usage-example","title":"Usage example","text":"The following is an example of how a workflow may look that calls CI - Activate auto-merging for PRs. It is meant to be complete as is.
name: CI - Activate auto-merging for Dependabot PRs\n\non:\n pull_request_target:\n branches:\n - ci/dependency-updates\n\njobs:\n update-dependency-branch:\n name: Call external workflow\n uses: SINTEF/ci-cd/.github/workflows/ci_automerge_prs.yml@v2.8.3\n if: github.repository_owner == 'SINTEF' && ( ( startsWith(github.event.pull_request.head.ref, 'dependabot/') && github.actor == 'dependabot[bot]' ) || ( github.event.pull_request.head.ref == 'ci/update-pyproject' && github.actor == 'CasperWA' ) )\n secrets:\n PAT: ${{ secrets.RELEASE_PAT }}\nA couple of usage examples when adding changes:
Here, referencing a bash script file for the changes.
name: CI - Activate auto-merging for Dependabot PRs\n\non:\n pull_request_target:\n branches:\n - ci/dependency-updates\n\njobs:\n update-dependency-branch:\n name: Call external workflow\n uses: SINTEF/ci-cd/.github/workflows/ci_automerge_prs.yml@v2.8.3\n if: github.repository_owner == 'SINTEF' && ( ( startsWith(github.event.pull_request.head.ref, 'dependabot/') && github.actor == 'dependabot[bot]' ) || ( github.event.pull_request.head.ref == 'ci/update-pyproject' && github.actor == 'CasperWA' ) )\n with:\n perform_changes: true\n git_username: \"Casper Welzel Andersen\"\n git_email: \"CasperWA@github.com\"\n changes: \".ci/pre_automerge.sh\"\n secrets:\n PAT: ${{ secrets.RELEASE_PAT }}\nHere, writing out the changes explicitly in the job.
name: CI - Activate auto-merging for Dependabot PRs\n\non:\n pull_request_target:\n branches:\n - ci/dependency-updates\n\njobs:\n update-dependency-branch:\n name: Call external workflow\n uses: SINTEF/ci-cd/.github/workflows/ci_automerge_prs.yml@v2.8.3\n if: github.repository_owner == 'SINTEF' && ( ( startsWith(github.event.pull_request.head.ref, 'dependabot/') && github.actor == 'dependabot[bot]' ) || ( github.event.pull_request.head.ref == 'ci/update-pyproject' && github.actor == 'CasperWA' ) )\n with:\n perform_changes: true\n git_username: \"Casper Welzel Andersen\"\n git_email: \"CasperWA@github.com\"\n changes: |\n PYTHON=\"$(python --version || :)\"\n if [ -z \"${PYTHON}\" ]; then\n echo \"Python not detected on the system.\"\n exit 1\n fi\n\n PIP=\"$(python -m pip --version || :)\"\n if [ -z \"${PIP}\" ]; then\n echo \"pip not detected to be installed for ${PYTHON}.\"\n exit 1\n fi\n\n echo \"Python: ${PYTHON}\"\n echo \"pip: ${PIP}\"\n\n python -m pip install -U pip\n pip install -U setuptools wheel\n pip install pre-commit\n\n pre-commit autoupdate\n pre-commit run --all-files || :\n secrets:\n PAT: ${{ secrets.RELEASE_PAT }}\nFile to use: ci_cd_updated_default_branch.yml
Keep your permanent_dependencies_branch branch up-to-date with changes in your main development branch, i.e., the default_repo_branch.
Furthermore, this workflow can optionally update the latest mike+MkDocs+GitHub Pages-framework documentation release alias, which represents the default_repo_branch. The workflow also alternatively supports the Sphinx framework.
Warning
If a PAT is not passed through for the PAT secret and GITHUB_TOKEN is used, beware that any other CI/CD jobs that run for, e.g., pull request events, may not run since GITHUB_TOKEN-generated PRs are designed to not start more workflows to avoid escalation. Hence, if it is important to run CI/CD workflows for pull requests, consider passing a PAT as a secret to this workflow represented by the PAT secret.
Important
If this is to be used together with the CI - Update dependencies PR workflow, the pr_body_file supplied to that workflow (if any) should match the update_depednencies_pr_body_file input in this workflow and be immutable within the first 8 lines, i.e., no check boxes or similar in the first 8 lines. Indeed, it is recommended to not supply pr_body_file to the CI - Update dependencies PR workflow as well as to not supply the update_dependencies_pr_body_file in this workflow in this case.
Note
Concerning the changelog generator, the specific input changelog_exclude_labels defaults to a list of different labels if not supplied, hence, if supplied, one might want to include these labels alongside any extra labels. The default value is given here as a help: 'duplicate,question,invalid,wontfix'
If used together with the Update API Reference in Documentation, please align the relative input with the --relative option, when running the hook. See the proper section to understand why and how these options and inputs should be aligned.
The repository contains the following:
package_dirs input.docs directory.README.md file must exist and desired to be used as the documentation's landing page if the update_docs_landing_page is set to true, which is the default.The following inputs are general inputs for the workflow as a whole.
Name Description Required Default Typegit_username A git username (used to set the 'user.name' config option). Yes string git_email A git user's email address (used to set the 'user.email' config option). Yes string runner The runner to use for the workflow. Note, the callable workflow expects a Linux/Unix system.. No ubuntu-latest string default_repo_branch The branch name of the repository's default branch. More specifically, the branch the PR should target. No main string test Whether to do a \"dry run\", i.e., run the workflow, but avoid pushing to 'permanent_dependencies_branch' branch and deploying documentation (if 'update_docs' is 'true'). No false boolean pip_index_url A URL to a PyPI repository index. No https://pypi.org/simple/ string pip_extra_index_urls A space-delimited string of URLs to additional PyPI repository indices. No Empty string string Inputs related to updating the permanent dependencies branch.
Name Description Required Default Typeupdate_dependencies_branch Whether or not to update the permanent dependencies branch. No true boolean permanent_dependencies_branch The branch name for the permanent dependency updates branch. No ci/dependency-updates string update_dependencies_pr_body_file Relative path to a PR body file from the root of the repository, which is used in the 'CI - Update dependencies PR' workflow, if used.Example: '.github/utils/pr_body_update_deps.txt'. No Empty string string Inputs related to building and releasing the documentation.
Name Description Required Default Typeupdate_docs Whether or not to also run the 'docs' workflow job. No false boolean update_python_api_ref Whether or not to update the Python API documentation reference.Note: If this is 'true', 'package_dirs' is required. No true boolean package_dirs A multi-line string of paths to Python package directories relative to the repository directory to be considered for creating the Python API reference documentation.Example: 'src/my_package'.Important: This is required if 'update_docs' and 'update_python_api_ref' are 'true'.See also Single vs multi-line input. Yes (if 'update_docs' and 'update_python_api_ref' are 'true') string update_docs_landing_page Whether or not to update the documentation landing page. The landing page will be based on the root README.md file. No true boolean python_version The Python version to use for the workflow.Note: This is only relevant if update_pre-commit is true. No 3.9 string doc_extras Any extras to install from the local repository through 'pip'. Must be encapsulated in square parentheses ([]) and be separated by commas (,) without any spaces.Example: '[docs]'. No Empty string string relative Whether or not to use install the local Python package(s) as an editable. No false boolean exclude_dirs A multi-line string of directories to exclude in the Python API reference documentation. Note, only directory names, not paths, may be included. Note, all folders and their contents with these names will be excluded. Defaults to '__pycache__'.Important: When a user value is set, the preset value is overwritten - hence '__pycache__' should be included in the user value if one wants to exclude these directories.See also Single vs multi-line input. No __pycache__ string exclude_files A multi-line string of files to exclude in the Python API reference documentation. Note, only full file names, not paths, may be included, i.e., filename + file extension. Note, all files with these names will be excluded. Defaults to '__init__.py'.Important: When a user value is set, the preset value is overwritten - hence '__init__.py' should be included in the user value if one wants to exclude these files.See also Single vs multi-line input. No __init__.py string full_docs_dirs A multi-line string of directories in which to include everything - even those without documentation strings. This may be useful for a module full of data models or to ensure all class attributes are listed.See also Single vs multi-line input. No Empty string string full_docs_files A multi-line string of relative paths to files in which to include everything - even those without documentation strings. This may be useful for a file full of data models or to ensure all class attributes are listed.See also Single vs multi-line input. No Empty string string special_file_api_ref_options A multi-line string of combinations of a relative path to a Python file and a fully formed mkdocstrings option that should be added to the generated MarkDown file for the Python API reference documentation.Example: my_module/py_file.py,show_bases:false.Encapsulate the value in double quotation marks (\") if including spaces ( ).Important: If multiple package_dirs are supplied, the relative path MUST include/start with the appropriate 'package_dir' value, e.g., \"my_package/my_module/py_file.py,show_bases: false\".See also Single vs multi-line input. No Empty string string landing_page_replacements A multi-line string of replacements (mappings) to be performed on README.md when creating the documentation's landing page (index.md). This list always includes replacing 'docs/' with an empty string to correct relative links, i.e., this cannot be overwritten. By default '(LICENSE)' is replaced by '(LICENSE.md)'.See also Single vs multi-line input. No (LICENSE),(LICENSE.md) string landing_page_replacement_separator String to separate a mapping's 'old' to 'new' parts. Defaults to a comma (,). No , string changelog_exclude_tags_regex A regular expression matching any tags that should be excluded from the CHANGELOG.md. No Empty string string changelog_exclude_labels Comma-separated list of labels to exclude from the CHANGELOG.md. No Empty string string docs_framework The documentation framework to use. This can only be either 'mkdocs' or 'sphinx'. No mkdocs string system_dependencies A single (space-separated) or multi-line string of Ubuntu APT packages to install prior to building the documentation.See also Single vs multi-line input. No Empty string string Finally, inputs related only to the Sphinx framework when building and releasing the documentation.
Name Description Required Default Typesphinx-build_options Single (space-separated) or multi-line string of command-line options to use when calling sphinx-build.See also Single vs multi-line input. No Empty string string docs_folder The path to the root documentation folder relative to the repository root. No docs string build_target_folder The path to the target folder for the documentation build relative to the repository root. No site string"},{"location":"workflows/ci_cd_updated_default_branch/#secrets","title":"Secrets","text":"Name Description Required PAT A personal access token (PAT) with rights to update the permanent_dependencies_branch. This will fallback on GITHUB_TOKEN. No"},{"location":"workflows/ci_cd_updated_default_branch/#usage-example","title":"Usage example","text":"The following is an example of how a workflow may look that calls CI/CD - New updates to default branch. It is meant to be complete as is.
name: CI - Activate auto-merging for Dependabot PRs\n\non:\n push:\n branches:\n - stable\n\njobs:\n updates-to-stable:\n name: Call external workflow\n uses: SINTEF/ci-cd/.github/workflows/ci_cd_updated_default_branch.yml@v2.8.3\n if: github.repository_owner == 'SINTEF'\n with:\n git_username: \"Casper Welzel Andersen\"\n git_email: \"CasperWA@github.com\"\n default_repo_branch: stable\n permanent_dependencies_branch: \"ci/dependency-updates\"\n update_docs: true\n package_dirs: |\n my_python_package\n my_other_python_package\n doc_extras: \"[docs]\"\n exclude_files: __init__.py,config.py\n full_docs_dirs: models\n landing_page_replacements: \"(LICENSE);(LICENSE.md)|(tools);(../tools)\"\n landing_page_replacements_mapping_separator: \";\"\n secrets:\n PAT: ${{ secrets.PAT }}\nFile to use: ci_check_pyproject_dependencies.yml
This workflow runs an Invoke task to check dependencies in a pyproject.toml file.
The reason for having this workflow and not using Dependabot is because it seems to not function properly with this use case.
Warning
If a PAT is not passed through for the PAT secret and GITHUB_TOKEN is used, beware that any other CI/CD jobs that run for, e.g., pull request events, may not run since GITHUB_TOKEN-generated PRs are designed to not start more workflows to avoid escalation. Hence, if it is important to run CI/CD workflows for pull requests, consider passing a PAT as a secret to this workflow represented by the PAT secret.
Info
The generated PR will be created from a new branch named ci/update-pyproject. If you wish to change this value, see the branch_name_extension input option.
To ignore or configure how specific dependencies should be updated, the ignore input option can be utilized. This is done by specifying a line per dependency that contains ellipsis-separated (...) key/value-pairs of:
dependency-name Ignore updates for dependencies with matching names, optionally using * to match zero or more characters. versions Ignore specific versions or ranges of versions. Examples: ~=1.0.5, >= 1.0.5,<2, >=0.1.1. update-types Ignore types of updates, such as SemVer major, minor, patch updates on version updates (for example: version-update:semver-patch will ignore patch updates). This can be combined with dependency-name=* to ignore particular update-types for all dependencies. Supported update-types values
Currently, only version-update:semver-major, version-update:semver-minor, and version-update:semver-patch are supported options for update-types.
The ignore option is essentially similar to the ignore option of Dependabot. If versions and update-types are used together, they will both be respected jointly.
Here is an example of different lines given as value for the ignore option that accomplishes different things:
# ...\njobs:\n check-dependencies:\n uses: SINTEF/ci-cd/.github/workflows/ci_check_pyproject_dependencies.yml@v2.8.3\n with:\n # ...\n # For Sphinx, ignore all updates for/from version 4.5.0 and up / keep the minimum version for Sphinx at 4.5.0.\n # For pydantic, ignore all patch updates\n # For numpy, ignore any and all updates\n ignore: |\n dependency-name=Sphinx...versions=>=4.5.0\n dependency-name=pydantic...update-types=version-update:semver-patch\n dependency-name=numpy\n# ...\nThe repository contains the following:
pyproject.toml file with the Python package's dependencies.git_username A git username (used to set the 'user.name' config option). Yes string git_email A git user's email address (used to set the 'user.email' config option). Yes string runner The runner to use for the workflow. Note, the callable workflow expects a Linux/Unix system.. No ubuntu-latest string target_branch The branch name for the target of the opened PR.Note: If a value is not given for this nor permanent_dependencies_branch, the default value for permanent_dependencies_branch will be used until v2.6.0, whereafter providing an explicit value for target_branch is required. No Empty string string permanent_dependencies_branch DEPRECATED - Will be removed in v2.6.0. Use target_branch instead.The branch name for the permanent dependency updates branch. No ci/dependency-updates string python_version The Python version to use for the workflow. No 3.9 string install_extras Any extras to install from the local repository through 'pip'. Must be encapsulated in square parentheses ([]) and be separated by commas (,) without any spaces.Example: '[dev,release]'. No Empty string string pip_index_url A URL to a PyPI repository index. No https://pypi.org/simple/ string pip_extra_index_urls A space-delimited string of URLs to additional PyPI repository indices. No Empty string string pr_body_file Relative path to PR body file from the root of the repository.Example: '.github/utils/pr_body_deps_check.txt'. No Empty string string fail_fast Whether the task to update dependencies should fail if any error occurs. No false boolean pr_labels A comma separated list of strings of GitHub labels to use for the created PR. No Empty string string ignore Create ignore conditions for certain dependencies. A multi-line string of ignore rules, where each line is an ellipsis-separated (...) string of key/value-pairs. One line per dependency. This option is similar to the ignore option of Dependabot.See also Single vs multi-line input. No Empty string string branch_name_extension A string to append to the branch name of the created PR. Example: '-my-branch'. It will be appended after a forward slash, so the final branch name will be ci/update-pyproject/-my-branch. No Empty string string debug Whether to run the workflow in debug mode, printing extra debug information. No false boolean skip_unnormalized_python_package_names Whether to skip dependencies with unnormalized Python package names. Normalization is outlined here. No false boolean"},{"location":"workflows/ci_check_pyproject_dependencies/#secrets","title":"Secrets","text":"Name Description Required PAT A personal access token (PAT) with rights to create PRs. This will fallback on GITHUB_TOKEN. No"},{"location":"workflows/ci_check_pyproject_dependencies/#usage-example","title":"Usage example","text":"The following is an example of how a workflow may look that calls CI - Check pyproject.toml dependencies. It is meant to be complete as is.
name: CI - Check dependencies\n\non:\n schedule:\n - cron: \"30 5 * * 1\"\n workflow_dispatch:\n\njobs:\n check-dependencies:\n name: Call external workflow\n uses: SINTEF/ci-cd/.github/workflows/ci_check_pyproject_dependencies.yml@v2.8.3\n if: github.repository_owner == 'SINTEF'\n with:\n git_username: \"Casper Welzel Andersen\"\n git_email: \"CasperWA@github.com\"\n target_branch: \"ci/dependency-updates\"\n python_version: \"3.9\"\n install_extras: \"[dev]\"\n pr_labels: \"CI/CD\"\n secrets:\n PAT: ${{ secrets.PAT }}\nFile to use: ci_tests.yml
A basic set of CI tests.
Several different basic test jobs are available in this workflow. By default, they will all run and should be actively \"turned off\".
"},{"location":"workflows/ci_tests/#ci-jobs","title":"CI jobs","text":"The following sections summarizes each job and the individual inputs necessary for it to function or to adjust how it runs. Note, a full list of possible inputs and secrets will be given in a separate table at the end of this page.
"},{"location":"workflows/ci_tests/#globalgeneral-inputs","title":"Global/General inputs","text":"These inputs are general and apply to all jobs in this workflow.
Name Description Required Default Typerunner The runner to use for the workflow. Note, the callable workflow expects a Linux/Unix system.. No ubuntu-latest string install_extras Any extras to install from the local repository through 'pip'. Must be encapsulated in square parentheses ([]) and be separated by commas (,) without any spaces.Example: '[dev,pre-commit]'. No Empty string string"},{"location":"workflows/ci_tests/#run-pre-commit","title":"Run pre-commit","text":"Run the pre-commit tool for all files in the repository according to the repository's configuration file.
pre-commit should be setup for the repository. For more information about pre-commit, please see the tool's website: pre-commit.com.
This job should not be run if the repository does not implement pre-commit.
run_pre-commit Run the pre-commit test job. No true boolean python_version_pre-commit The Python version to use for the pre-commit test job. No 3.9 string pip_index_url_pre-commit A URL to a PyPI repository index. No https://pypi.org/simple/ string pip_extra_index_urls_pre-commit A space-delimited string of URLs to additional PyPI repository indices. No Empty string string skip_pre-commit_hooks A comma-separated list of pre-commit hook IDs to skip when running pre-commit after updating hooks. No Empty string string"},{"location":"workflows/ci_tests/#run-pylint-safety","title":"Run pylint & safety","text":"Run the pylint and/or safety tools.
The pylint tool can be run in different ways. Either it is run once and the pylint_targets is a required input, while pylint_options is a single- or multi-line optional input. Or pylint_runs is used, a single- or multi-line input, to explicitly write out all pylint options and target(s) one line at a time. For each line in pylint_runs, pylint will be executed.
Using pylint_runs is useful if you have a section of your code, which should be run with a custom set of options, otherwise it is recommended to instead simply use the pylint_targets and optionally also pylint_options inputs.
The safety tool checks all installed Python packages, hence the install_extras input should be given as to install all possible dependencies.
There are no expectations or pre-requisites. pylint and safety can be run without a pre-configuration.
run_pylint Run the pylint test job. No true boolean run_safety Run the safety test job. No true boolean python_version_pylint_safety The Python version to use for the pylint and safety test jobs. No 3.9 string pip_index_url_pylint_safety A URL to a PyPI repository index. No https://pypi.org/simple/ string pip_extra_index_urls_pylint_safety A space-delimited string of URLs to additional PyPI repository indices. No Empty string string pylint_targets Space-separated string of pylint file and folder targets.Note: This is only valid if pylint_runs is not defined. Yes, if pylint_runs is not defined Empty string string pylint_options Single (space-separated) or multi-line string of pylint command line options.Note: This is only valid if pylint_runs is not defined.See also Single vs multi-line input. No Empty string string pylint_runs Multi-line string with each line representing a separate pylint run/execution. This should include all desired options and targets.Important: The inputs pylint_options and pylint_targets will be ignored if this is defined.See also Single vs multi-line input. No Empty string string safety_options Single (space-separated) or multi-line string of safety command line options.See also Single vs multi-line input. No Empty string string"},{"location":"workflows/ci_tests/#build-distribution-package","title":"Build distribution package","text":"Test building the Python package.
This job is equivalent to building the package in the CD - Release workflow, but will not publish anything.
"},{"location":"workflows/ci_tests/#expectations_2","title":"Expectations","text":"The repository should be a \"buildable\" Python package.
"},{"location":"workflows/ci_tests/#inputs_2","title":"Inputs","text":"Name Description Required Default Typerun_build_package Run the build package test job. No true boolean python_version_package The Python version to use for the build package test job. No 3.9 string pip_index_url_package A URL to a PyPI repository index. No https://pypi.org/simple/ string pip_extra_index_urls_package A space-delimited string of URLs to additional PyPI repository indices. No Empty string string build_libs A space-separated list of packages to install via PyPI (pip install). No Empty string string build_cmd The package build command, e.g., 'flit build' or 'python -m build' (default). No python -m build string"},{"location":"workflows/ci_tests/#build-documentation","title":"Build Documentation","text":"Test building the documentation.
Two frameworks are supported: MkDocs and Sphinx.
By default the MkDocs framework is used. To use the Sphinx framework set the input use_sphinx to true. The input use_mkdocs can also explicitly be set to true for more transparent documentation in your workflow.
Note, if both use_sphinx and use_mkdocs are false (as is the default value for both), the workflow will fallback to using MkDocs, i.e., it is equivalent to setting use_mkdocs to true.
For MkDocs users
If using mike, note that this will not be tested, as this would be equivalent to testing mike itself and whether it can build a MkDocs documentation, which should never be part of a repository that uses these tools.
If used together with the Update API Reference in Documentation, please align the relative input with the --relative option, when running the hook. See the proper section to understand why and how these options and inputs should be aligned.
Is is expected that documentation exists, which is using either the MkDocs framework or the Sphinx framework. For MkDocs, this requires at minimum a mkdocs.yml configuration file. For Sphinx, it requires at minimum the files created from running sphinx-quickstart.
General inputs for building the documentation:
Name Description Required Default Typerun_build_docs Run the build package test job. No true boolean python_version_docs The Python version to use for the build documentation test job. No 3.9 string pip_index_url_docs A URL to a PyPI repository index. No https://pypi.org/simple/ string pip_extra_index_urls_docs A space-delimited string of URLs to additional PyPI repository indices. No Empty string string relative Whether or not to use the locally installed Python package(s), and install it as an editable. No false boolean system_dependencies A single (space-separated) or multi-line string of Ubuntu APT packages to install prior to building the documentation.See also Single vs multi-line input. No Empty string string warnings_as_errors Build the documentation in 'strict' mode, treating warnings as errors.Important: If this is set to false, beware that the documentation may not be rendered or built as one may have intended.Default: true. No true boolean use_mkdocs Whether or not to build the documentation using the MkDocs framework. Mutually exclusive with use_sphinx. No false boolean use_sphinx Whether or not to build the documentation using the Sphinx framework. Mutually exclusive with use_mkdocs. No false boolean MkDocs-specific inputs:
Name Description Required Default Typeupdate_python_api_ref Whether or not to update the Python API documentation reference.Note: If this is true, package_dirs is required. No true boolean update_docs_landing_page Whether or not to update the documentation landing page. The landing page will be based on the root README.md file. No true boolean package_dirs A multi-line string of path to Python package directories relative to the repository directory to be considered for creating the Python API reference documentation.Example: 'src/my_package'.See also Single vs multi-line input. Yes, if update_python_api_ref is true (default) Empty string string exclude_dirs A multi-line string of directories to exclude in the Python API reference documentation. Note, only directory names, not paths, may be included. Note, all folders and their contents with these names will be excluded. Defaults to '__pycache__'.Important: When a user value is set, the preset value is overwritten - hence '__pycache__' should be included in the user value if one wants to exclude these directories.See also Single vs multi-line input. No __pycache__ string exclude_files A multi-line string of files to exclude in the Python API reference documentation. Note, only full file names, not paths, may be included, i.e., filename + file extension. Note, all files with these names will be excluded. Defaults to '__init__.py'.Important: When a user value is set, the preset value is overwritten - hence '__init__.py' should be included in the user value if one wants to exclude these files.See also Single vs multi-line input. No __init__.py string full_docs_dirs A multi-line string of directories in which to include everything - even those without documentation strings. This may be useful for a module full of data models or to ensure all class attributes are listed.See also Single vs multi-line input. No Empty string string full_docs_files A multi-line string of relative paths to files in which to include everything - even those without documentation strings. This may be useful for a file full of data models or to ensure all class attributes are listed.See also Single vs multi-line input. No Empty string string special_file_api_ref_options A multi-line string of combinations of a relative path to a Python file and a fully formed mkdocstrings option that should be added to the generated MarkDown file for the Python API reference documentation.Example: my_module/py_file.py,show_bases:false.Encapsulate the value in double quotation marks (\") if including spaces ( ).Important: If multiple package_dirs are supplied, the relative path MUST include/start with the appropriate 'package_dir' value, e.g., \"my_package/my_module/py_file.py,show_bases: false\".See also Single vs multi-line input. No Empty string string landing_page_replacements A multi-line string of replacements (mappings) to be performed on README.md when creating the documentation's landing page (index.md). This list always includes replacing 'docs/' with an empty string to correct relative links, i.e., this cannot be overwritten. By default '(LICENSE)' is replaced by '(LICENSE.md)'.See also Single vs multi-line input. No (LICENSE),(LICENSE.md) string landing_page_replacement_separator String to separate a mapping's 'old' to 'new' parts. Defaults to a comma (,). No , string debug Whether to do print extra debug statements. No false boolean Sphinx-specific inputs:
Name Description Required Default Typesphinx-build_options Single (space-separated) or multi-line string of command-line options to use when calling sphinx-build.Note: The -W option will be added if warnings_as_errors is true (default).See also Single vs multi-line input. No Empty string string docs_folder The path to the root documentation folder relative to the repository root. No docs string build_target_folder The path to the target folder for the documentation build relative to the repository root. No site string"},{"location":"workflows/ci_tests/#usage-example","title":"Usage example","text":"The following is an example of how a workflow may look that calls CI - Tests. It is meant to be complete as is.
name: CI - Tests\n\non:\n pull_request:\n pull:\n branches:\n - 'main'\n\njobs:\n tests:\n name: Run basic tests\n uses: SINTEF/ci-cd/.github/workflows/ci_tests.yml@v2.8.3\n with:\n python_version_pylint_safety: \"3.8\"\n python_version_docs: \"3.7\"\n install_extras: \"[dev,docs]\"\n skip_pre-commit_hooks: pylint\n pylint_options: --rcfile=pyproject.toml\n pylint_targets: my_python_package\n build_libs: flit\n build_cmd: flit build\n update_python_api_ref: false\n update_docs_landing_page: false\nHere is another example using pylint_runs instead of pylint_targets and pylint_options.
name: CI - Tests\n\non:\n pull_request:\n pull:\n branches:\n - 'main'\n\njobs:\n tests:\n name: Run basic tests\n uses: SINTEF/ci-cd/.github/workflows/ci_tests.yml@v2.8.3\n with:\n python_version_pylint_safety: \"3.8\"\n python_version_docs: \"3.7\"\n install_extras: \"[dev,docs]\"\n skip_pre-commit_hooks: pylint\n pylint_runs: |\n --rcfile=pyproject.toml --ignore-paths=tests/ my_python_package\n --rcfile=pyproject.toml --disable=import-outside-toplevel,redefined-outer-name tests\n build_libs: flit\n build_cmd: flit build\n update_python_api_ref: false\n update_docs_landing_page: false\nHere follows the full list of inputs available for this workflow. However, it is recommended to instead refer to the job-specific tables of inputs when considering which inputs to provide.
See also General information.
Name Description Required Default Typerunner The runner to use for the workflow. Note, the callable workflow expects a Linux/Unix system.. No ubuntu-latest string install_extras Any extras to install from the local repository through 'pip'. Must be encapsulated in square parentheses ([]) and be separated by commas (,) without any spaces.Example: '[dev,pre-commit]'. No Empty string string run_pre-commit Run the pre-commit test job. No true boolean python_version_pre-commit The Python version to use for the pre-commit test job. No 3.9 string pip_index_url_pre-commit A URL to a PyPI repository index. No https://pypi.org/simple/ string pip_extra_index_urls_pre-commit A space-delimited string of URLs to additional PyPI repository indices. No Empty string string skip_pre-commit_hooks A comma-separated list of pre-commit hook IDs to skip when running pre-commit after updating hooks. No Empty string string run_pylint Run the pylint test job. No true boolean run_safety Run the safety test job. No true boolean python_version_pylint_safety The Python version to use for the pylint and safety test jobs. No 3.9 string pip_index_url_pylint_safety A URL to a PyPI repository index. No https://pypi.org/simple/ string pip_extra_index_urls_pylint_safety A space-delimited string of URLs to additional PyPI repository indices. No Empty string string pylint_targets Space-separated string of pylint file and folder targets.Note: This is only valid if pylint_runs is not defined. Yes, if pylint_runs is not defined Empty string string pylint_options Single (space-separated) or multi-line string of pylint command line options.Note: This is only valid if pylint_runs is not defined. No Empty string string pylint_runs Single or multi-line string with each line representing a separate pylint run/execution. This should include all desired options and targets.Important: The inputs pylint_options and pylint_targets will be ignored if this is defined. No Empty string string safety_options Single (space-separated) or multi-line string of safety command line options. No Empty string string run_build_package Run the build package test job. No true boolean python_version_package The Python version to use for the build package test job. No 3.9 string pip_index_url_package A URL to a PyPI repository index. No https://pypi.org/simple/ string pip_extra_index_urls_package A space-delimited string of URLs to additional PyPI repository indices. No Empty string string build_libs A space-separated list of packages to install via PyPI (pip install). No Empty string string build_cmd The package build command, e.g., 'flit build' or 'python -m build' (default). No python -m build string run_build_docs Run the build package test job. No true boolean python_version_docs The Python version to use for the build documentation test job. No 3.9 string pip_index_url_docs A URL to a PyPI repository index. No https://pypi.org/simple/ string pip_extra_index_urls_docs A space-delimited string of URLs to additional PyPI repository indices. No Empty string string relative Whether or not to use the locally installed Python package(s), and install it as an editable. No false boolean system_dependencies A single (space-separated) or multi-line string of Ubuntu APT packages to install prior to building the documentation. No Empty string string warnings_as_errors Build the documentation in 'strict' mode, treating warnings as errors.Important: If this is set to false, beware that the documentation may not be rendered or built as one may have intended.Default: true. No true boolean use_mkdocs Whether or not to build the documentation using the MkDocs framework. Mutually exclusive with use_sphinx. No false boolean use_sphinx Whether or not to build the documentation using the Sphinx framework. Mutually exclusive with use_mkdocs. No false boolean update_python_api_ref Whether or not to update the Python API documentation reference.Note: If this is true, package_dirs is required. No true boolean update_docs_landing_page Whether or not to update the documentation landing page. The landing page will be based on the root README.md file. No true boolean package_dirs A multi-line string of path to Python package directories relative to the repository directory to be considered for creating the Python API reference documentation.Example: 'src/my_package'. Yes, if update_python_api_ref is true (default) Empty string string exclude_dirs A multi-line string of directories to exclude in the Python API reference documentation. Note, only directory names, not paths, may be included. Note, all folders and their contents with these names will be excluded. Defaults to '__pycache__'.Important: When a user value is set, the preset value is overwritten - hence '__pycache__' should be included in the user value if one wants to exclude these directories. No __pycache__ string exclude_files A multi-line string of files to exclude in the Python API reference documentation. Note, only full file names, not paths, may be included, i.e., filename + file extension. Note, all files with these names will be excluded. Defaults to '__init__.py'.Important: When a user value is set, the preset value is overwritten - hence '__init__.py' should be included in the user value if one wants to exclude these files. No __init__.py string full_docs_dirs A multi-line string of directories in which to include everything - even those without documentation strings. This may be useful for a module full of data models or to ensure all class attributes are listed. No Empty string string full_docs_files A multi-line string of relative paths to files in which to include everything - even those without documentation strings. This may be useful for a file full of data models or to ensure all class attributes are listed. No Empty string string special_file_api_ref_options A multi-line string of combinations of a relative path to a Python file and a fully formed mkdocstrings option that should be added to the generated MarkDown file for the Python API reference documentation.Example: my_module/py_file.py,show_bases:false.Encapsulate the value in double quotation marks (\") if including spaces ( ).Important: If multiple package_dirs are supplied, the relative path MUST include/start with the appropriate 'package_dir' value, e.g., \"my_package/my_module/py_file.py,show_bases: false\". No Empty string string landing_page_replacements A multi-line string of replacements (mappings) to be performed on README.md when creating the documentation's landing page (index.md). This list always includes replacing 'docs/' with an empty string to correct relative links, i.e., this cannot be overwritten. By default '(LICENSE)' is replaced by '(LICENSE.md)'. No (LICENSE),(LICENSE.md) string landing_page_replacement_separator String to separate a mapping's 'old' to 'new' parts. Defaults to a comma (,). No , string debug Whether to do print extra debug statements. No false boolean sphinx-build_options Single or multi-line string of command-line options to use when calling sphinx-build.Note: The -W option will be added if warnings_as_errors is true (default). No Empty string string docs_folder The path to the root documentation folder relative to the repository root. No docs string build_target_folder The path to the target folder for the documentation build relative to the repository root. No site string"},{"location":"workflows/ci_update_dependencies/","title":"CI - Update dependencies PR","text":"File to use: ci_update_dependencies.yml
This workflow creates a PR if there are any updates in the permanent_dependencies_branch branch that have not been included in the default_repo_branch branch.
This workflow works nicely together with the CI - Check pyproject.toml dependencies workflow, and the same value for permanent_dependencies_branch should be used. In this way, this workflow can be called on a schedule to update the dependencies that have been merged into the permanent_dependencies_branch branch into the default_repo_branch branch.
The main point of having this workflow is to have a single PR, which can be squash merged, to merge several dependency updates performed by Dependabot or similar.
As a \"bonus\" this workflow supports updating pre-commit hooks.
PR branch name
The generated branch for the PR will be named ci/update-dependencies.
Warning
If a PAT is not passed through for the PAT secret and GITHUB_TOKEN is used, beware that any other CI/CD jobs that run for, e.g., pull request events, may not run since GITHUB_TOKEN-generated PRs are designed to not start more workflows to avoid escalation. Hence, if it is important to run CI/CD workflows for pull requests, consider passing a PAT as a secret to this workflow represented by the PAT secret.
Important
If this is to be used together with the CI/CD - New updates to default branch workflow, the pr_body_file supplied (if any) should be immutable within the first 8 lines, i.e., no check boxes or similar in the first 8 lines. Indeed, it is recommended to not supply a pr_body_file in this case.
There are no expectations of the repo when using this workflow.
"},{"location":"workflows/ci_update_dependencies/#inputs","title":"Inputs","text":"Name Description Required Default Typegit_username A git username (used to set the 'user.name' config option). Yes string git_email A git user's email address (used to set the 'user.email' config option). Yes string runner The runner to use for the workflow. Note, the callable workflow expects a Linux/Unix system.. No ubuntu-latest string permanent_dependencies_branch The branch name for the permanent dependency updates branch. No ci/dependency-updates string default_repo_branch The branch name of the repository's default branch. More specifically, the branch the PR should target. No main string pr_body_file Relative path to PR body file from the root of the repository.Example: '.github/utils/pr_body_update_deps.txt'. No Empty string string pr_labels A comma separated list of strings of GitHub labels to use for the created PR. No Empty string string extra_to_dos A multi-line string (insert \\n to create line breaks) with extra 'to do' checks. Should start with - [ ].See also Single vs multi-line input. No Empty string string update_pre-commit Whether or not to update pre-commit hooks as part of creating the PR. No false boolean python_version The Python version to use for the workflow.Note: This is only relevant if update_pre-commit is true. No 3.9 string install_extras Any extras to install from the local repository through 'pip'. Must be encapsulated in square parentheses ([]) and be separated by commas (,) without any spaces.Example: '[dev,pre-commit]'.Note: This is only relevant if update_pre-commit is true. No Empty string string pip_index_url A URL to a PyPI repository index.Note: This is only relevant if update_pre-commit is true. No https://pypi.org/simple/ string pip_extra_index_urls A space-delimited string of URLs to additional PyPI repository indices.Note: This is only relevant if update_pre-commit is true. No Empty string string skip_pre-commit_hooks A comma-separated list of pre-commit hook IDs to skip when running pre-commit after updating hooks.Note: This is only relevant if update_pre-commit is true. No Empty string string"},{"location":"workflows/ci_update_dependencies/#secrets","title":"Secrets","text":"Name Description Required PAT A personal access token (PAT) with rights to create PRs. This will fallback on GITHUB_TOKEN. No"},{"location":"workflows/ci_update_dependencies/#usage-example","title":"Usage example","text":"The following is an example of how a workflow may look that calls CI - Update dependencies PR. It is meant to be complete as is.
name: CI - Update dependencies\n\non:\n schedule:\n - cron: \"30 6 * * 3\"\n workflow_dispatch:\n\njobs:\n check-dependencies:\n name: Call external workflow\n uses: SINTEF/ci-cd/.github/workflows/ci_update_dependencies.yml@v2.8.3\n if: github.repository_owner == 'SINTEF'\n with:\n git_username: \"Casper Welzel Andersen\"\n git_email: \"CasperWA@github.com\"\n permanent_dependencies_branch: \"ci/dependency-updates\"\n default_repo_branch: stable\n pr_labels: \"CI/CD\"\n extra_to_dos: \"- [ ] Make sure the PR is **squash** merged, with a sensible commit message.\\n- [ ] Check related `requirements*.txt` files are updated accordingly.\"\n update_pre-commit: true\n python_version: \"3.9\"\n install_extras: \"[pre-commit]\"\n skip_pre-commit_hooks: \"pylint,pylint-models\"\n secrets:\n PAT: ${{ secrets.PAT }}\nThe content of repository files can be updated to use the new version where necessary.