Feature: Update Python pyproject.toml files to set the minimum Python version to >=3.8

[meliz19]

check-pyproject-updates.zip

Description

Currently, aiSSEMBLE sets the minimum Python version to 3.11.4 for its Python modules. We originally did this due to a misunderstanding of how Poetry works. There is no reason why we cannot allow users to use lower Python versions. To extend the use of aiSSEMBLE to a broader user base, this ticket will update the minimum Python version in the pyproject.toml files to >=3.8.

DOD

• Create a fork of Krausening to update the minimum version of Python required for the library.(Change the minimum Python version to >=3.8 instead of ^3.9 to allow for Python 3.8 support TechnologyBrewery/krausening#29)
• Update aiSSEMBLE's PyPi modules to reflect the minimum version of Python required for each library

Test Strategy/Script

• git pull the latest version of the repo
• Run mvn clean install
• Download the attached zip file. Unzip the Python file & run it with the command below to verify all 20 of the METADATA files are updated with python 3.8 - 3.12
  python check-pyproject-updates.py </path/to/aissemble>

For OTS only

• Generate a new project using the command below.

mvn archetype:generate -B -DarchetypeGroupId=com.boozallen.aissemble \
                          -DarchetypeArtifactId=foundation-archetype \
                          -DarchetypeVersion=1.9.0-SNAPSHOT \
                          -DartifactId=test-project\
                          -DgroupId=org.test \
                          -DprojectName='Test' \
                          -DprojectGitUrl=test.org/test-project\
&& cd test-project

• Add the following pipelines to test-project-pipeline-models/src/main/resources/pipelines/.
  DataDeliveryPipeline.json:

{
    "name": "DataDeliveryPipeline",
    "package": "com.boozallen",
    "type": {
      "name": "data-flow",
      "implementation": "data-delivery-pyspark"
    },
    "steps": [
      {
        "name": "PysparkStep",
        "type": "synchronous"
      }
    ]
  }

MachineLearningPipeline.json

{
    "name": "MachineLearningPipeline",
    "package": "com.boozallen",
    "type": {
      "name": "machine-learning",
      "implementation": "machine-learning-mlflow"
    },
    "steps": [
      {
        "name": "TrainingStep",
        "type": "training"
      },
      {
        "name": "InferenceStep",
        "type": "inference"
      },
      {
        "name": "SagemakeTrainingStep",
        "type": "sagemakertraining"
      }
    ]
  }

• Run mvn clean install and resolve all the Manual Actions.
  • In each module that includes a pyproject.toml, you will need to update the relevant aissemble-generated packages to reference the local path. Note Only replace the relevant packages that are listed in the pyproject.toml.

  aissemble-foundation-core-python = {path = "/home/mblack/aissemble/foundation/aissemble-foundation-core-python", develop = true}
  aissemble-foundation-pdp-client-python = {path = "/home/mblack/aissemble/foundation/foundation-security/aissemble-foundation-pdp-client-python", develop = true}
  aissemble-foundation-encryption-policy-python = {path = "/home/mblack/aissemble/foundation/foundation-encryption/aissemble-foundation-encryption-policy-python", develop = true}
  aissemble-extensions-encryption-vault-python = {path = "/home/mblack/aissemble/extensions/extensions-encryption/aissemble-extensions-encryption-vault-python", develop = true}
  aissemble-extensions-data-delivery-spark-py = {path = "/home/mblack/aissemble/extensions/extensions-data-delivery/aissemble-extensions-data-delivery-spark-py", develop = true}
  

• Run with the command below to verify all 4 of the METADATA files are updated with python 3.8 - 3.12
  python check-pyproject-updates.py </path/to/downstream/project>

References/Additional Context

The current Python modules (1.8.2) on PyPi show that Python 3.12 or just 3 are required.

• https://pypi.org/project/aissemble-foundation-encryption-policy-python/1.8.2/
• https://pypi.org/project/aissemble-foundation-encryption-policy-python/1.8.2/
• https://pypi.org/project/aissemble-extensions-data-delivery-spark-py/1.8.2/
• https://pypi.org/project/aissemble-foundation-model-lineage/1.8.2/
• https://pypi.org/project/aissemble-foundation-model-lineage/1.8.2/
• https://pypi.org/project/aissemble-extensions-model-training-api-sagemaker/

[cpointe-ibllanos]

Completed an over-the-shoulder. Python upgrade worked as did the mda file changes.

[jaebchoi]

Test Passed

• aissemble built Successfully,
• running check-pyproject-updates.py updated 20 METADATA as expected.