This is a template repo for AI/ML model module.
- AI/ML model
- Python module/package
- Jupyter notebook
- Research
- Project Structure
- Template
- CI/CD
- Install Python (>= v3.9) and pip (>= 23):
- [RECOMMENDED] Miniconda (v3)
- [arm64/aarch64] Miniforge (v3)
- [Python virutal environment] venv
- [OPTIONAL] For GPU (NVIDIA):
- NVIDIA GPU driver (>= v452.39)
- NVIDIA CUDA (>= v11) and cuDNN (>= v8)
[OPTIONAL] For DEVELOPMENT environment:
- Install git
- Setup an SSH key (video tutorial)
2.1. Prepare projects directory (if not exists):
# Create projects directory:
mkdir -pv ~/workspaces/projects
# Enter into projects directory:
cd ~/workspaces/projects2.2. Follow one of the below options [A] or [B]:
OPTION A. Clone the repository:
git clone git@github.com:bybatkhuu/model.python-template.git simple_model && \
cd simple_modelOPTION B. Download source code:
- Download archived zip file from releases.
- Extract it into the projects directory.
- Rename the extracted directory from
model.python-templatetosimple_model.
Note
Choose one of the following methods to install the module [A ~ E]:
OPTION A. Install directly from git repository:
pip install git+https://github.com/bybatkhuu/model.python-template.gitOPTION B. Install from the downloaded source code:
# Install directly from the source code:
pip install .
# Or install with editable mode:
pip install -e .OPTION C. Install for DEVELOPMENT environment:
pip install -r ./requirements/requirements.dev.txtOPTION D. Install from pre-built package files (for PRODUCTION):
- Download
.whlor.tar.gzfile from releases. - Install with pip:
# Install from .whl file:
pip install ./simple_model-[VERSION]-py3-none-any.whl
# Or install from .tar.gz file:
pip install ./simple_model-[VERSION].tar.gzOPTION E. Copy the module into the project directory (for testing):
# Install python dependencies:
pip install -r ./requirements.txt
# Copy the module source code into the project:
cp -r ./src/simple_model [PROJECT_DIR]
# For example:
cp -r ./src/simple_model /some/path/project/## Standard libraries
import sys
import logging
from typing import Any
## Third-party libraries
import numpy as np
from numpy.typing import NDArray
## Internal modules
from simple_model import SimpleModel
logger = logging.getLogger(__name__)
if __name__ == "__main__":
logging.basicConfig(stream=sys.stdout, level=logging.INFO)
# Pre-defined variables (for customizing and testing)
_model_dir = "./models"
_model_name = "linear_regression.v0.0.1-24"
_X_train = np.array([[1], [2], [3], [4], [5]])
_y_train = np.array([2, 4, 6, 8, 10])
_X_test = np.array([[6], [7], [8]])
_y_test = np.array([10, 14, 16])
# Create the model instance
_config = {"models_dir": _model_dir, "model_name": _model_name}
_model = SimpleModel(config=_config)
# Train or load the model
if not SimpleModel.is_model_files_exist(**_config):
_model.train(X=_X_train, y=_y_train)
else:
_model.load()
# Predict the target values
_y_pred: NDArray[Any] = _model.predict(X=_X_test)
logger.info(f"Predicted values for {_X_test.flatten()}: {_y_pred.flatten()}")
# Evaluate the model
_r2_score: float = _model.score(y_true=_y_test, y_pred=_y_pred)
logger.info(f"R^2 score: {_r2_score}")
_is_similar: bool = _model.is_similar(X=_X_test, y=_y_test)
logger.info(f"Is similar: {_is_similar}")
# Save the model
if _model.is_trained() and (not SimpleModel.is_model_files_exist(**_config)):
_model.save()
logger.info("Done!")👍
simple_model: # Just an example to group the configs (Not necessary)
models_dir: "./models" # Directory where the models are saved
model_name: "linear_regression.v0.0.1-240101" # Name of the model as sub-directory
threshold: 0.5 # Threshold for similarity check# ENV=development
# DEBUG=trueTo run tests, run the following command:
# Install python test dependencies:
pip install -r ./requirements/requirements.test.txt
# Run tests:
python -m pytest -sv -o log_cli=true
# Or use the test script:
./scripts/test.sh -l -v -cTo build the python package, run the following command:
# Install python build dependencies:
pip install -r ./requirements/requirements.build.txt
# Build python package:
python -m build
# Or use the build script:
./scripts/build.shTo build the documentation, run the following command:
# Install python documentation dependencies:
pip install -r ./requirements/requirements.docs.txt
# Serve documentation locally (for development):
mkdocs serve
# Or use the docs script:
./scripts/docs.sh
# Or build documentation:
mkdocs build
# Or use the docs script:
./scripts/docs.sh -b