Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Code Cleanup #23

Merged
merged 29 commits into from
Oct 25, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
96fc28e
code cleanup
aMahanna Oct 5, 2023
1687bbe
Merge branch 'master' into code-cleanup
aMahanna Oct 5, 2023
094a06b
attempt fix: `pip install torch`
aMahanna Oct 6, 2023
3bdbedd
attempt fix: remove `-e`
aMahanna Oct 6, 2023
539bb02
attempt fix: remove cache
aMahanna Oct 6, 2023
d38254a
fix: bad comment
aMahanna Oct 6, 2023
26125ad
attempt fix: `pip install torch-sparse`
aMahanna Oct 6, 2023
4925a49
attempt fix: torch installations
aMahanna Oct 6, 2023
632fccd
attempt fix: upgrade pip
aMahanna Oct 6, 2023
92cd0c0
cleanup: workflows
aMahanna Oct 6, 2023
3005ba2
cleanup: `adapter.py`
aMahanna Oct 6, 2023
3669f5e
cleanup: workflows
aMahanna Oct 6, 2023
fbb1fba
fix: install publishing dependencies
aMahanna Oct 6, 2023
2dd0626
extracting more logic into separate functions
aMahanna Oct 12, 2023
586ce56
fix: mypy
aMahanna Oct 12, 2023
c33107e
rename kwargs
aMahanna Oct 12, 2023
48160f1
bump `adbpyg-adapter` to `1.1.2` in notebooks
aMahanna Oct 12, 2023
a983de9
fix param order
aMahanna Oct 15, 2023
377ebb3
bump
aMahanna Oct 15, 2023
4cfabc4
Delete build_self_hosted.yml
aMahanna Oct 17, 2023
29b151d
cleanup workflows
aMahanna Oct 19, 2023
340beeb
Update build.yml
aMahanna Oct 19, 2023
554e59f
Update build.yml
aMahanna Oct 19, 2023
3d8bb27
Update build.yml
aMahanna Oct 19, 2023
8a957c2
parameter renaming, new `use_async` param
aMahanna Oct 19, 2023
e6b6920
fix abc
aMahanna Oct 19, 2023
fdf4d8a
fix mypy
aMahanna Oct 19, 2023
d2390ad
add `strict` param to other adb -> pyg methods
aMahanna Oct 19, 2023
de5267e
Update setup.py
aMahanna Oct 19, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/analyze.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ jobs:

steps:
- name: Checkout repository
uses: actions/checkout@v2
uses: actions/checkout@v4

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
Expand Down
30 changes: 21 additions & 9 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
name: build
on:
workflow_dispatch:
push:
branches: [ master ]
pull_request:
push:
branches: [ master ]
env:
PACKAGE_DIR: adbpyg_adapter
Expand All @@ -16,31 +15,44 @@ jobs:
python: ["3.8", "3.9", "3.10", "3.11"]
name: Python ${{ matrix.python }}
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4

- name: Setup Python ${{ matrix.python }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python }}
cache: 'pip'
cache-dependency-path: setup.py

- name: Set up ArangoDB Instance via Docker
run: docker create --name adb -p 8529:8529 -e ARANGO_ROOT_PASSWORD= arangodb/arangodb

- name: Start ArangoDB Instance
run: docker start adb

- name: Install packages
run: pip install torch && pip install -e .[dev]
run: |
pip install torch
pip install torch-scatter torch-sparse -f https://data.pyg.org/whl/torch-$(python -c 'import torch; print(torch.__version__.split("+")[0])')+cpu.html
pip install -e .[dev]

- name: Run black
run: black --check --verbose --diff --color ${{env.PACKAGE_DIR}} ${{env.TESTS_DIR}}

- name: Run flake8
run: flake8 ${{env.PACKAGE_DIR}} ${{env.TESTS_DIR}}

- name: Run isort
run: isort --check --profile=black ${{env.PACKAGE_DIR}} ${{env.TESTS_DIR}}

- name: Run mypy
run: mypy ${{env.PACKAGE_DIR}} ${{env.TESTS_DIR}}
- name: Set up ArangoDB Instance via Docker
run: docker create --name adb -p 8529:8529 -e ARANGO_ROOT_PASSWORD= arangodb/arangodb
- name: Start ArangoDB Instance
run: docker start adb

- name: Run pytest
run: pytest --cov=${{env.PACKAGE_DIR}} --cov-report xml --cov-report term-missing -v --color=yes --no-cov-on-fail --code-highlight=yes

- name: Publish to coveralls.io
if: matrix.python == '3.8'
if: matrix.python == '3.10'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: coveralls --service=github
51 changes: 0 additions & 51 deletions .github/workflows/build_self_hosted.yml

This file was deleted.

69 changes: 10 additions & 59 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,70 +3,23 @@ on:
workflow_dispatch:
release:
types: [published]
env:
PACKAGE_DIR: adbpyg_adapter
TESTS_DIR: tests
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
python: ["3.8", "3.9", "3.10", "3.11"]
name: Python ${{ matrix.python }}
steps:
- uses: actions/checkout@v3
- name: Setup Python ${{ matrix.python }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python }}
cache: 'pip'
cache-dependency-path: setup.py
- name: Install packages
run: pip install torch && pip install -e .[dev]
- name: Run black
run: black --check --verbose --diff --color ${{env.PACKAGE_DIR}} ${{env.TESTS_DIR}}
- name: Run flake8
run: flake8 ${{env.PACKAGE_DIR}} ${{env.TESTS_DIR}}
- name: Run isort
run: isort --check --profile=black ${{env.PACKAGE_DIR}} ${{env.TESTS_DIR}}
- name: Run mypy
run: mypy ${{env.PACKAGE_DIR}} ${{env.TESTS_DIR}}
- name: Set up ArangoDB Instance via Docker
run: docker create --name adb -p 8529:8529 -e ARANGO_ROOT_PASSWORD= arangodb/arangodb
- name: Start ArangoDB Instance
run: docker start adb
- name: Run pytest
run: pytest --cov=${{env.PACKAGE_DIR}} --cov-report xml --cov-report term-missing -v --color=yes --no-cov-on-fail --code-highlight=yes
- name: Publish to coveralls.io
if: matrix.python == '3.8'
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: coveralls --service=github

release:
needs: build
runs-on: ubuntu-latest
name: Release package
steps:
- uses: actions/checkout@v3
- name: Setup Python 3.8
uses: actions/setup-python@v4
with:
python-version: "3.8"
cache: 'pip'
cache-dependency-path: setup.py
- uses: actions/checkout@v4

- name: Fetch complete history for all tags and branches
run: git fetch --prune --unshallow

- name: Install packages
run: |
pip install setuptools wheel twine setuptools-scm[toml]
pip install torch
pip install .[dev]
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: "3.10"

- name: Remove (old) distribution
run: rm -rf dist
- name: Install release packages
run: pip install setuptools wheel twine setuptools-scm[toml]

- name: Build distribution
run: python setup.py sdist bdist_wheel
Expand All @@ -88,7 +41,7 @@ jobs:
runs-on: ubuntu-latest
name: Update Changelog
steps:
- uses: actions/checkout@v3
- uses: actions/checkout@v4
with:
fetch-depth: 0

Expand All @@ -100,12 +53,10 @@ jobs:
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

- name: Setup Python 3.8
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: "3.8"
cache: 'pip'
cache-dependency-path: setup.py
python-version: "3.10"

- name: Install release packages
run: pip install wheel gitchangelog pystache
Expand Down
30 changes: 5 additions & 25 deletions adbpyg_adapter/abc.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
# -*- coding: utf-8 -*-

from abc import ABC
from typing import Any, List, Set, Union
from typing import Any, Set, Union

from arango.graph import Graph as ArangoDBGraph
from torch_geometric.data import Data, HeteroData
Expand All @@ -16,17 +16,17 @@ def __init__(self) -> None:
raise NotImplementedError # pragma: no cover

def arangodb_to_pyg(
self, name: str, metagraph: ADBMetagraph, **query_options: Any
self, name: str, metagraph: ADBMetagraph, **adb_export_kwargs: Any
) -> Union[Data, HeteroData]:
raise NotImplementedError # pragma: no cover

def arangodb_collections_to_pyg(
self, name: str, v_cols: Set[str], e_cols: Set[str], **query_options: Any
self, name: str, v_cols: Set[str], e_cols: Set[str], **adb_export_kwargs: Any
) -> Union[Data, HeteroData]:
raise NotImplementedError # pragma: no cover

def arangodb_graph_to_pyg(
self, name: str, **query_options: Any
self, name: str, **adb_export_kwargs: Any
) -> Union[Data, HeteroData]:
raise NotImplementedError # pragma: no cover

Expand All @@ -37,30 +37,10 @@ def pyg_to_arangodb(
metagraph: PyGMetagraph = {},
explicit_metagraph: bool = True,
overwrite_graph: bool = False,
**import_options: Any,
**adb_import_kwargs: Any,
) -> ArangoDBGraph:
raise NotImplementedError # pragma: no cover

def etypes_to_edefinitions(self, edge_types: List[EdgeType]) -> List[Json]:
raise NotImplementedError # pragma: no cover

def ntypes_to_ocollections(
self, node_types: List[str], edge_types: List[EdgeType]
) -> List[str]:
raise NotImplementedError # pragma: no cover

def __fetch_adb_docs(self) -> None:
raise NotImplementedError # pragma: no cover

def __insert_adb_docs(self) -> None:
raise NotImplementedError # pragma: no cover

def __build_tensor_from_dataframe(self) -> None:
raise NotImplementedError # pragma: no cover

def __build_dataframe_from_tensor(self) -> None:
raise NotImplementedError # pragma: no cover


class Abstract_ADBPyG_Controller(ABC):
def _prepare_pyg_node(self, pyg_node: Json, node_type: str) -> Json:
Expand Down
Loading
Loading