chore(ci/cd): add pipelines + modernize repo (#193)

This PR adds a CD pipeline for pydgraph. We also bring the repo
into the modern era to support Python 3.11, and use modern build
tools. Some noteworthy points:
 * python setup.py [command] is being deprecated (see here e.g.).
 * The correct approach now is to use "front end" build tools like python
     -m build and pip install -e . instead of invoking setup.py directly
 * We introduce a pyproject.toml file, which deprecates setup.py, requirements.txt, setup.cfg, etc.
 * We remove requirements.txt since this information can be found in pyproject.toml
 * Dev requirements introduced as optional dependencies in pyproject.toml
     (use pip install .[dev] from local repo, or pip install pydgraph[dev] once it is pushed to pypi)
 * Test framework is unified (i.e. local tests and CI tests use the same script local-test.sh)
 * README and PUBLISHING updated to reflect the above changes

.github/workflows/cd-pydgraph.yml

@@ -0,0 +1,37 @@
+name: cd-pydgraph
+on:
+  workflow_dispatch:
+    inputs:
+      releasetag:
+        description: 'releasetag'
+        required: true
+        type: string
+jobs:
+  pydgraph-build:
+    runs-on: ubuntu-20.04
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          ref: '${{ github.event.inputs.releasetag }}'
+      - name: Set up Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: ${{ env.PYTHON_VERSION }}
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install .[dev]
+          # use twine to upload to pypi
+          python -m pip install twine
+      - name: Verify that protobufs are checked in
+        run: |
+          # should be a no-op
+          python scripts/protogen.py
+          git diff --exit-code -- .
+      - name: Build pydgraph
+        run: |
+          rm -rf dist
+          # build binary wheel and a source tarball
+          python -m build
+      - name: Upload to PyPi
+        run: twine upload --username dgraph-io --password ${{ secrets.DGRAPH_PYPI_PASSWORD }} dist/*       

.github/workflows/ci-pydgraph-tests.yml

@@ -57,7 +57,6 @@ jobs:
         run: |
           cd pydgraph
           python -m pip install --upgrade pip
-          # see https://github.com/pypa/pip/issues/8559
-          pip install -r requirements.txt --use-pep517
+          python -m pip install .[dev]
       - name: Run tests
         run: cd pydgraph && bash scripts/local-test.sh

.gitignore

@@ -35,3 +35,4 @@ pydgraph.iml
 # Python Virtual Environments
 venv
 .venv
+pyvenv.cfg

PUBLISHING.md

@@ -6,58 +6,14 @@ This document contains instructions to publish pydgraph to [PyPI].
 
 ## Before deploying
 
-- Get access to credentials for Dgraph's account on PyPI with username dgraph-io
-- Setup you  `~/.pypirc` file like this: (remember to replace `${password}` with
-  the actual password)
-
-```
-[distutils]
-index-servers=
-    pypi
-    testpypi
-
-[pypi]
-username: dgraph-io
-password: ${password}
-
-[testpypi]
-repository: https://test.pypi.org/legacy/
-username: dgraph-io
-password: ${password}
-```
+- Build pydgraph locally (see readme)
+- Push to testpypi (`twine upload --repository testpypi dist/*`)
+- Verify readme and metadata look correct
 
 ## Deploying
 
-- Build and test the code that needs to be published
+- Regenerate protobufs if api.proto was changed
 - Bump version by modifying the `VERSION` variable in `pydgraph/meta.py` file
-- If necessary, update the `CHANGELOG.md` file to reflect new changes
-- Commit the changes
-- Make sure you have [setuptools], [wheel], [twine], and [pypandoc]
-  installed. You can install them by running the following:
-
-  ```
-  pip install -r publishing-requirements.txt
-  ```
-- Run the following commands:
-
-```sh
-# Remove build and dist directories
-rm -rf build
-rm -rf dist
-
-# Package you project: source distribution and wheel
-python setup.py sdist
-python setup.py bdist_wheel
-
-# Upload it to PyPI
-twine upload dist/*
-# For testing, try uploading to testpypi:
-# twine upload --repository testpypi dist/*
-```
-
-- If necessary, create a new release tag on the Github repository
-
-[setuptools]: https://pypi.org/project/setuptools/
-[wheel]: https://pypi.org/project/wheel/
-[twine]: https://pypi.org/project/twine/
-[pypandoc]: https://pypi.org/project/pypandoc/
+- Update the `CHANGELOG.md` file to reflect new changes
+- Tag pydgraph locally (`git tag x.y.z`) and push to origin (`git push x.y.z`)
+- Run CD pipeline

README.md

@@ -6,8 +6,6 @@ This is the official Dgraph database client implementation for Python (Python >=
 
 This client follows the [Dgraph Go client][goclient] closely.
 
-**Use [Discuss Issues](https://discuss.dgraph.io/c/issues/35/clients/46) to report issues about this repository.**
-
 [goclient]: https://github.com/dgraph-io/dgo
 
 Before using this client, we highly recommend that you read the [Dgraph Python
@@ -45,24 +43,22 @@ Install using pip:
 pip install pydgraph
 ```
 
-### Install Notes
-
-To avoid issues when adding composite credentials or when using client authorization, please install gRPC version 1.19.0:
-
-```sh
-pip install grpcio==1.19.0
-```
-
 ## Supported Versions
 
 Depending on the version of Dgraph that you are connecting to, you will have to
 use a different version of this client.
 
 | Dgraph version |   pydgraph version   |
 |:--------------:|:--------------------:|
-|     1.0.X      |      <= *1.2.0*      |
-|     1.1.X      |      >= *2.0.0*      |
-|     1.2.X      |      >= *2.0.0*      |
+|     1.0.X      |      *1.2.0*         |
+|     1.1.X      |      *2.0.0*         |
+|     1.2.X      |      *2.0.0*         |
+|    20.3.X      |      *20.3.0*        |
+|    20.7.X      |      *20.7.0*        |
+|    20.11.X     |      *20.7.0*        |
+|    21.X.Y      |      *21.3.0*        |
+|    22.X.Y      |      *21.3.0*        |
+|    23.X.Y      |      *23.0.0*        |
 
 ## Quickstart
 
@@ -100,7 +96,7 @@ op = pydgraph.Operation(schema=schema)
 client.alter(op)
 ```
 
-Starting with Dgraph version 20.03.0, indexes can be computed in the background.
+Indexes can be computed in the background.
 You can set the `run_in_background` field of `pydgraph.Operation` to `True`
 before passing it to the `Alter` function. You can find more details
 [here](https://docs.dgraph.io/master/query-language/#indexes-in-background).
@@ -465,12 +461,22 @@ except Exception as e:
 
 ## Development
 
-### Building the source
+### Setting up environment
+
+There are many ways to set up your local Python environment.  We suggest some sane defaults here.
+
+- Use [pyenv](https://github.com/pyenv/pyenv) to manage your Python installations.
+- Most recent versions of Python should work, but the version of Python officially supported is located in
+`.python-version`
+- Create a Python virtual environment using `python -m venv .venv`
+- Activate virtual environment via `source .venv/bin/activate`
+
+### Build from source
+
+To build and install pydgraph locally, run
 
 ```sh
-python setup.py install
-# To install for the current user, use this instead:
-# python setup.py install --user
+pip install -e .[dev]
 ```
 
 If you have made changes to the `pydgraph/proto/api.proto` file, you need need
@@ -494,16 +500,15 @@ The required change is outlined below as a diff.
 
 ### Running tests
 
-To run the tests in your local machine, install and run `tox`:
+To run the tests in your local machine, run:
 
 ```bash
-pip install tox
-tox
+bash scripts/local-test.sh
 ```
 
-This script assumes Dgraph and [dgo](https://github.com/dgraph-io/dgo) (Go
-client) are already built on the local machine and that their code is in
-`$GOPATH/src`. It also requires that docker and docker-compose are installed in
+This script assumes dgraph is located on your path.  Dgraph release binaries can 
+be found [here](https://github.com/dgraph-io/dgraph/releases).
+The test script also requires that `docker` and `docker compose` are installed on
 your machine.
 
 The script will take care of bringing up a Dgraph cluster and bringing it down

pydgraph/meta.py

@@ -14,4 +14,4 @@
 
 """Metadata about this package."""
 
-VERSION = '21.3.2'
+VERSION = '23.0.0rc1'