Refactor test suite to use tox and pytest, and have separate director…

…ies for unit and integration tests. Update READMEs, and fishbowl scripts and jinja templates. Refactor fishbowl to generate import strings.

README.md

@@ -14,7 +14,7 @@
 
 ## Installation
 
-Gafferpy requires Python 3.6+. We don't currently release gafferpy on pypi, but you can install it over ssh with:
+`gafferpy` requires Python 3.6+. We do not currently release `gafferpy` on PyPI, but you can install it over ssh with:
 
 ```bash
 pip install git+https://github.com/gchq/gafferpy.git
@@ -28,22 +28,24 @@ pip install -e .
 
 ## Quick Start
 
-The python shell connects to a running Gaffer REST API.  
-You can start the Gaffer road-traffic-demo rest server from the Gaffer repository, using the command:
+The Python shell connects to a running Gaffer REST API.  
+You can start the Gaffer [`road-traffic-demo`](https://github.com/gchq/Gaffer/blob/master/example/road-traffic/README.md) REST server from the Gaffer repository, using the command:
 
 ```bash
 mvn clean install -pl :road-traffic-demo -Proad-traffic-demo,quick
 ```
-
+To connect to the running Gaffer API from Python (more information on the Python shell can be found [here](https://gchq.github.io/gaffer-doc/latest/user-guide/apis/python-api/)):
 ```python
 # Import the client library and connector
 from gafferpy import gaffer as g
 from gafferpy import gaffer_connector
 
 # Instantiate a connector
 gc = gaffer_connector.GafferConnector("http://localhost:8080/rest/latest")
-
-# You can use the connector to perform get requests
+```
+Then perform requests against it:
+```python
+# You can use the connector to perform GET requests
 schema = gc.execute_get(g.GetSchema())
 
 # And also run operations
@@ -73,38 +75,50 @@ elements = gc.execute_operation_chain(
 )
 ```
 
-See [operation examples](https://gchq.github.io/gaffer-doc/v1docs/getting-started/operations/contents) for more examples of operations in python.
+See [Operations Guide](https://gchq.github.io/gaffer-doc/latest/reference/operations-guide/operations) for more examples of operations in Python.
 
-## Coding Style
-Please ensure that your coding style is consistent with the rest of the Gaffer project. Guides on the coding style for Gaffer can be found [here](https://gchq.github.io/gaffer-doc/latest/ways-of-working/#coding-style)
+## Developer Guide
 
-## Testing
+### Coding Style
+Please ensure that your coding style is consistent with the rest of the Gaffer project. Guides on the coding style for `gafferpy` and Gaffer can be found [here](https://gchq.github.io/gaffer-doc/latest/development-guide/ways-of-working).
 
-```bash
-# To run all of the tests, first deploy the road traffic example
-# This needs to be done from the Gaffer java repository
-mvn clean install -pl :road-traffic-demo -Proad-traffic-demo,quick
+### Testing
 
-# Then run
-python -m unittest discover
+The `gafferpy` tests are implemented using `tox` and `pytest`.
+To run the tests, install the `gafferpy` 'dev' extra:
+```bash
+pip install -e ".[dev]" # the quotes ensure compatibilty with zsh
 ```
+This will install extra development dependecies for running tests and building documentation.
 
-## Building the documentation
+`gafferpy` has both unit tests and integration tests - the integration tests use the Road Traffic Example to test the Python API. If this is not running, the integration tests are skipped. It is advisable that the integration tests are run prior to any code commits to ensure they do not fail due to any code changes.
 
-To build the docs locally, assuming you have Python installed, install the docs dependencies:
+*For help starting the Road Traffic Example, see the [Quick Start](#quick-start) section above.*
+
+To run the tests, execute the below from the root directory of `gafferpy`:
 ```bash
-cd docs
-pip install -r requirements.txt
+tox
+```
+By default, `tox` will and try run the tests in multiple test envs (different Python versions) - if they do not exist then they are skipped.
+To run the test for a specifc test env e.g. Python3.9, run:
+```bash
+tox -e py39
 ```
 
-Then use the sphinx Makefile:
+### Building the documentation
+
+To build the docs locally, assuming you have Make and Python installed, and `gafferpy` installed with the 'dev' extra:
 ```bash
+cd docs
 make html
 ```
 
+### Generating the Python API
+`gafferpy` has the ability to regenerate the Python API based upon the Gaffer REST API that a `GafferConnector` object is pointing at - a more detailed description and examples of how to do this can be found [here](./src/fishbowl/README.md).
+
 ## License
 
-Copyright 2016-2022 Crown Copyright
+Copyright 2016-2023 Crown Copyright
 
 Licensed under the Apache License, Version 2.0 \(the "License"\); you may not use this file except in compliance with the License. You may obtain a copy of the License at
 

setup.py

@@ -1,13 +1,13 @@
 import re
 from codecs import open
-from os import path
+from pathlib import Path
 from setuptools import setup, find_packages
 
 ###############################################################################
 
 name = "gafferpy"
 packages = find_packages(where="src")
-meta_path = path.join("src", "gafferpy", "__init__.py")
+meta_path = Path("src", "gafferpy", "__init__.py")
 keywords = ["class", "attribute", "boilerplate"]
 classifiers = [
     "Development Status :: 4 - Beta",
@@ -24,20 +24,27 @@
 python_requires = ">3.6"
 install_requires = []
 extras_require = {
-    "requests": ["requests>=2.4.0"]
+    "requests": ["requests>=2.4.0"],
+    "dev": [
+        "tox",
+        "pytest",
+        "requests>=2.4.0",
+        "sphinx~=7.2.6",
+        "sphinx-rtd-theme~=1.3.0"
+    ]
 }
 
 ###############################################################################
 
-here = path.abspath(path.dirname(__file__))
+here = Path(__file__).parent
 
 
-def read(*parts):
+def read(parts):
     """
     Build an absolute path from *parts* and and return the contents of the
     resulting file.  Assume UTF-8 encoding.
     """
-    with open(path.join(here, *parts), "rb", "utf-8") as f:
+    with open(here / parts, "rb", "utf-8") as f:
         return f.read()
 
 
@@ -54,14 +61,14 @@ def find_meta(meta):
     )
     if meta_match:
         return meta_match.group(1)
-    raise RuntimeError("Unable to find __{meta}__ string.".format(meta=meta))
+    raise RuntimeError(f"Unable to find __{meta}__ string.")
 
 
 version = find_meta("version")
 uri = find_meta("uri")
 
 # Get the long description from the README.md file
-with open(path.join(here, "README.md"), encoding="utf-8") as f:
+with open(here / "README.md", encoding="utf-8") as f:
     long = f.read()
 
 setup(

src/fishbowl/README.md

@@ -1,7 +1,7 @@
 # Fishbowl
 
 ## Features
-Fishbowl generates code for the core gafferpy classes by pointing to a running Gaffer rest api.
+Fishbowl generates code for the core `gafferpy` classes by pointing to a running Gaffer REST API.
 It generates the classes for:
 
 - [Operations](../gafferpy/generated_api/operations.py)
@@ -10,22 +10,27 @@ It generates the classes for:
 - [Binary Operators](../gafferpy/generated_api/binary_operators.py)
 - [Config Endpoints](../gafferpy/generated_api/config.py)
 
-This is done automatically on every Gaffer release from a standard spring-rest api and placed into gafferpy's [generated_api](../gafferpy/generated_api) directory.
+This is done automatically on every Gaffer release from a standard spring-rest API and placed into `gafferpy`'s [generated_api](../gafferpy/generated_api) directory.
 
-If you have custom Gaffer Java classes from the list above, fishbowl can generate the gafferpy classes for them.
+If you have custom Gaffer Java classes from the list above, `fishbowl` can generate the `gafferpy` classes for them.
 
 ## Quick Start
 
+To regenerate the core `gafferpy` library from a spring-rest Gaffer API, first start the Gaffer API from the root directory of the Gaffer repository:
+```bash
+mvn spring-boot:run -pl :spring-rest -Pdemo
+```
+You can then use `fishbowl` to generate the Python API:
 ```python
 from gafferpy.gaffer_connector import GafferConnector
 from fishbowl.fishbowl import Fishbowl
 
 gc = GafferConnector("http://localhost:8080/rest")
 fb = Fishbowl(gaffer_connector=gc)
 ```
-Your python files will be appear in a folder called `generated`.
+Your Python files will be appear in a folder called `generated`.
 
-You can then use these classes in normal gafferpy operations:
+You can then use these classes in normal `gafferpy` operations:
 ```python
 import generated
 gc.execute(generated.operations.YourCustomOperation(custom_field=True))
@@ -39,7 +44,7 @@ gc.execute(generated.operations.OperationChain(
 
 ## License
 
-Copyright 2019-2022 Crown Copyright
+Copyright 2019-2023 Crown Copyright
 
 Licensed under the Apache License, Version 2.0 \(the "License"\); you may not use this file except in compliance with the License. You may obtain a copy of the License at