Preprint: https://arxiv.org/abs/2106.07327
Image classification is an important task in various machine learning applications. In recent years, a number of classification methods based on quantum machine learning and different quantum image encoding techniques have been proposed. In this paper, we study the effect of three different quantum image encoding approaches on the performance of a convolution-inspired hybrid quantum-classical image classification algorithm called quanvolutional neural network (QNN). We furthermore examine the effect of variational - i.e. trainable - quantum circuits on the classification results. Our experiments indicate that some image encodings are better suited for variational circuits. However, our experiments show as well that there is not one best image encoding, but that the choice of the encoding depends on the specific constraints of the application. This repository is the official implementation of Variational Quanvolutional Neural Networks with enhanced image encoding.
For the purpose of reproducibility we use a Docker environment. We provide different options depending on the available resources and the need for development options:
docker-compose.yml for simply running the experiments with the default parameters.
docker-compose-dev.yml same parameters but also makes the code available for quick edits in the docker container.
Additionally, GPUs can be utilized with the following two environments when "app_params["q-device"] = "qulacs.simulator"
is set in main.py.
docker-compose-gpu.yml for simply running the experiments with the default parameters under utilization of a GPU.
docker-compose-dev-gpu.yml same parameters but also makes the code available within the container to allow for easy execution of modified the code (no rebuild of the image necessary).
In the top-level code directory run the following commands to set up the container and start bash:
docker-compose -f docker-compose.yml up -d
docker-compose exec experiments bash
From now on all commands are run in this bash shell, unless stated otherwise.
To train the models in the paper, run the following procedure.
First reproduce our model parameters in yaml files:
python generate_experiments.py
Then you can run the training procedure with the yaml file corresponding to the desired kernel size and random seed:
python main.py experiments_seed_2_filtersize_2x2_imgsize_14.yaml
The code will generate entries in the save directory, from which you can inspect the parameters and the performance of the models and reload the pre-trained models.
The execution of the training can be sped up significantly by pre-encoding the MNIST data and saving the results to disk. To use pre-encoding, change the parameter app_params["pre_encoding"] in main.py to True. Important: Pre-encoding works only with the "default.qubit"-simulator. Once the images are pre-encoded, you are free to use any other simulator-backend (e.g. qulacs.simulator).
You can observe the performance of the models, while they are training or afterwards, through tensorboard. Enter this address into your browser, after the experiments have been started, to reach the tensorboard started by default.
localhost:6006
You can also recreate all of the graphs from our paper with the Jupyter Notebook located at:
code/evaluation/Evaluation.ipynb
You can test models on before unseen data. Simply insert the save location of the models, you wish to test into the paths entry of the test_params dictionary in test_models.py. For newly trained models this is already performed in main.py as specified in the default training parameters.
python test_models.py
All our models can be reproduced using the commands above. However, this is time consuming. We provide pre-trained model checkpoints in the evaluation/MNIST14x14-folder.
These following two tables show the average values of the training, validation and test accuracy grouped by filter size, encoding algorithm and trainability of the quantum circuit and averaged per group over all different random circuits (seeds 0-9). The mean values for training and validation are additionally averaged over the last 20 epochs in order to better estimate only the effects of trainability and encoding algorithm. The max values are not averaged but belong to a certain model of the specific group over the whole course of training. All models had been tested after 50 epochs of training on the same set of 1000 images.
| Encoding | training | validation | test | |||
|---|---|---|---|---|---|---|
| mean | max | mean | max | mean | max | |
| FRQI untrainable | 0.793 | 0.940 | 0.789 | 0.95 | 0.806 | 0.878 |
| FRQI trainable | 0.862 | 0.935 | 0.853 | 0.93 | 0.854 | 0.884 |
| NEQR untrainable | 0.785 | 0.875 | 0.773 | 0.93 | 0.760 | 0.830 |
| NEQR trainable | 0.786 | 0.860 | 0.769 | 0.89 | 0.774 | 0.839 |
| Threshold untrainable | 0.818 | 0.910 | 0.796 | 0.90 | 0.792 | 0.882 |
| Threshold trainable | 0.827 | 0.925 | 0.796 | 0.92 | 0.825 | 0.881 |
| Encoding | training | validation | test | |||
|---|---|---|---|---|---|---|
| mean | max | mean | max | mean | max | |
| FRQI untrainable | 0.639 | 0.870 | 0.623 | 0.84 | 0.602 | 0.826 |
| FRQI trainable | 0.737 | 0.865 | 0.732 | 0.87 | 0.715 | 0.836 |
| NEQR untrainable | 0.755 | 0.870 | 0.756 | 0.92 | 0.727 | 0.789 |
| NEQR trainable | 0.783 | 0.880 | 0.780 | 0.92 | 0.759 | 0.846 |
| Threshold untrainable | 0.818 | 0.890 | 0.796 | 0.91 | 0.828 | 0.859 |
| Threshold trainable | 0.812 | 0.880 | 0.793 | 0.89 | 0.802 | 0.845 |
These now following two tables show the average training duration for each experimental configuration of filter size, encoding and trainability. The training was conducted on a compute cluster with 4 GPUs and 40 CPUs. The quantum states of the input images had been pre-encoded to accelerate the training.
| Encoding | untrainable | trainable |
|---|---|---|
| FRQI | 1.34 | 7.78 |
| NEQR | 3.51 | 13.75 |
| Threshold | 1.9 | 8.19 |
| Encoding | untrainable | trainable |
|---|---|---|
| FRQI | 0.53 | 7.31 |
| NEQR | 1.55 | 12.3 |
| Threshold | 24.52 | 78.32 |
- Denny Mattern - denny.mattern@fokus.fraunhofer.de
- Darya Martyniuk - darya.martyniuk@fokus.fraunhofer.de
- Henri Willems - henri.willems@fokus.fraunhofer.de
- Fabian Bergmann - fabian.bergmann@fokus.fraunhofer.de
- Adrian Paschke - adrian.paschke@fokus.fraunhofer.de
Copyright 2021 Denny Mattern, Darya Martyniuk, Henri Willems, Fabian Bergmann and Adrian Paschke.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.