BioDex is a species classification tool for Taxonomists & Collection workers developed by ETH Library Lab.
The project consists of three parts/repos
- mobile app for photographing specimens and viewing results
- prediction API for classifying images
- prediction model for training new models and data processing workflow
See more general info at biodex.ethz.ch/about/
- Project Overview
- Prediction API & Website
- Mobile App & API
- Image Repo (Docker Hub)
- Project Maintenance & Contribution
- Additional Info
This project is kindly hosted by ETH Library. Training new prediction models, improvements to the app or general any development work relies on contributions from contributors.
For general enquiries
contact.biodex@library.ethz.ch
questions about Entomology, use in collections or discussing research colloborations
michael.greeff@usys.ethz.ch
technical enquiries, prediction models, classification techniques etc.
barry.sunderland@librarylab.ethz.ch
if there is website or api services not available
contact.biodex@library.ethz.ch
For general bugs or requesting new features, please submit an issue to the relevant github repo, or better yet, submit a pull request.
Task | ETH Library | ETH EC | Community |
---|---|---|---|
Hosting | ✓ | ||
Deployment | ✓ | ||
Development | ✓ | ||
Approving Pull Requests | ✓ | ||
Prediction Model Training | ✓ | ||
User Account Administration | ✓ | ||
Replying to Contact Emails | ✓ |
<<<<<<< Updated upstream
BioDex is a species classification tool for Taxonomists & Collection workers developed by ETH Library Lab.
See more at biodex.ethz.ch/about/
- Project Overview
- Prediction API & Website
- Mobile App & API
- Image Repo (Docker Hub)
- Project Maintenance & Contribution
- Additional Info
- Taxonomy
- Podman
- Copyright
- Django Basic Queries
=======
Stashed changes
The Prediction API is built with Django REST Framework. It is used to:
- store image and taxonomic data for model training
- act as a gateway to the prediction model (by preprocessing posted images and postprocessing model response results)
- gather response metadata (e.g. example images)
- store data from prediction requests to be used for future analysis
The mobile API is not in this codebase, for a detailed readme see that repo.
data processing is performed in a seperate repo
model training is performed in a seperate repo
This Project is kindly being hosted ETH Library. Developing new features and bug fixing is open to the community. We use Github Issues to track requests and bugs. If you would like to contribute significant changes please get in contact.
This section includes some background information and useful commands for some of the frameworks used on this project.
species are named using the binomial format where the combination of the Genus and specific epithet define a species, e.g. Homo sapiens.
sp. is used when the genus can be identified but the exact species cannot or does not need to be determined. e.g. Homo sp. refers to some unidentified species of the genus Homo.
When using images from other people/collections/entities it is very important to ensure that we have permission to distribute the images.
The BioDex project is distributed under a creative commons licence; Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0) http://creativecommons.org/licenses/by-nc-sa/3.0/
open an interactive shell on a running container
podman exec -it <container-number> /bin/bash
save docker image as a tar file
docker save -o <path to generated file.tar> <image name>
Then copy your image to the host server with regular file transfer tools such as cp, scp.
Then load the image into Docker:
docker load -i <path to image tar file>
first login to dockerhub biodex in the cmd line
docker login
images can then be pushed or pulled from the docker hub repo for example;
docker push biodex/main:prediction_model__201911171137
note that there is only one private repo in the free tier so version tagging is added after double underscore instead of colon
generate new secret keys
from django.core.management.utils import get_random_secret_key
print(get_random_secret_key())
create an admin/superuser in the django app
podman exec -it WEB-CTR-NAME python manage.py createsuperuser
load fixture files using the convenience bash script
podman exec web sh load_fixtures.sh
connect to an interactive shell for the postgres database
podman exec DB-CTR-NAME psql --username=admin --dbname=biodex_dev
Uses Djoser
list of all available endpoints https://djoser.readthedocs.io/en/latest/getting_started.html#available-endpoints
token endpoints https://djoser.readthedocs.io/en/latest/token_endpoints.html
http://127.0.0.1:8000/api/auth/users/
http://127.0.0.1:8000/api/auth/token/login/
curl -X POST -d '{"username": "admin","password": "1234"}' -H 'Content-Type: application/json' http://127.0.0.1:8000/api/auth/token/login/
curl -X POST http://127.0.0.1:8000/api/predict/ -H 'Authorization: Token a21e26bd12a16542f940d641e840e32ad16a26d0' [{"id":1,"name":"admin"]
get the first 5 records from a model
qryset = Image.objects.all()[:5]
display the field names and values for those records
qryset.values()
look up fields in a related model/table (i.e. join) connect the field in the current model to the desired field in the other model
cls.values('species_key','image_key__image')