NornsVAE is a machine-learning-powered drum sequencer for the Norns platform. It uses Google's MusicVAE model to generate drum sequences. Because of the limited computing power on a Rasberry Pi, computations are performed by a separate Python server, which needs to be independently run on a computer in the same network.
It is being developed as part of my Master's thesis in computer science, which aims to research interactive music generation with deep learning.
The project consists of two parts:
- a client, written as a Norns script in Lua
- a server, implemented as a Python server (Flask) that wraps a MusicVAE (Tensorflow) instance
There are two options for running server:
- Pre-built distributable (Windows)
- Running from source (Mac/Windows)
If successfully started, the console will display:
* Running on http://<ip address>:5000/ (Press CTRL+C to quit)
Sadly, installing the server requires a large number of dependencies (in particular Tensorflow).
Download the distributables here and run server.exe
.
To uninstall, simply delete the folder; nothing is installed locally.
On Mac, you'll need to run the server from source. You'll need Python and Miniforge.
-
Download Miniforge.
If you have an M1 Mac, you'll need the arm64 (Apple Silicon) version. Download the script here and install it:
bash Miniforge3-MacOSX-arm64.sh
If you have an Intel Mac, you'll need the x86_64 version. Download the script here and install it:
bash Miniforge3-MacOSX-arm64.sh
-
Once this is complete, you can set up a new Conda environment where we will install the required packages:
conda create --name nornsvae python=3.8
Now you can use (activate) this environment:
conda activate nornsvae
The command line should now display
(nornsvae)
. -
Install Tensorflow inside the new environment:
conda install -c apple tensorflow-deps
If you have an M1 Mac, install
tensorflow-macos
:pip install tensorflow-macos
If you have an Intel Mac, install
tensorflow
:pip install tensorflow
-
Download the NornsVAE Server here and unzip it. Run the following commands from within the
server
folder:pip install -r requirements.txt
Make sure that you are in the NornsVAE environment:
conda activate nornsvae
If you have trouble installing the requirements, see the Troubleshooting Guide.
If you wish to uninstall, delete the nornsvae
environment with conda env remove --name nornsvae
or uninstall Miniforge altogether.
Once you have installed all dependencies, simply navigate to the server
folder and run the server using python server.py
. Make sure that you are in the NornsVAE environment: conda activate nornsvae
The terminal will display the local IP address of your computer, if everything works.
- From maiden, type
;install https://github.com/jacbz/nornsvae_client
- Write the IP address of the server into the file
server-ip
, located inside the script folder - You can now start using the script!