Playing an instrument is a lot of fun, but what if you could have... MORPHFUN!
MorphFun is a Timbre Transfer application, that relies on a Pose Estimation approach to let the user choose the instrument. The core libraries used are:
- magenta/ddsp and its Timbre Transfer demo module.
- Mediapipe, for the Pose Estimation part.
- PyQT5 for the Graphical User Interface.
The entire project is developed in Python and Jupyter Notebook.
Here, you can find the presentation of the prototype, developed between Python and Processing. If you want to take a look at the code see this branch.
Here, instead in the main branch you can take a look to the final code of the application.
Apart from the main and the Prototype branches, already introduced, this Repo has two more branches: Develop and gui-develop. Both are dedicated to the project development.
The code is in the Source folder. The main modules are:
- main.py, which creates the application, as well as the main thread and data queues.
- controller.py, which orchestrate the behaviour of all the modules and the interaction between the GUI and the backend.
- gui.py contains the structure of the application GUI.
- audio_utilities, which manage the audio acquisition and the reproduction of the morphed audios.
- ddsp_functions, which contains the functions used to morph the recorded audio with magenta/ddsp.
- pose_estimation, which contains all the necessary modules for the pose_estimation part.
- CONFIG, images, utils.py and models allow to build the application and the pipeline with all the correct configurations.
To test this project, clone the Repo and install the libraries listed in requirements.txt in a virtual environment of your choice.
| Name | Contact |
|---|---|
| Armando Boemio | armando.boemio@mail.polimi.it |
| Lorenzo Brugioni | lorenzo.brugioni@mail.polimi.it |
| Gabriele Maucione | gabriele.maucione@mail.polimi.it |
| Paolo Sani | paolo1.sani@mail.polimi.it |