MIDI Improvements

[alcomposer]

MIDI implementation in Godot improvement ideas.

Prior discussions:
Audio Improvements, DSP software
Add an absolute time (DSP time)
Add support for MIDI output

MIDI is an important feature. It is an old, yet universally supported standard.
It is trivial for users to plug in any MIDI device, or setup a virtual MIDI software port.
It represents a simple, and effective way to control Godot, or in the case of output, to output to MIDI aware software / sound modules.

It's important to understand that Godot is itself a development platform, and thus should have basic, streamlined MIDI support. This isn't about making Godot into a DAW, MIDI support should be there to allow easy integration & extension however the end developer chooses, and give them a great developer experience.

Use cases:

MIDI IN:

• MIDI instrument tutorial game (eg: playing piano keyboard, and being scored on how accurate the player is)
• MIDI controlled level design
• MIDI controlled app modes (click midi buttons to debug, go to levels etc)
• Motion graphics / VJ / Realtime control
• Render realtime graphics control (video recorder)

MIDI OUT:

• play game music externally (as many classic games allowed)
• control other MIDI softwares (in whatever way the game developer wants)
• control logic for MIDI interfaces (lights switch on / off dependent on what button / logic is active)

To implement

• Print human readable MIDI event output (Note On/Off style events) #68820
• Move from ALSA raw -> seq seq wip
  seq allows us to:

• send data directly to Godot MIDI driver (instead of recreating it)
• share MIDI devices with other apps instead of binding them
• send / receive MIDI from other apps, via direct ports (app -> Godot midi port), ALSA MIDI-Through, or patching.

• MIDI output (direct, and timestamped) Could be possible without API breakage.
• Name MIDI client with project name, not Godot.
• Add "id" InputEventMIDI member variable to allow filtering of events from multiple devices (if event.id == 0)
• Musical note name enums?

Possible API breaking changes

• Move MIDI initiation to project setting: Audio->MIDI->Enabled [On | Off]
  This would give Godot a client pointer to attach in/out ports to later. (coreMIDI & ALSA Seq function similar, while Windows can skip this step?)
• Don't automatically connect to all MIDI inputs. Have another method: for connect_all(), connect(port) & disconnect(port).
• Naming cleanup (see below table)
• Deal with sysex
• Pitch-bend as unique variable (Currently Godot uses pitch for both MIDI Note Number & Pitch Bend)
• hot-plug MIDI driver emit signal
• move MIDI enums to MIDI namespace, eg: MIDI::MIDI_MESSAGE_NOTE_OFF

InputEventMIDI Naming Conventions

current	proposed	notes
n/a	id	connected device number
channel	channel
message	message
pitch	note_number
n/a	pitch_bend	currently pitch-bend uses pitch variable
velocity	velocity
instrument	instrument
pressure	pressure
controller_number	cc_number	controller is used internally, consider if we should be consistent
controller_value	cc_value

We may also want to consider initialising each variable in InputEventMIDI to -1 as default. So that it's clear when a message has altered the value, as each variable can be set to 0 via MIDI.

[oeleo1]

Okay, but at first read, this is too technical and detailed. Could you summarize what are the current drawbacks of Godot MIDI support and what your proposal is to remedy these drawbacks. Also, highlihting a typical use case would be appreciated.

[alcomposer]

From looking at the code, and using MIDI in Godot I formulated this list.

I have been using Godot for realtime installations. To do this, I have been controlling Godot via an Akai MIDIMIX. This controller has buttons (MIDI NOTEON/OFF) and sliders, (MIDI CC).

This sort of controller is typical of how a user would interact with Godot in a control scenario. There are thousands of such generic controllers available, and quite inexpensive (many the same price as a computer keyboard).

With such a MIDI controller, I can program any slider/knob/button to trigger anything I wish in Godot. Such as moving objects around, changing scenes, controlling shader parameters, changing opacity, triggering events etc.

Thats my current use case, however MIDI control can be anything one needs, and a flexible system should allow the developer to achieve what they want to, and have a good experience in doing so.

This list is intended to allow discussion of "what, and how" to improve.

Current drawbacks are discussed in the document. Such as:

• Both MIDI pitch-bend & MIDI note number using pitch variable. [inconsistency]
• No timestamp in ALSA raw [missing feature necessary for future timestamp alignment]
• Inconsistent naming conventions
• No ability to filter based on where a MIDI note came from (which device) [QOL improvement]
• No MIDI output support

So instead of me writing "please fix MIDI" post, I think a detailed technical discussion could be helpful, as I would like to make sure how the changes happen are acceptable.

I am also interested in having a larger discussion about how MIDI out should be implemented.

From a GDScript POV, I would think making a new MIDIoutput() would work by overloading the constructor as either a port number, or string (in which case the string would be a named virtual port:

var midi_out_device_0 = MIDIOut.new(0) 
var midi_out_virtual = MIDIOut.new("virtual port A") // other software will see "virtual port A" as Godot output

func _process(delta):
    midi_out_device_0.channel(0).note_on(44,127)  // channel, note_number, velocity
    midi_out_virtual.channel(0).note_on(22,66)

[oeleo1]

Thank you fo this explanation. This sounds terrific since it is proof that Godot enters and interests the large music authoring world. It sounds like you are using Godot for real-time music visualisations as a reactor to MIDI events. And that the status quo doesn’t allow you to perform a decent mapping of these events (or input signals) to Godot’s objects, mainly due to lack of more extensive input event/signal support. Well, this may very well be the case, indeed. I presume it is the case simply due to the lack of knowledgeable and interested contributors, Godot limiting itself to a basic interface of the midi driver/lib of the day. I am unfortunately not knowledgeable enough in this area, but I fully support the idea and would welcome to see experts in the area joining the discussion to bring it to the next level!

[alcomposer]

MIDI in is already quite good in Godot, much more fleshed out than it is possibly given credit for.

These are only small things I have outlined. The largest being moving to ALSA seq.

MIDI out is already a concept that has been discussed, but I get the feeling it will be much more painless with ALSA seq, which gives us timestamp & direct output, along with virtual MIDI ports, routing capabilities and non-exclusive access.