MIDI Hand Controller User Guide
The app uses real-time image detection and recognition to convert hand moves and gestures into MIDI messages.
It is like a contactless XY-pad – a device used in MIDI controllers for adjusting two parameters at the same time.
The app does not produce or manipulate sound directly. Its purpose is to control other software or hardware that understands MIDI.
Compatibility
External MIDI receivers
Wireless
The application works in two wireless modes: Wi-Fi and Bluetooth.
The app can connect via Bluetooth both as a peripheral and as a host.
In addition to physical wireless MIDI instruments, the application also works with Mac computers (and PCs, but only in the Wi-Fi mode).
This allows you to control the parameters of VST plugins installed in your DAW on the computer.
Wired
All USB-MIDI class-compliant devices connected to the iOS device (preferably via a powered USB hub) are visible to the application.
Audiobus
The application has an Audiobus port of the MIDI Sender type and is visible in the MIDI screen as INPUT.
Other iOS apps
The MIDI Hand controller works with all Core MIDI compatible applications, including GarageBand.
Setup
Connection to external controllers
Mac and PC
A detailed tutorial on how to establish MIDI connections (wired and wireless) with Mac and PC computers can be found here:
MIDI Controller - Connection Guide
In addition, it is worth reading the guides available on the Apple website:
Audio MIDI Setup User Guide: Bluetooth
Audio MIDI Setup User Guide: Wi-Fi
MIDI BT devices
For a list of available Bluetooth devices, see Settings → Wireless MIDI Connection → Bluetooth MIDI Devices.
To connect to a given device, just click on its name.
USB MIDI devices
All MIDI devices wired via USB do not require any additional configuration. They are visible in the application settings in the MIDI Outputs section, where they can be activated and deactivated.
Audiobus
In the MIDI screen:
- Add a new track
- Select MIDI Hand Controller as the INPUT (Apps tab)
- For the OUTPUT, select the virtual instrument you want to control – this can be an IAA app, an AU, or even an external MIDI instrument connected to an iOS device via USB or Bluetooth
After selecting a virtual instrument, it will automatically appear in the AUDIO section as the INPUT on a new track that you can additionally configure (add audio effects, etc.)
Other iOS apps
The app sends MIDI events via the virtual MIDI port. The list of active and inactive MIDI ports is available in the application settings in the MIDI Settings → Outputs section.
If other applications provide their own MIDI virtual port, it will also be visible here. To prevent sending duplicate events, avoid having more than one active port.
Background mode
The app will recognize your hand and your hand gestures only if it is in foreground mode, i.e., active and visible!
In the background mode, the camera will be automatically suspended, preventing the application from working properly.
For this reason, other music applications that are controlled by the MIDI Hand Controller should run in the background mode.
Parameters
- X/Y Axis
The following types of MIDI messages can be assigned independently to each axis:- Control Change (CC)
- Pitch Bend
- After-touch (Channel Pressure)
For each of them you can choose a channel number, a range and the axis direction.
In addition, the controller number can be selected for CC messages. - Closed hand mode
The application recognizes open/closed hand gestures. It can be used in two ways:- Lock Position Closing your hand freezes the point on the XY grid.
- Bypass effect Closing/opening your hand will send a single CC value, 0 or 127, respectively. The CC number is additionally configurable and should be different from that assigned to the X/Y axis.
Any hand movement with the palm closed does not change the XY value.
Thanks to this, you can lock the parameter values in a specific place and safely move your hand (also beyond the camera range) without causing audible changes.
This function is also useful if you want to incrementally change or assign a parameter value without its rise/fall.
The purpose of this additional event is to enable/disable the effect by appropriately assigning and mapping it in the controlled application, plug-in, or device.