Look Ma, No Keyboard user interface: Tools for music pre-production

1. Look Ma, No Keyboard user interface: Tools for music pre-production Farhan Mohamed, Sidney Fels Human Communication Technologies Laboratory, UBC Current Sequencing Setup LMNKui Setup Piano Function Mappings “LMNKui System” An ergonomic method for controlling music sequencing software from a piano keyboard by adding a momentary foot switch. General Edit Octave (C6-B6) An octave mapped to handle general editing tasks. Mapping Example: F#6 Undo • Design • The design of the layout incorporates the windows and the functions most commonly used by the composer. The functions associated with each window map to a specific octave on the piano controller keyboard as shown in the Piano Function Mappings. • To distinguish between keystrokes that represent notes and those that represent macros, a momentary foot switch is used as a ‘mode switch’. • For continuous parameters, a single point sensitive touch pad is used in conjunction with the touch pad octave. Sequence Key Edit Octave (C5-B5) An octave used by the composer very widely for editing notes and their values. Mapping Example: What is music sequencing? A music pre-production practice involving recording, arranging and playing MIDI data. Sequencing programs allow composers to edit out mistakes, play music in various tempos, change keys without replaying parts, rearrange an entire song, draw volume graphs, and more. Touch Pad Interface Why cumbersome? The composer’s creative work is constantly interrupted by the time spent switching between multiple input devices and figuring out their functions. The current setup as shown above, also occupies excessive physical space. E5 Move to the previous note Note • Preliminary user studies • Two interaction methods were tested: The Computer Keyboard and The Piano Controller. • With an average learning time of approximately 5 minutes, subjects performed both tasks significantly better with the piano controller in Test1 (p<0.05, paired t-Test). • Although Test2 (p<0.06, paired t-Test) was done faster than Test1, the performance was not significant with the piano controller. This was due to the minimal learning time, and small number of subjects (6) used for our study. • Subjects found the piano interaction intuitive and an effective link between playing and editing. Arrange Octave (C4-B4) An octave used by the composer to arrange music by selecting and creating tracks. Mapping Example: Momentary foot switch CUBASE VST by Steinberg An industry standard music sequencing package used for our study. CUBASE allows the composer to perform complex pre-production tasks. Touch Pad Interface Tracks Parts C4 Create a new track • Commonly Performed Tasks • Recording MIDI events on selected channels • Playing back recorded channels • Spatial positioning • Effects and dynamic processing • Editing notes etc. Transport Octave (C3-B3) An octave used by the composer to play, stop, record, mute and solo a sequence or a specific MIDI channel. Mapping Example: Mean Time with 95% Confidence Error Bars • Current Developments • Applying the principles to other music instruments • Continuing user studies • Exploring musical structures for good mappings • Investigating different key boundaries C5 G5 G3 F6 C3 F4 E4 E2 G6 C2 Move to the next note Go to the previous part Move down the track Open/Close key edit window Paste Emulates the left button down of the mouse Forward Selecting volume Copy a sequence to the clipboard Open/Close transport window G3 Click On/Off (B3) Touch Pad Octave (C2-B2) A touch pad is used for constrained vertical or horizontal actions, for tasks like sliding a fader and drawing a modulation graph. Mapping Example: A volume graph for a selected channel References http://hct.ece.ubc.ca/research/lmnkui/ C5 B5 C4 B4 C3 B3 C6 B6 C2 B2 User Testing Key Maps Human Communication Technologies Laboratory The University of British Columbia Department of Electrical and Computer Engineering E2