overview
For people with motor impairments like arthritis or repetitive strain injuries, traditional keyboards are often painful and difficult to use.
That's the problem our team set out to solve in the Graduate Level Interaction Techniques course. We created PianoProse, a new way to type using a piano keyboard that reduces physical strain while making typing more enjoyable.
the problem
Traditional computer keyboards require precise finger movements and hand positions that don't consider the needs of those with motor impairments.
Precise Movements
Small keys require exact finger placement
Hand Strain
Awkward positions cause discomfort
Accessibility Gap
Not designed for motor impairments
We saw a clear opportunity: a typing method that feels natural, reduces strain, and works with the body rather than against it.
research
Understanding Our Target Audience
2 main requirements for our new typing system:
- Stable, guided movements that help reduce errors from involuntary motions like tremors
- Consistency across different devices, since they often switch between computers to manage fatigue
Why a Piano Keyboard?
Playing piano can actually help people with hand dexterity issues. Music therapists have documented how arthritis patients improve their hand motion and finger strength through regular practice. The larger surface area of piano keys also provides more comfortable hand positioning compared to small QWERTY keys.
Exploring Existing Solutions
Traditional QWERTY keyboards clearly cause tendon strain, so we knew to avoid that approach. The Dvorak keyboard's ergonomic layout was the most promising.
Traditional QWERTY Layout
Dvorak Ergonomic Layout
Outreach Challenges
Reached out to professors and accessibility organizations to find participants with motor impairments. Wanted to understand their daily challenges with keyboards and get their thoughts on piano-based input.
Unfortunately, we couldn't connect with our target users within the project timeline. So we adapted our approach to test with able-bodied users while keeping accessibility needs in mind.
Initial attempt at finding participants
design process & considerations
Main design goal: minimize the distance fingers need to travel and reduce strain on joints.
My team and I deliberating on key design
Key Layout & Letter Mapping
Letter frequency data guided layout decisions. The most common letters (E, T, A, O, I) went on the central white keys for easy access, while less common letters like Q, X, and Z were placed on the outer edges. We also grouped common letter pairs like "th," "in," and "er" close together to make typing flow more naturally.
Progress in key mapping
Punctuation, Space & Backspace
The principle of proximity organized the important keys. Space and backspace went on the far left since they're used so frequently (Space accounts for over 10% of all keystrokes!) Commas and periods, the most common punctuation marks, were placed centrally for easy access.
Shift & Caps Lock
A one-handed Shift chord was designed using 3 spaced white keys. This reduces strain by avoiding unnecessary hand movement. To activate, press the thumb, middle, and pinky fingers simultaneously.
For Caps Lock, we used the same chord pattern but required both hands since it's used less frequently. This kept our gesture design consistent and predictable.
Shift and Caps Lock design
Feedback Design
Audio: We decided to disable keyboard sounds to avoid distracting discordant notes during typing.
Visual: We considered adding predictive lighting to guide users to the right keys, but we kept it simple with clear labels due to time constraints.
Haptic: The piano keys themselves provide excellent tactile feedback, which research shows improves typing speed and accuracy. This natural feedback helps users understand how the system works.
prototyping & testing
Our Final Setup
After several iterations, we settled on a keyboard layout where space and backspace keys went on the far left for easy access. We grouped all the vowels together in the center row, and placed less common letters like m, q, v, and z on the outer edges.
Final Keyboard Layout
To help users find the right keys, we added labeled circular stickers to each piano key. This made the layout much more discoverable, similar to how traditional keyboards have labels.
For testing, we used a Korg Minilogue XD synthesizer connected to a MacBook through Chrome's built-in MIDI support.
Participant testing prototype
Our engineering lead, Ember, created a custom typing test website at piano-to-text.vercel.app using React.js. This test could receive input directly from the piano keyboard and measure:
- How many words were typed correctly vs. incorrectly
- Typing speed in words per minute
Solution Demo
outcomes
We tested our prototype on 3 piano-novice participants. Each participant completed 5 typing tests, followed by a short interview.
🧠 Learnability
All participants improved by the 5th test. Participants found space and backspace keys intuitive and easy to remember.
🎯 Accuracy
Most errors were punctuation-related. No errors occurred when using the backspace key, showing correction was intuitive.
✋ Comfort & Experience
2/3 participants defaulted to hunt-and-peck positioning, prioritizing familiarity over ergonomic hand positions.
Key Takeaway
- While PianoProse initially felt awkward, it showed strong potential to become an enjoyable, learnable, and comfortable typing method with practice
what's next
Immediate Next Steps
Plan to test with participants who actually have hand dexterity concerns in order to refine the design.
Future Updates
- Shortcut chords for common words and letter combinations
- Support for all special characters from traditional keyboards
- Cross-application compatibility (like Microsoft Word)
credits
Ember Shan — Master of Human-Computer Interaction (MHCI)
Jackie Strawbridge — Master of Human-Computer Interaction (MHCI)
Zoe Mercado — Master of Human-Computer Interaction (MHCI)
Dr. Brad A. Myers — Professor of Interaction Techniques
