Research: Physical & Virtual Keyboard Input Using a Head-worn Display

The Virtual Console research project introduced the notion of wearable displays to replace large, heavy, expensive, workstations onboard airborne platforms. Adding head-mounted displays, which obscured some or all of the users external vision, introduced new challenges around standard input methods. This study evaluated traditional and non-traditional input methods while using a prototype version of the Virtual Console hardware and software.

Results of this study were presented on 2010/09/10.

This is a summary of our methodology. Full details of the final report are unavailable pursuant to distribution restrictions.

My Role

As a primary researcher I co-authored the project test plan, as well as conducted multiple of the testing seasons. Additional seasons were conducted by the fellow researcher. Following the completion of the testing, I collated all data a wrote the final test report.

Summary

Participants were given multiple short phrase (e.g., “watch out for low objects”) and alpha-numeric codes (e.g., “123fd”) input tasks while wearing a head worn display (HWD) similar to that pictured below. The six different input methods tested were:

  1. Control: using a standard keyboard and monitor
  2. Peek: See the physical keyboard by peeking underneath the HWD
  3. Voice Recognition: Speak aloud inputs to the Windows 7 Voice Recognition software
  4. WebCam: Video of the keyboard is displayed inside the HWD
  5. On-Screen Soft-Keyboard: Use a mouse to type with the Windows Software Keyboard app
  6. Touch Type: The physical keyboard cannot be seen.

The evaluation used standard Windows software, as opposed to the Virtual Console prototypes, in order to minimize development work and focus attention of participants towards the input method and not the prototype system.

Objective

The Virtual Console’s unique visual interface creates new challenges and new opportunities for inputting text and system commands. This test was designed to evaluate the feasibility of six different input methods while wearing a head worn display similar to that that might be used with the Virtual Console.

Design

The design of the test focused on two independent variables:

  1. Input Method
    • Control: No HWD, standard keyboard and monitor input
    • Peek: Wearing HWD, but can “peek” under bottom of the HWD to see standard keyboard
    • Voice Recognition: Wearing HWD, use Windows voice recognition software with the standard keyboard removed
    • WebCam: Wearing HWD, a window displays a video of the standard keyboard positioned at a comfortable typing position
    • On-Screen Soft-Keyboard: Wearing HWD, use mouse to type with the Windows software keyboard
    • Touch Type: Wearing HWD, use standard keyboard with no visual line-of-sight
  2. Input Task
    • Mixed Alpha-Numeric: Five character string of the format: NNNaa (e.g., “123ge”) mimicking track numbers
    • Short Phrases: Multi word phrases intended to mimic instant message chat.

Metrics were collected both automatically and through subjective interviews after each test. Objective metrics automatically collected included: task completion time, error rate and characters per second. Subjective metrics collected after the test included information such as perceived effectiveness and comfort of each input method.

Method

Participants were provided with written instructions for the overall evaluation and for each specific input method. All participants performed the control condition first with the remaining conditions randomized. The tasks within each condition were administered in a fixed order, consisting of 1) Short Phrases and 2) Mixed Alpha-Numeric codes.

Participants were allowed to practice with each input methods until they felt comfortable. Because participants were unlikely to be familiar with the voice recognition and on-screen keyboard input methods they were given additional instruction and practice time in these conditions.

Quantitative metrics collected during each test included task completion time, characters per second, and error rates. Following the completion of all input methods each participant was given a questionnaire consisting of a Likert scale to gather the perceived effectiveness and comfort of each input method.