Comparing direct and indirect interaction in stroke rehabilitation

We explore the differences of direct (DI) vs. indirect (IDI) interaction in stroke rehabilitation. Direct interaction is when the patients move their arms in reaction to changes in the augmented physical environment; indirect interaction is when the patients move their arms in reaction to changes on a computer screen. We developed a rehabilitation game in both settings evaluated by a within-subject study with 10 patients with chronic stroke, aiming to answer 2 major questions: (i) do the game scores in either of the two interaction modes correlate with clinical assessment scores' and (ii) whether performance is different using direct versus indirect interaction in patients with stroke. Our experimental results confirm higher performance in use of DI over IDI. They also suggest better correlation of DI and clinical scores. Our study provides evidence for the benefits of direct interaction therapies vs. indirect computer-assisted therapies in stroke rehabilitation.


Introduction
Stroke is a leading cause of serious long-term disability in adults. More than 795,000 people in the United States suffer from a stroke each year [1]; this costs the country an estimated $38.6 billion that includes the cost of healthcare services and missed days of work. Computer-assisted technology has a key role in enhancing the traditional physical and occupational therapy, improving healthcare service, and decreasing the associated costs [2].

HCI Studies in Stroke Rehabilitation
Prior HCI research in stroke rehabilitation ranges from behavior change through design and persuasive technology to developing systems for compensation control and upper extremity rehabilitation. Balaam et al. [3] reported on their experiences with building systems that keep patients with stroke motivated to engage in upper limb rehabilitation exercise. Alankus et al. [4] focused on reducing compensatory motions which can hinder the recovery progress and cause new health issues for patients with stroke. Digital box and blocks [5] was built as an in-home assessment apparatus for individuals with stroke. This is an example of rehab games that leverage indirect interaction, i.e., patients need to manipulate the blocks in real-world while the effect of their action can be monitored indirectly in a screen. In another study, Alankus et al. [6] reflected on the lessons they learnt about what makes games useful from a therapeutic point of view including increasing social connectedness, connecting with family members and friends. Us'em [7] presented a watch-like device that provides feedback to patients regarding the usage of their impaired arm hand in relation to their non-affected upper extremity in order to motivate them to use their affected arm more. Comparing our work to the above research, we investigate which interaction technique is most appropriate within the practical domain of stroke rehabilitation.

Direct versus Indirect Interaction: HCI Studies
Direct and indirect interactions have been the subject of research interest in different domains of human computer interaction such as large displays, pen input, 2D-3D spaces, multi-display environments, etc. Here, we discuss a few examples from each domain.
One area of interest for the HCI community to explore the effect of these two interaction modalities has been large displays. Schmidt et al. [8] studied direct and indirect interaction in multi-touch input for large displays. They examined the two modes of interaction in terms of quantitative performance, qualitative observation, and user preference. The results indicated performance loss in indirect interaction due to "blindly keeping arms and hands at distance to the input device" [8]. Cheng et al. [9] developed a system to use an infrared laser pointer and an infrared tracking device to achieve a more direct interaction with large displays. Their main argument was that large scale display systems usually provide users with an indirect interaction which is in line with the use of conventional desktop-oriented devices to control the wall-sized display. However, they showed direct interaction with the laser pointer and infrared tracking device reduced

Work-in-Progress
CHI 2014, One of a CHInd, Toronto, ON, Canada "the cognitive load of the user and improved their mobility".
Another domain of interest is stylus input and tactile interfaces. For example, Forlines et al. [10] explored the effects of direct versus indirect pen input on pointing and crossing selection tasks. They investigated users' performance with pointing and crossing interfaces controlled via two input devices, i.e., when the pen-input and display are separate (indirect) and co-located (direct). They concluded that direct input significantly outperforms indirect input for crossing selection, but the two modalities are essentially equivalent in pointing selection.
As an example of a study in 2D-3D spaces, Knoedel et al. [11] investigated the impact of directness on users' performance for multi-touch RST (rotation, scaling, and translation) in 2D and 3D spaces. This study showed that direct-touch reduces completion times, but indirect interaction improves efficiency and precision specifically in 3D visualizations. The study also presented that users' trajectories in 2D/3D space with direct/indirect interaction are comparable which proves that indirect RST control may be of value for interactive visualization of 3D content.
In the domain of multi-display environments, the Ubiquitous Cursor system [12]  Although direct interaction has been generally preferred in the above domains, indirect interaction has variety of advantages depending on the application domain too. For example, Malik et al. [13] and Moscovich et al. [14] suggested indirect interaction can be of help in the following conditions: (i) when distant interaction is required; (ii) when multiple users need separate input interfaces; (iii) when avoiding occlusion is necessary; or (iv) when one surface serves as an input to multiple displays.
Although the above studies shed light on the usage of direct and indirect interaction based on specific domains, and provide strong hints for the benefits of direct interaction, stroke rehabilitation research lacks studies on usage of direct and indirect interaction modalities. Our paper is the first to provide results with patients with stroke.

Methodology
We developed a simple version of Fruit Ninja, a topranked game in iTunes and Google Play [15]. Our game includes repeated goal-directed wrist/hand reaching tasks which are similar to distal and proximal movements in Box and Blocks Test (BBT), a measure of gross manual dexterity often used as a post-stroke assessment [16]. Subjects held a cup-shaped colormarker in the paretic hand, then reached for a virtual fruit target that is sliced in two when the color marker overlapped the target. Then the next fruit target appeared in a different random corner, cueing the subject to reach for the next target. The game continued non-stop for 1 minute while the subject's score was displayed on the screen. The game's goal is to slice as many virtual fruits as possible within the specific amount of time (1 min). This game was  Figure 1 shows a patient playing the game in the two conditions.

Setup
There were two setups [17]- [19]: direct and indirect interaction settings. Both setups used the same table on which the subject played the game. A camera was used to capture the subjects' hand movements. The camera was connected to a conventional computer that processed the video feed and produced audio/visual feedback in real time. This setup has the potential to be used in clinical as well as home settings (as a telerehabilitation system).

1) Indirect interaction setup:
The subject looks at a computer monitor displaying the target fruits with a circle representing the subject's hand position while reaching for the fruits (Figure 2 (top)). A computer vision algorithm locates and tracks the subject's hand.
2) Direct interaction setup: Here, instead of having a monitor, we use a projector to superimpose the virtual fruits directly onto the tabletop. Same as IDI, the subject's hand movements are captured by a camera, while (s)he is looking directly at the table and interacting with the virtual objects projected onto it (Figure 2(bottom)).

Study
We conducted a within-subject study with 10 patients with chronic stroke, age = 59±10 year (mean±SD), were with prior acute of >6months. The patients underwent baseline assessments (arm motor Fugl-Meyer scale (FMA) 2 and Box and Blocks (BBT)). Our patients had FMA score = 57±11 (range 31-66), Hand/Wrist FMA subscore = 22±3 (range 15-24), and BBT score = 41±13 (range 16-58). The university's IRB approved the study procedures and all subjects were provided with informed consent before we briefed them on how to play the game. All patients played the game with their affected hand and had a warm-up run to learn it. Each patient was asked to play the game in both direct and indirect interaction settings. To cancel out the effect of order of playing, based on random choices, half of patients performed the game in direct interaction setting first while the rest performed the game in indirect interaction setting first. Performance of the patients in both settings was assessed based on the total number of sliced fruits in 1 minute. We repeated the same game in the same setting for 3 rounds. The independent variable of the study was interaction technique while other variables specifically the range of movements was maintained identical across the two settings.

Statistical Data Analysis
We measured the correlation of FMA and BBT scores with each of the two interaction (DI and IDI) scores of interest using Spearman's rank order correlation (using α=0.05). To compare performance in DI/IDI, a paired t-test was used because the data in both DI and IDI cases had normal distribution.

DI Scores IDI Scores
Work-in-Progress CHI 2014, One of a CHInd, Toronto, ON, Canada

Clinical correlation
DI and IDI scores were both correlated with the clinical scores of FMA Hand/Wrist and BBT (See Figure 4).

Discussion and Conclusion
This study shows promising results with use of direct interaction in patient-computer interfaces for stroke rehabilitation. The specific aims of the study were to investigate: (i) whether indirect and direct interaction scores correlated with clinical assessment scores; and (ii) whether performance is different using direct versus indirect interaction in patients with stroke. Our results suggest higher performance in use of DI over IDI. They also confirm better correlation of DI and clinical scores. These advantages may be due to visuospatial transformation in IDI which is eliminated in DI; DI is more similar to activities of daily living in terms of hand-eye coordination. A long-term study is being planned to investigate whether the cognitive differences measured in this pilot study translate into measurable benefits in stroke recovery.