Artif Life Robotics DOI 10.1007/s10015-014-0144-2

ORIGINAL ARTICLE

bioToys: biofeedback toys for playful and self-determined physiotherapeutic activities Tomoya Shimokakimoto • Asaki Miura Kenji Suzuki

•

Received: 20 December 2013 / Accepted: 15 April 2014 Ó ISAROB 2014

Abstract In the field of physiotherapy, physical therapists assist children with impaired motor functions or congenital loss of limbs to recover their motor functions or to adapt to the use of prostheses controlled by electromyography signals, respectively. However, children often quit training in the use of artificial limbs because it becomes boring with repetition. During a limited time, physical therapists are required to perform many operations on the biofeedback systems in order to personalize them for each user. It is important for children to feel excited about the therapeutic activities. Furthermore, the biofeedback systems should be easy for the therapists to use. In this paper, we propose a building-block-based biofeedback toy called ‘‘bioToys.’’ This system consists of input blocks to receive physical or physiological signals and output blocks to generate different action or effects. This building block system allows users including therapists, children, and their family to program and personalize the biofeedback systems. Finally, we show that by using the developed system, This work was presented in part at the 18th International Symposium on Artificial Life and Robotics,Daejeon, Korea, January 30–February 1, 2013. T. Shimokakimoto (&)
A. Miura Graduate School of System and Information Engineering, University of Tsukuba, Tennodai 1-1-1, Tsukuba 305-8573, Japan e-mail: [email protected] K. Suzuki Center for Cybernics Research, University of Tsukuba, 1-1-1 Tennodai, Tsukuba 305-8573, Japan e-mail: [email protected] K. Suzuki Japan Science and Technology Agency, 4-1-8 Honcho, Kawaguchi 332-0012, Japan

it is possible to control the toys and to record the muscle activities in real time. Keywords Playware
Biofeedback
Toy
Children
Education
Tangible interaction

1 Introduction Electromyography (EMG) biofeedback is one of the physiotherapy methods used for augmented motor function [1]. EMG biofeedback therapy is widely used for training for patients with neurotmesis, impaired motor function, or congenital loss of limbs. Therapists help patients to recover their motor functions, and several biofeedback devices have been proposed thus far to achieve this purpose, for example, by presenting a waveform on a LCD display (MyoBoy, Ottobock, Inc.). We have also reported several wearable devices for EMG biofeedback that convert EMG signals into sounds [2, 3] or displaying the shape and brightness of the activated muscles directly on the body skin [4]. Rehabilitation training generally consists of simple tasks such as picking up and moving cubes or tying a knot with a towel using the prosthesis; however, few adult patients endure this training because of their strong motivation to go back to their daily life. Similarly, children often refuse the training because it becomes boring after some repetitions or because they simply dislike the training or wearing a prosthesis. Further, Postema et al. [5] reported that children with a unilateral congenital arm defect reject the use of a prosthesis because they or their parents are depressed owing to the limited functional possibilities of the prosthesis. From a motivational viewpoint, Curtis et al. employed interactive games designed to address diverse handwriting abilities for the rehabilitation of stroke patients

123

Artif Life Robotics

cooperative tasks between the physical therapist and family members. Additionally, children show different medical conditions and distinct preferences and interests. Therefore, it is important to be flexible regarding training programs on site and to modify them to better fit the user performance. Finally, because therapy sessions are conducted over several weeks or months to be effective, the children have to be motivated to self-initiate playing and training.

Fig. 1 Concept of bioToys: playful and self-determined therapeutic activities

by using an interactive pen graphic display. They suggested that these games are more motivating than traditional approaches [6]. It is also important to take into account the fact that biofeedback systems require physical therapists to perform many operations including setting instruments, tuning parameters, and maintaining other assistive devices such as a prosthesis for each patient within a limited training time. Moreover, during training, therapists are required to observe the patient’s activity, check the output from the system, and instruct the patient about the next action to be performed all at the same time. In the case of child patients, therapists face the additional task of keeping them motivated with the therapeutic activities, because this represents an important factor that influences the therapeutic achievements. In this paper, we propose a novel block-type modular system called bioToys, as shown in Fig. 1. The concept is a personalized biofeedback device for playful and selfdetermined physiotherapy that encourages people to promote collaboration and competition with the therapist, families, friends, or others. The definition of this concept is based on four important characteristics: meaningful, accessible, flexible, and motivating. Meaningful refers to the user being able to understand the causality and contingency between the biosignals and the device’s output. Because biosignal measurement is very sensitive to noise generated by the body movements, it is important to process the biosignals in real-time to generate appropriate stable output. Examples of the definition of meaningful include a method for heart-beat pulse detection, estimation, and visualization for community support [7] and a method for the real-time conversion of EMG signals into sonification to understand the motion dynamics [8]. The other three characteristics—accessible, flexible, and motivating—are based on the ‘‘Social Playware’’ concept, which aims at flexible sensing and encouraging interaction among people by using devices to promote social interaction (see also [9]). From a clinical perspective, accessible means that a device should be easy to manipulate and should promote

123

2 Building blocks system Many studies have reported on the benefits of tangible user interfaces [10], which provide children with educational benefits. In particular, building blocks systems have been proposed as a tool for learning programming languages [11–13]. These building blocks system allow users to freely build a system, and this system will produce an outcome under the combination of properly built blocks with a particular function. We consider building blocks systems to be effective educational tools for children to learn about system procedures and to encourage them to learn in a voluntary and creative manner. Additionally, it appears that tangible interfaces can enhance upper limb rehabilitation training consisting of simple tasks such as picking up and moving cubes. In this study, we therefore developed a building blocks system called bioToys. Promising applications of this system include use as a creative tool for physiotherapy. This system is designed for users to understand the causality between biosignals and system output in an intuitive manner, and it can encourage children to self-initiate playing and training. Furthermore, it is easy for physical therapists, children, and their family to handle. Lund et al. also developed the block module [11] and tried to realize a learning method for languages or logical thoughts during playful experiences through block built structure analysis. While referring to their work, we integrated biofeedback of physiological information with the block-type interface for motivating cooperative actions and for realizing the playful effect for maintaining children’s motivation for rehabilitation. In particular, we emphasize on cooperation and creativity in play activities among children, their family, and therapists. 2.1 Device specification Figures 2 and 3 show an overview and the internal structure of the developed blocks. We implemented the system on DUPLOÒ (LEGO Group), and we constructed four types of block devices; link block, power block, processing block, and action block. These block-type devices are

Artif Life Robotics Fig. 2 Overview of the developed blocks

(a)

(b)

Table 1 Examples of types of blocks

Top cover

Category

Specific type

Functions

Link block

Normal bridge

Disconnection between blocks

Power block

Battery

Supply voltage to the blocks and monitoring power line

Processing block

Signal processing

AD conversion, processing of sensor signal and transmitting to received block

Wireless receiver

Receiving data from sensor and transmitting data to power line

Parameter tuner

AD conversion of a volume voltage and transmitting data to power line

Motor LEDs sound

Get data form power line and interpret the data then generate output for driving each element

Top connector Circuit board Bottom connector Bottom cover

Fig. 3 Internal structure of the block

modifications of the original blocks with two connectors. On the upper side, each connecting part has two electrodes. The inner and outer pins are used as power and ground lines, respectively. On the lower side, the connecting hole is used as the ground and the outer pin, as the power. Because these electrodes are designed like a coaxial socket, the devices shape is almost the same as that of the original block, and they can be easily connected not only to a modified block but also to regular DUPLO blocks. The modified block consists of 5 parts: top cover, top connector, circuit board, bottom connector, and bottom cover. These parts are designed to be able to fit in the top cover, and therefore, the modified blocks can connect to ordinary DUPLO blocks. Users can handle them without limitations regarding connection rules. In the proposed system, power is provided to each block by the master supply from the battery block. Therefore, recharging only the battery block will suffice for maintenance. To reduce the number of cables between the blocks and to increase the flexibility of the system, power line communication is used to share information between the blocks. Therefore, even if the developed blocks have a particular electrical connection and function, the physical connections between blocks are not limited. This mechanism allows the user to use and handle the developed blocks in the same way as normal building blocks.

Action block

Connection between blocks

Table 1 shows an example of block types and their functions. The developed blocks are classified into four different types. (1) The link block is used to connect or disconnect the circuit and to determine the physical shape. Normal blocks are used to create shapes and separate electrical connections between blocks. Bridge blocks are used not only to create the shape but also to provide electrical connectivity and data communication between blocks through the in-built circuit between the upper and the lower sides. (2) The power block has a unique block type that has a battery. This block provides power to other blocks via physical electrical connections and drives the other blocks’ circuit. (3) The processing block, which is equipped with an electronic circuit with a microcontroller (LPC1113 /302, NXP), has different functions. These blocks are designed to obtain and transmit data to other blocks. Wireless receiver blocks are implemented to obtain biosignals or physical motion data from the wearable sensor via Bluetooth connection. Because biosignals and human motions are measured on the body skin, each receiver block is paired with a wearable sensor. It then receives messages from a signal processing pair block and broadcasts them to other blocks in the network. The Parameter Tuner block has a gain adjuster for tuning output

123

Artif Life Robotics Fig. 4 Network structure of blocks

Battery Block

Bluetooth Block

Sensor Block

Fig. 5 Example of action blocks and block assembly

Measurement Flexor EMG

Bluetooth Block

Parameter Block

illumination Block

Motor Block

Sensor Block

Signal processing Low pass Filter

Train Control Integration

IEMG

Action1: only forward (using single signel) Action2: only forward (using both signels)

Extensor EMG

Low pass Filter

Integration

IEMG

Action3: Forward and backword

Fig. 6 Variations of the train action

data. (4) The action blocks interpret messages from other blocks and convert the signal to provide several actions such as motion, vibrations, sounds, and illuminations. 2.2 System configuration Figure 4 shows an example of a network structure between the connected blocks. Note that data

123

communication is performed solely thought the power line, and the network topology used a bus network. When the battery block is switched on, blocks connected to the power line start to work according to their functions. Any blocks connected to the network broadcast data and share circuit signals by serial communication. These data consist of a header, data length, a block type indicator, signal data value/command, and CRC. To avoid conflicts or

Artif Life Robotics

at a rate of around 100 Hz. On the other hand, for changing parameters such as the parameter tuner, a rate of around 10 Hz would be sufficient. Therefore, when action blocks obtain data from the power line, the specific type of data is checked and processed.

bandwidth crunch, signal processing blocks perform analog-to-digital conversion of data to produce meaningful and low-frequency data depending on the features of the measured signal before transmission. For example, EMG signals are obtained from electrodes attached to the human skin and sampled at a rate of around 1 kHz. The resulting integrated EMG (iEMG) signals are transmitted

2.3 Action blocks and block assembly We have developed a train-toy system based on a motor action block. Figures 5, 6, and 7 show the system overview and data flow. The train has two types of connectors at the front and back sides of the roof that are respectively, assigned to control the forward and backward motion. A set of a battery, wireless receiver, and parameter tuner block are assembled on the train on a different side of the roof, and the train will move forward or backward according to the built block configuration. In a simple case, the train moves forward when the data values are above the given threshold determined by the parameter tuner (Action 1). Then, when another block set is added, the train moves forward when either of the data values is above the threshold (Action 2) (see Fig. 6). On the other hand, when each set of blocks is assembled on

Back side Fr ont side

Fig. 7 Train system overview

Fig. 8 Experimental scenario: playing by using muscle activity and LED blocks: she moved her wrist for using the flexor muscle. Then the orange LED blocks were illuminated

Fig. 9 iEMG, threshold, and generated output waveforms

3

System output

IEMG

Threshold

2.5

Voltage[V]

2

1.5

1 Td 0.5

0

0

1

2

3

4

5

6

7

8

9

10

Time[s]

123

Artif Life Robotics

each side of the roof, the train moves forward or backward when either data value is above the threshold (Action 3). These biofeedback systems are designed to manipulate blocks by simply arranging and rearranging them without consideration of the block order with the aid of communication on the battery line.

3 Experiment 3.1 System evaluation Figure 8 shows a user playing with the proposed system. The participant used ordinary blocks, developed blocks and

a wearable device that measure the muscle activity and transmits data to a receiver block via Bluetooth, at the same time. She made a building block system that consists of a battery, a wireless receiver, and two LED blocks. After making this block system, as she moved her wrist, the LED blocks illuminated, and their intensity was dependent on the iEMG. It was confirmed that the user is able to use the developed blocks along with the traditional ones. To evaluate the performance of the proposed system, we conducted an evaluation experiment to verify the latency of the system. We prepared a basic system consisting of an EMG wearable sensor, a wireless receiver, a parameter tuner, and an action block. In this experiment, the biosignal data were transmitted at 1 kHz and the parameter tuner block transmitted data at 10 Hz. Fig. shows an experimental graph. The train is controlled according to the EMG signal measured at the participant’s elbow. We measured the system latency Td (s) 16 times; the averaged result was 0.454 s. This includes the time required for the wireless communication and the conversion for power line communication. We consider that this latency does not cause a problem in inferring the causality between the body actions and the train movement. 3.2 Case-study: therapeutic activities for children

Fig. 11 Flexor and extensor iEMG waveforms for a a healthy participant and b a child with congenital loss of the left arm

Voltage[V]

Fig. 10 Physiotherapy by using the train system

We used the train system with a child suffering from a congenital loss of the left arm. Children with forearm impaired motor functions or a congenital loss of limbs are

Time[s]

Voltage[V]

(a)

Time[s]

(b)

123

Artif Life Robotics

trained to generate proper EMG signals for the use of the prosthetic limb and hand. The participant is trained to use the flexor and extensor muscles for opening and closing the prosthetic hand. We show how such training can be achieved by using the proposed biofeedback system. Before this experiment, the participant was not in favor of wearing a cosmetic artificial hand and was uncooperative even for EMG measurements. We then introduced the prototype setup shown in Fig. 10, which shows the therapeutic activity using the train system. We observed that the participant was motivated and willing to play with the EMG-based train toy. The physical therapists that attended to the session switched the action mode and measured the EMG signals. The proposed system can not only control the train but also record muscle activities in real time. Figure 11 shows the flexor and extensor iEMG waveforms. We asked the participant to move the flexor and extensor muscles in turn when playing with the toys. The train system made the participant realize that he still could not control the two muscles separately. The system enabled the therapist to monitor the children’s physiological activity while playing with toys.

4 Conclusions In this paper, we proposed a unique biofeedback toy called bioToys based on normal and modified building blocks. In the field of physiotherapy for children, therapists are struggling to design therapeutic programs and methods according to the patient’s response. They encounter problem in that some children easily lose focus and interest in the training to use the prosthesis because it becomes boring with repetition, owing to the fact that the rehabilitation training contents comprise only simple tasks. We developed special building blocks embedded with a fully functioning electronic circuit that can nonetheless be used in the traditional way. We verified that the developed blocks could be used with traditional ones for biofeedback training that allows one to use biological or physical signals in the processing blocks and to generate various actions. We investigate how the developed bioToys could encourage children to construct various biofeedback training systems and to play with them. The system can be constructed by arranging and rearranging these building blocks easily in

real time, and quantitative measurements can be performed using different combinations of blocks. Future investigations will include the design of different types of blocks.

References 1. Wolf SL (1983) Electromyographic biofeedback applications to stroke patients. A critical review. Phys Ther. 63(9):1448–1459 2. Tsubouchi Y, Suzuki K (2010) Biotones: a wearable device for EMG auditory biofeedback. In: Proceedings of annual international conference of the IEEE EMBS, pp 6543–6546 3. Matsubara M, Kadone H, Iguchi M, Terasawa H, Suzuki K (2013) The effectiveness of auditory biofeedback on a tracking task for ankle joint movements in rehabilitation. In: Proceedings of the 4th interactive sonification, workshop (ISon2013), pp 1–6 4. Igarashi N, Suzuki K, Kawamoto H et al. (2010) Biolights: light emitting wear for visualizing lower-limb muscle activity. In: Proceedings of annual international conference of the IEEE EMBS, pp 6393–6396 5. Postema K, van der Donk V, van Limbeek J, Rijken RA, Poelma MJ (1999) Prosthesis rejection in children with a unilateral congenital arm defect. Clin Rehabil 13(3):243–249 6. Curtis J, Ruijs L, de Vries M, Winters R, Martens JB (2009) Rehabilitation of handwriting skills in stroke patients using interactive games: a pilot study. In: Proceedings of the 27th intl. conf. extended abstracts on human factors in, computing systems, pp 3931–3936 7. Shimokakimoto T, Ayuzawa S, Suzuki K (2013) Real-time pulse detection for physiotherapy and its application to wearable device (in Japanese). J Inf Process 54(4):1480–1488 8. Matsubara M, Terasawa H, Kadone H, Suzuki K, Makino S (2012) Sonification of muscular activity in human movements using the temporal patterns in EMG. In: Proceedings of annual summit and conference of APSIPA ASC, pp 1–5 9. Lund HH, Klitbo T, Jessen C (2005) Playware technology for physically activating play. Artif Life Robot J 9(4):165–174 10. Hiroshi I, Brygg U (1997) Tangible bits: towards seamless interfaces between people, bits and atoms. In: Proceedings of the ACM conference on human factors in, computing systems, pp 234–241 11. Lund HH (2003) Intelligent artefacts. In: Sugisaka M, Tanaka H (eds) Proceedings of 8th international symposium on artificial life and robotics, pp I11–I14 12. Horn MS, Jacob RJK (2007) Designing tangible programming languages for classroom use. In: Proceedings of the 1st intl. conf. on tangible and embedded, interaction, pp 159–162 13. Schweikardt E, Gross MD (2008) The robot is the program: interacting with roBlocks. In: Proceedings of the 2nd intl. conf. on tangible and embedded, interaction, pp 167–168

123