Language, Cognition and Motor Development Through Video Game Play: What Are The Implications For Children With Disabilities?
Transmedia play lends itself to constructivist learning approaches (Herr-Stephenson, et al., 2013) that favor exploration, experimentation and the child as an active participant in creating knowledge (Piaget, 1985; Vygotsky, 1978).
Jenkins (Herr-Stephenson, Alper, Reilly & Jenkins, 2013) suggests that Sesame Street is an example of transmedia done right. The term transmedia means “across media” and encompasses the various types of media and their relationships to one another. Sesame Street uses transmedia to engage children through its television shows, full-length movies, plush toys, board games, live performances, books and interactive video games as well its rich history of research based pedagogy that provides both entertaining and effective learning experiences for children.
This game review seeks to demonstrate how the use of transmedia
and other learning strategies come together in Warner Bros. and Sesame Street Workshops’
Once Upon A Monster to provide young children educational video game play that also has promising implications for children with disabilities.
Once Upon A Monster is a single player video game developed by Tim Schaefer of Double Fine Productions and published by Warner Bros. Interactive Entertainment and the Sesame Street Workshop, in October of 2011.
The Sesame Street Workshop was founded by Joan Ganz Cooney and LLoyd Morrisett in the early 1970’s when these pioneers in technology and learning set out to create a television series that would both entertain and educate children. Sesame Street, revolutionary at the time of its inception, incorporated research related to how children learn as the foundation of its content. Many decades later this television program remains a preferred choice among young children and their parents as well as on the many platforms its content inspires: full-length movies, DVDs, books, video games and more.
Jenkins suggests that the, “… multi-modal, multi-sites nature of many transmedia productions challenge children to use varied textual, visual, and media literacy skills to decode and remix media elements
(Herr-Stephenson, et al., 2013 p. 1).”
Once Upon A Monster utilizes the Kinect controller created by Microsoft for the Xbox360 video game console. It is a motion sensing input device that allows the player to control the digital content through a “natural user interface” with the use of gestures and spoken commands. The Kinect makes use of an infrared projector and camera as well as a special microchip to track the movement of objects and individuals in three dimensions. The player is then allowed hands free control or interaction with the digital content.
This review provides observations of the video game and its affordances for children with special needs through my lens as a parent of a child with special needs as well as my experience as a speech-language pathologist.
Once Upon a Monster was designed for children ages 3-6 years. My daughter, Anna (not her real name) was eight years old at the time of this review. She presents with a severe language disorder, a coordination disorder, and difficulty sustaining attention. Although she was slightly older than the targeted audience, video game play in Once Upon A Monster was appropriate for her current language, cognitive and motor skills.
The Kinect controller recognizes the whole body movements of the player. Using these whole body functional movements such as running in place or jumping, gestures such as pointing and waving and speech, the player enters a simulated world that is animated and includes Sesame Street’s beloved friends- Elmo and Cookie Monster. The Kinect controller, along with other features of this game, allows the player to use their senses: visual, auditory and kinesthetic to immerse themselves in a virtual world. This experience leads to a sense of embodiment for the player; the player then feels and acts as if they are actually in this Sesame Street world and interacting with the characters themselves. Anna demonstrated evidence of this feeling of embodiment when she talked directly to the television, and the characters saying, “This is fun guys!” or “Wait for me!”
The absence of a hand held game controller, and the use of the players body in its place, made it easy for Anna to quickly learn how to operate the game. If the task required the character to jump, Anna jumped. If a task was completed and the digital storybook needed the page turned to begin a new task, Anna used the same gesture of the turning a page in a physical book.
Anna initially struggled with maintaining the appropriate distance away from the Kinect sensor. A square on the floor was created with masking tape that then helped her identify the optimal position for game play in front of the Kinect sensor and reduced her initial frustration with the game.
Once Upon A Monster, uses digital a storybook format to provide a narrative that imbeds video game play in the form of “mini games”. A new character named Marco needs help getting to his own birthday party. Cookie Monster, Elmo and Anna, luckily, were there to help him get there.
Each chapter of the game offers the player a task to complete that moves the characters closer and closer to the end goal, the birthday party. These tasks have educational underpinnings that engage the player to use their language, cognitive and motor skills.
Due to Sesame Street’s rich history of using researched based methods for providing content that is both entertaining and educational for young children, Once Upon a Monster is chalk full of sound pedagogical strategies that indeed accomplish this task. This video game is unique in its use of both transmedia and the Kinect sensor.
Does it then lend additional support or considerations for
children with disabilities?
An estimated 49 million children grades K-12 attend the U.S. public schools. Approximately 13% of this population, present with disabilities. Students with disabilities represent a heterogeneous population in terms of the disabilities they present with as well as the pedagogy used to address their individual needs and provide them with a Free and Appropriate Public Education (FAPE). Teachers face many challenges educating America’s children both with and without disabilities: shrinking budgets, reduced school calendars, changing standards, increasingly diverse populations and more. As globalization increases both the demand for and innovation of technology (Freedman, 2005), discussions and research on these technologies and their potential role in adding educational value for our children is increasing.
One way to consider the affordances that technology offers children with disabilities in the K-12 school setting, is the Universal Design for Learning framework
(Rose & Meyer, 2000).
This framework offers three principles for teacher instruction and student learning: multiple means of representation (of content), multiple means of expression (expressing knowledge) and multiple means of engagement. Looking at the video game Once Upon a Monster through this framework sheds light on its potential to provide a valuable educational experience for children with disabilities.
Transmedia done well, as Jenkins suggests (Herr-Stephenson, et al., 2013) offers children multiple points of entry. Once Upon a Monster offers children the opportunity to play with Elmo and Cookie Monster in a virtual world. But, children likely have significant experience with these characters long before they play this game through Sesame Street television viewing, its many published books, DVDs, full length movies and even toys or stuffed animals. It provides children multiple opportunities to learn both from and more importantly with its content. Children are not blank slates or empty vessels waiting to be filled with knowledge; instead they are active participants in creating knowledge and learning (Vygotsky, 1985). Anna immediately related to the characters and the virtual Sesame Street world because she entered the game play with existing background knowledge. She was also familiar with the script of a birthday party from her past experience both having parties of her own as well as attending the parties’ of others.
Westendorp, et al. (2002) conducted a study that compared 104 children with Learning Disabilities with 104 typically developing children to examine if there were specific relationships between two subsets of gross motor skills (locomotor skills and object-control skills) and different domains of academic performance. Results revealed a statistically significant relationship between gross motor skills (locomotor specifically) and reading; the poorer the reading scores the poorer the gross motor skills. This study and those like it lend support to the idea that the mind and the body have a reciprocal relationship and influence each other thus, embodied learning. Therefore, the embodied learning that is inherent in Once Upon A Monster provides students with disabilities multiple opportunities to work on the motor skills, language and cognitive skills, necessary for later academic success.
Embodied learning suggests that the physical aspects of our body shape or influences our cognition. Another way to think of it; our motor system influences our cognition in much the same ways that our mind influences our actions (Wilson, 2002). “All experiences are in some way grounded in the body…” and that, “…embodied experiences can lead to more effective learning (Smallab, 2012 para 1).”
Once Upon a Monster, with its use of the Kinect sensor, requires players to act out the specific action necessary to complete tasks in the game. If Marco needs to jump to get over the log, the player needs to jump. This offers the player an opportunity to imitate or initiate an action while also seeing and and hearing the results of that action, offering the redundancy necessary for accommodating new skills (Piaget, 1959).
Anna played the game for three hours, a testament to its ability to engage her. But, more important, was its ability to motivate her to engage in language, cognitive and learning tasks that were not initially easy for to complete. She was asked to follow novel complex directions such as, “Dress the monster in the outfit that matches the color of the flag it is holding”. She was asked to engage in complex motor tasks that required her to cross midline, use both of her hands simultaneously or engage in two actions simultaneously such as running in place while reaching to the left and the right to pick flowers for points. She was asked to do cognitive tasks such as sortingt items such as trash for the recycling bin or the landfill.
In my role as a therapist, I provide the scaffolding or support necessary for children to improve their existing skills or gain new ones.
Vygotsky (1985) refers to this as the Zone of Proximal Development (ZPD). Providing enough support so that the task is not so hard that it causes frustration or shut down, while also being cautious of providing too much support, so the task becomes too easy and merely practice. This requires a delicate balance to get it “just right” and ensure that the child is indeed learning from the experience.
My observations of Anna suggested that this video game was just right, if not slightly too difficult. A rule of thumb I commonly use to assess “just right” is often eighty percent accuracy on a task. Anna was somewhere closer to seventy, and she frequently expressed her frustration to the characters, “I am doing what you say!” or “Wait for me, I need to do it again”. But, the scaffolding and her engagement, unique to this video game play, kept her working on difficult tasks despite her frustration. Gee (2005) might suggest that Anna’s learning behavior in this game was characteristic of the learning that occurs in well-designed learning environments that share features of video games such as: the ability to customize the experience for the player (learner), the player’s ability to identity with the task, “pleasantly frustrating” experiences, and information that is presented "on demand” and “just in time”, among others. It could be argued that Once Upon a Monster was providing her with learning tasks within her ZPD.
Jenkins suggests that, “In a hunting society, children learn to play with bows and arrows. In an information society, they learn to play with information
(Herr-Stephenson, et al., 2013 p. 7).”
In a therapeutic or educational setting, I argue that Once Upon A Monster offers children with disabilities a quality educational environment built on Sesame Street’s use of transmedia play, its rich history of educational content and use of sound pedagogical strategies for teaching children and its, perhaps accidental, but effective, use of the principles of UDL.
Birchfield, D., & Johnson-Glenberg, M. (2010). A next gen interface for embodied learning: SMALLab and the geological layer cake. International Journal of Gaming and Computer-Mediated Simulations (IJGCMS), 2(1), 49-58.
Bushnell, E. W., & Boudreau, J. P. (1993). Motor development and the mind: The potential role of motor abilities as a determinant of aspects of perceptual development. Child development, 64(4), 1005-1021
Freidman, T. (2005). The world is flat. New York: Farrar, Straus and Giroux.
Gee, J., P. (2005). Learning by design: Good video games as learning machines. E-Learning and Digital Media, 2(1), 5-16.
Gee, J., P. (2013). The Anti-Education Era. New York: Macmillan.
Herr-Stephenson, B., Alper, M., Reilly, E. and Jenkins, H. (2013). T is for transmedia: Learning through trans- media play. Los Angeles and New York: USC Annenberg Innovation Lab and The Joan Ganz Cooney Center at Sesame Workshop. Available: http://www.annenber- glab.com/viewresearch/46
Piaget, L. (1985). Equilibrium of cognitive structures: The central problem with intellectual development. Chicago: University of Chicago Press.
Piaget, J. (1959). The language and thought of the child (Vol. 5). Psychology.
Gallese, V., & Sinigaglia, C. (2011). What is so special about embodied simulation? Trends in Cognitive Sciences. Retrieved July 18, 2013 from: http://www.unipr.it/arpa/mirror/pubs/pdffiles/Gallese/2011/tics_20111007.pdf.
Rose, D., & Meyer, A. (2000). Universal Design for Learning. Journal of Special Education Technology, 15(1), 67-70.
Smallab. (2012). Smallab Learning. Retrieved July 18, 2013 from: http://smallablearning.com/embodied_learning.
Vygotsky, L. S.(1978) Mind in society: The development of higher psychological processes. Cambridge, MA: Harvard University Press.
Westendorp, M., Hartman, E., Houwen, S., Smith, J., & Visscher, C. (2011). The relationship between gross motor skills and academic achievement in children with learning disabilities. Research in developmental disabilities, 32(6), 2773-2779.
Wilson, M. (2002). Six views of embodied cognition. Psychometric Bulletin & Review (9) 4.
This App challenges students to sort fruits and vegetables as they come down a conveyor belt that empties into baskets. Students can start out sorting just two different types of foods, but can LEVEL UP as their skills increase to sorting three or more foods. The fruits and veggies have scared or surprised facial expressions as they travel down the conveyer belt and change to happy when they are are pushed onto the correct conveyor belt and sorted into their basket.
This App offers opportunities for students to learn categorical vocabulary (fruits and vegetables), emotions (happy, scared, frightened, surprised), sorting (2+ items), skills for manipulating content on a touch screen/fine motor skills and would be a great activity for use with a stylus. This App lends itself to individual, small group and whole classroom play. *Remember, you can hook up the tablet to a projector if you have the correct dongle!
Age: 3-8 years (but I think it could appeal to older students as well)
Preliminary research and case studies are lending support for using the iPad3 with Retina Display with children with Cortical Visual Impairment (CVI). Many children with CVI are responding to the Retina Display (the screen's high resolution- roughly 300 pixels or more per inch) by visually attending to images or movement on the screen. Many of these children were reported to have not previously responded to the television or the computer screen.
The children with CVI who are responding visually to this technology suddenly have a platform for accessing and developing many skills. Children have been observed to engage with the content of specific Apps by using sight guided reach. It is also offering opportunities to improve fine motor skills as children isolate fingers for activating content and other movements such as swiping and pinching. And it goes without saying- the cognitive and language learning opportunities that visual attention and recognition afford these children is abounding.
The following offers a slide show of several apps to try with children with CVI. Several are free or offer a free light version and several offer alternative access and are switch accessible.
This set up includes an iPad3, two switches (the blue and red squares that are taped down!) and a Bluetooth Switch Interface (BSI) from R.J. Cooper.
The BSI allows my student with Cerebral Palsy (CP) to successfully access the iPad to communicate. This particular student also has Cortical Visual Impairment (CVI) and appears to be benefiting from the Retina Display and Picture Communication Symbols (PCS) that are modified for low-vision (his visually guided reach to activate the touch screen has improved!). His communication is further supported by the GoTalk App that allows for two switch scanning. It is also set to provide auditory preview of each message and the communication program is a modified Pragmatically Organized Dynamic Display (PODD).
He is rock'in it!
Eric Rosenbaum and Jay Silver have created this clever tool called the Makey Makey. For those of us who work in Special Education with students who need alternative access, it is essentially a low cost, super flexible switch or series of switches. The creators use the Makey Makey to turn common objects such as bananas into musical instruments. The bananas can be programmed to play music when activated by a simple touch. A fan of the Makey Makey even goes as far as turning a row of labrador retrievers into a musical instrument!
The Makey Makey wasn't necessarily created as an Assistive Technology tool, but it encourages imagination, play and access, promising to create options for students with alternative assess needs. I know I will be using mine to assist students to increase independence in controlling/accessing the computer, participating in art and music activities, creating alternative video game controllers and more. But, the most exciting possibility, is what the students decide to do with it!
Check out their website here! Pictures on this post came from this site.
A student of mine recently donated a Beamz to his classroom. This innovative technology connects to a PC and allows students to create music. Students "break" the lazers to create the music. My students were eager to engage with the device, enjoyed listening to their peers creations and even created music together. I found this piece of "off-the-shelf" technology to be highly adaptable for students with disabilities.
This video provides a brief and compelling argument for both using and teaching about technology in the k-12 and higher ed settings.
This site: bagtheweb, helps you curate resources from the web . I used this resource to curate various AAC resources for the SLPs and teachers I work with. I also created the following "How-To" video. Bag the Web offeres a flexible tool and a site full of resources for the k-12 professional.
Robin Shobe is a Speech-Language Pathologist and Assistive Technology Specialist (AT) with a passion for technology and its uses in the k-12 setting.