The Possibilities of DIY-Assistive Technology for Learning and Therapy


In the last few years, I have become very interested in the possibilities of applying Maker, Hacker and DIY methods and practices to assistive technology and digital art therapy. To understand and investigate the possibilities of these approaches my supervisor, Melanie Baljko, and myself, decided to propose, design and conduct a series of Maker workshops (Collaborative MakerShops) at our lab, the GaMaY Lab, at the Department of Electrical Engineering and Computer Science at York University, Toronto.

In the makeshops, we explored topics ranging from 3D modeling and printing to open-source embedded hardware and wearable technology. As part of the workshops, we organized a fieldtrip to the local Maker Faire, the Toronto mini Maker Faire, held in November 2013. At Maker Faire, I met Ray Feraday, a special education teacher working at the Toronto Catholic District School Board. In addition to being a teacher for many years, Ray also has previous research and design experience.

The project that Ray was showcasing at Maker Faire involved the innovative use of the Makey Makey kit to create an alternative communication board for one of his students who had difficulty using conventional devices. Ray had embedded the Makey Makey into a costume chassis made out of foam core and conductive tape, and had indicated the choices with printed photographs and symbols. I felt his invention was an excellent use of the maker approach to design new assistive technology. Ray’s project had several elements that were characteristic of maker methods: he had identified a design possibility that was relevant to his student (and himself by proxy), had combined innovative open source prototyping hardware (i.e., Makey Makey) with low-fi craft and hardware material (i.e., foam core and conductive tape) in a new context (assistive technology in the classroom), and had used online resources (i.e., YouTube videos, Instructables tutorials, … ) and help from techy friends and family to fabricate his design.

Screen Shot 2014-08-19 at 12.31.31 PM

Shortly after the Faire, Ray and I met and discussed possibilities for collaboration. Ray had since made improvements to his initial designs and wanted to replace the Makey Makey with a Raspberry Pi and make the unit compact and mobile and also bring down its cost. This task required programming and electrical engineering skills. At this time, Toni Kunic, a second year Electrical Engineering student with excellent programming and software design skills, who had participated in the Makeshops expressed interest in getting involved in a project that required more hands-on design and implementation tasks and went beyond the workshops. This was an excellent opportunity for collaboration and our small team, consisting of Ray, Toni, Melanie and myself started meeting every weekend to work on a DIY alternative communication device that came to be known as TalkBox.


We have described the design of TalkBox in detail elsewhere. Here, we will briefly describe some of the aspects, specifically with respect to deployment, that we have not discussed previously. icchp3

From the very beginning, TalkBox was designed to be accessible, affordable and makeable by non-professionals. We were committed to make the design open and available to people who needed it. To this end several documents were made available online ranging from descriptions of how to make your own TalkBox to the software image of the program. We are currently planning to make detailed picture instructions on how to fabricate and make your own TalkBox. Our hope is that someday, children with disabilities themselves can assemble TalkBoxes (for themselves or others).

Making TalkBox affordable and open-source allowed us to explore new alternative deployment models (even at this early stage). We met with the Tetra Society of North America, a non-profit volunteer organization providing assistive technology for users with disabilities with 45 chapters across North America, and they agreed to support us in the production of a new series of TalkBoxes. We also have plans for a workshop on DIY Assistive Technology at the 2014 Toronto Mini Maker Faire where we aim to teach how to make TalkBoxes. These plans not only provide us with an opportunity to develop and evaluate a set of instructions on how to assemble and make TalkBoxes but also a chance to be connected to a community of users who will provide us with a wealth of data and feedback that can be used to improve and refine our design and design process.

Scratch in the Classroom

Before collaborating with our lab, Ray had experimented with the Scratch visual programming language. Scratch, developed at the Lifelong Kindergarten research group at MIT, is designed for use by children and youth (between 7-18) and is friendly, easy to use and surprisingly versatile. While Ray had not programmed before, he found Scratch intuitive and felt perhaps even some of his students with severe autism and other disabilities can use it to play and design games.

Ray’s sister and brother-in-law are both game designers and last year had visited the school to demo a multiplayer arcade game they had made. To everybody’s surprise, several of the special needs children who usually have a hard time staying concentrated and quiet for extended lengths of time had stayed throughout the session and one was so good at the game that he had beat everyone at school including the game’s authors! Read more about this event here.image copy

Observing this particular student who in addition to cognitive disabilities, also has some behavioral disorders, inspired Ray to try some sample Scratch exercises with him. He setup a Raspberry Pi computer and a Scratch video tutorial that described how to make the game of Pong. Astonishingly, the student followed the tutorial to the end and implemented the game. This is a remarkable result considering that he was able to do this task in one sitting and without help (beyond the setup of the video and computer).

Following this exercise, Ray, experimented with the Hour of Code tutorials. These are a series of Scratch tutorials offered by the foundation and designed to teach children basic programming concepts, such as conditional structures and loops, using popular game design with Scratch. Again, the student followed the instructions and finished the exercises, getting an Hour of Code certificate at the end.

At this point, with summer approaching, Ray decided to send in a proposal to teach a coding to children with disabilities through a summer school (MEDD/AU). This gave him a chance to try his hypothesis that children with various cognitive disabilities can learn to code with 75 students. The students had a range of special needs but most were verbal and could work at a ratio of 4 students to 1 adult. Many are on the autism spectrum and some show signs of OCD or hyperactivity. Only two of the students had previous (minimal) experience with programming. They were divided into groups of 15, with each group receiving a 45-minute session of instruction each day. The summer school ran for 4 weeks but there were only 14 actual teaching days.

Ray started the classes with 3 days of Computer Science Unplugged exercises. These mainly involved doing pixel art where images were created by the kids by colouring in spaces in checkered paper and were meant to warm up the kids (and their Educational Assistants) and also give them a flavour of how computers store graphical data.

After this phase, Ray started teaching programming using the Hour of Code maze activity. These exercises use a visual programming language called Blockly, which is very similar to Scratch and, in a very playful manner, uses blocks and visual elements to teach programming concepts. In the maze activity, the programming commands control the movement of popular characters from the Angry Birds and Plants and Zombie games and teach concepts such as control blocks, condition testing and loops. Also, for the particular group that Ray was teaching, the exercises were used to convey more specific concepts such as counting, distinguishing left from right, and simple instruction sequencing.

Ray used a Smart Board to illustrate and teach the simple dynamics of the game to the students as a group and then had them work on individual computers on the same exercises. While the students had varying degrees of special needs, they all participated in the exercises.


The Smart Board was useful as it allowed the students to collaborate with Ray and each other and write a program. While the touch interface was easy to use for some of the students, Ray used a stick for ones who had trouble reaching it (e.g., because of being in a wheelchair) or could not control their hand movements precisely or touched other parts of the Smart Board, Ray used a non-digital stick pointer that provided space between them and the screen.

tumblr_inline_n8sexnxaRT1s66zlpAfter doing exercises of the Hour of Code, Ray started to teach concepts using the Scratch programming language. Each Scratch class started with a period of interactive instruction during which Ray used the Smart Board to show and modify Scratch examples. Students would ask and answer questions and help with writing programs by going up to the board and touching and dragging blocks and sprites to appropriate locations. After this time, the students had time to work on individual stations on Scratch examples. While they were encouraged to work on exercises, if they persisted to do other things (look at YouTube videos or continue to play a Scratch game), as long as they were not disruptive to others their choice was tolerated.

Screen Shot 2014-08-28 at 6.09.46 PM

During the final week of the summer school, I visited Ray’s class to see a final project that the students were working on. The project involved designing an interactive digital greeting card for the Vice President of the school. The first thing I noticed was how engaged the students were with the task. I really enjoyed seeing Ray in action as well. He is very patient, clear and creative in class and makes an effort to include all the students in the activities. I noticed that he used simple metaphors and a lot of movement and action to illustrate points, techniques that I, in a much less refined way, have employed when teaching young children. I’ve described this approach as “poetic presentation” in previous work.

The greeting card was set as the scene of a party where several avatars (or sprites as they are called in Scratch) were placed in a room and danced and talked to each other once the program was run. This was an excellent choice for a collaborative project because by changing different parameters in the code, the students can change the behaviour of the avatars. For example, by changing a number they can increase or decrease the speed with which a particular sprite is dancing or walking across the dance floor and by changing another parameter, they can change what he or she (or it!) says. I think the notion of having control over a character that you possibly identify with and exercising the idea that by changing a parameter you can change and influence their behaviour is an important therapeutic element. Many of the children who have cognitive disabilities (especially autism) have problems in social settings and lack in social skills. Perhaps helping them create and control the behaviour of avatars in a safe social environment can lead to behaviour change. While in this exercise, the avatars were chosen arbitrarily and did not explicitly correspond to the students, in another exercise, where the students were asked to create avatars, many of them created avatars that resembled themselves, providing signs that they identified with the avatars they created. This approach could be the very beginning of a new field of a collaborative digital interactive art therapy.

Visiting the class was very inspiring to me: the students would get happy and excited as they saw the consequences of the choices and changes made to the program in real-time. Often, there was applause and laughter as an avatar was made to dance or move, faster or slower. Ray would suggest a change to the program and would invite a student to go up to the board and make a change. Other students would offer suggestions and encouragements when changes were made. Ray was facilitating by creating a safe space, which is essential for such experimentation with programming and creativity. In fact, Ray observed that an important aspect of using computer environments was that they provided an environment that was, or at least perceived to be, safe to make mistakes and errors in. In this way, he encouraged students to experiment and try unusual parameters to see what would happen. For example, they would try to enter a very high value for the speed a character would dance and see how the result would be different. This often would lead to humorous situations that can lead to a sense of safety and playfulness and can facilitate learning and engagement.

After the collective teaching period, the students would work on individual computer stations. A couple of students were very interested in copying the code from the Smart Board and enthusiastically ran back and forth between the board and their computers. Several students had formed groups and would try to help each other make changes to the program. At the end of the class, some students really wanted to stay and continue, a good sign that they enjoyed the class and were engaged with the material.

The summer school is an exceptional research opportunity, given the number and diversity of the students participating in the program. It would be great to conduct a user study, examining the efficacy of Ray’s approach and the dynamics that arise in its application.

For more information about the summer school and other projects, check out Ray’s blog.


Several weeks after the summer school, Ray and I, attended the Scratch@MIT conference. The conference held at the beginning of August at MIT’s campus in Boston is an international forum and workshop where people from around the world and with different backgrounds (teachers, developers, researchers, children, …) come together and share their experiences of using Scratch in different context and for different ends. Ray and I wanted to deepen our ties with the international community and also become aware of more advanced technical possibilities of Scratch.

As it turned out, in addition to attending many engaging and informative workshops, presentations and panels, we met many people who were specifically interested in the possibilities of using Scratch for and with children with disabilities. This expressed interest in the community, gave us a chance to organize an “Unconference Session” with Kyle Keane, a software developer from the Boston area with a keen interest in assistive technology and specially technology developed for children with visual impairment. The session was dedicated to discussing the potentials and challenges of using Scratch with children with different disabilities. In total, 14 people attended the session. The participants were from 4 countries (US, Canada, Italy, UK) and representaed a wide range of backgrounds, including 3 members of the Scratch Team (MIT Scratch developers), 2 software developers, 4 researchers, 4 teachers and 1 therapist in private practice.

After the session, we decided to keep in touch through initially a mailing list and later on in a more suitable format (possibly a website, Google group or blog).


4 general directions or potential areas of interest and future development could be identified from our discussions:

  • Tools: Many participants were interested in tools designed or modified specifically for children with disabilities. These included tangible Scratch blocks, replacing text with symbols, …
  • Processes: What are some mechanisms that can deploy learning and creativity through Scratch possible in different settings? We had some great examples of how to motivate teachers and parents to work with Scratch and how to connect academics and teachers through community engagement in initiatives. Perhaps more mechanisms can be identified and developed.
  • Community: Having a way of connecting the diverse (and still relatively small) international community of educators, developers, researchers and other stakeholders who are interested in this specific area (Scratch and children with disabilities). I think this meeting (and mailing list) is a great starting point and we should explore ways to keep it alive and connected.
  • Communication: What are some ways of communicating and disseminating knowledge and experience in this area to a wide variety of stakeholders? Can we develop a curriculum (or a supplementary section) specifically for children with disabilities and people who work with them and care for them? I think the key here is to have different channels of communication open so that people can access them in different ways.