Baking Pi’s 2: Touch HAT

In the “Baking Pi’s” series of posts, I describe my adventures with the Raspberry Pi embedded computer. Each segment describes a specific setup. This series is fairly technical and is meant to be used to create digital prototypes.

Screen Shot 2016-02-10 at 7.03.48 PM

Raspberry Pi with Touch Hat and Fruit!

In this post, I will describe my experience using Raspberry Pi with a touch sensor to create an interactive sound box. This post assumes that you have a Raspberry Pi B+ setup and connected (either to a monitor or running in headless mode). See my previous post or the official Raspberry Pi website on how to setup your Pi.

Using Pi to create an interactive sound box

A really fun activity is playing music using fruit and other conductive things (even humans)! You can do this by connecting the fruit to a computer using a touch sensor that in effect treats them as a key on the keyboard, making it possible to activate sounds when they are touched.  Some examples of how to do this easily is using a Makey Makey board or a Bare Conductive touch board. In the past, we connected an MPR121 Capacitive touch sensor to a Raspberry Pi to create TalkBox, a DIY communication board for non-verbal users. One of the challenges of putting together TalkBoxes was to connect all the wires from the MPR121 breakout board to the Raspberry Pi. Recently, capacitive touch HATs designed specifically for Raspberry Pi have become available that make it much easier to create an interactive sound box with the Raspberry Pi.

Screen Shot 2016-02-10 at 7.04.05 PM

Preparing the Touch HAT

For this project, you need a Raspberry Pi (already setup with a Linux distribution system and connected to the Internet, see above) and Adafruit’s capacitative touch HAT. You have to do a bit of soldering of a 2×20 solder header (included in the kit) to the touch HAT which makes it fit perfectly on top of the Raspberry Pi B+ (for older Raspberry Pi’s you need to solder a different taller solder header, see here).

Screen Shot 2016-02-10 at 7.03.55 PM

Pi wearing its new HAT!

One you have soldered the header and placed the HAT on the Pi, you need to install software following instructions here. The one thing that is missing from the tutorial is that you also have to enable the I2C protocol so that the Raspberry Pi can talk to the HAT. To do this use sudo raspi-config and choose Advanced Options. Here, you can enable I2C support and set it to load by default. Once you reboot, the libraries should be loaded.

Following is an example of code that runs a simple version of TalkBox (with a small set of constant sounds):

# Copyright (c) 2016 Foad Hamidi
# Author: Foad Hamidi based on code originally written by Tony Dicola from Adafruit Industries
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.

import sys
import time
import pygame.mixer

import Adafruit_MPR121.MPR121 as MPR121

print 'TalkBox Test'

# Create MPR121 instance.
cap = MPR121.MPR121()

# Initialize communication with MPR121 using default I2C bus of device, and 
# default I2C address (0x5A).  On BeagleBone Black will default to I2C bus 0.
if not cap.begin():
    print 'Error initializing MPR121.  Check your wiring!'

#Initialize the sound system, and create empty lists of channels and sounds
pygame.mixer.init(48000, -16, 1, 1024)
soundChannelList = [None] * 12
soundList = [None] * 12

#Populate your list of sounds and channels, there are only 7 channels, so you
#will have to reuse some channels. Channels are used so that sounds can be played
#I'm using some semi-random voice samples from a previous project. 
sound1 = pygame.mixer.Sound("/home/pi/Music/Feeling/happy.wav")
soundChannel1 = pygame.mixer.Channel(1)
soundList[0] = sound1
soundChannelList[0] = soundChannel1
sound2 = pygame.mixer.Sound("/home/pi/Music/Feeling/excited.wav")
soundChannel2 = pygame.mixer.Channel(2)
soundList[1] = sound2
soundChannelList[1] = soundChannel2
sound3 = pygame.mixer.Sound("/home/pi/Music/Feeling/feeling.wav")
soundChannel3 = pygame.mixer.Channel(3)
soundList[2] = sound3
soundChannelList[2] = soundChannel3
sound4 = pygame.mixer.Sound("/home/pi/Music/Feeling/proud.wav")
soundChannel4 = pygame.mixer.Channel(4)
soundList[3] = sound4
soundChannelList[3] = soundChannel4
sound5 = pygame.mixer.Sound("/home/pi/Music/Feeling/sad.wav")
soundChannel5 = pygame.mixer.Channel(5)
soundList[4] = sound5
soundChannelList[4] = soundChannel5
sound6 = pygame.mixer.Sound("/home/pi/Music/Feeling/sick.wav")
soundChannel6 = pygame.mixer.Channel(6)
soundList[5] = sound6
soundChannelList[5] = soundChannel6
sound7 = pygame.mixer.Sound("/home/pi/Music/Feeling/tired.wav")
soundChannel7 = pygame.mixer.Channel(7)
soundList[6] = sound7
soundChannelList[6] = soundChannel7
sound8 = pygame.mixer.Sound("/home/pi/Music/Feeling/good_morning_f.wav")
soundChannel8 = pygame.mixer.Channel(1)
soundList[7] = sound8
soundChannelList[7] = soundChannel8
sound9 = pygame.mixer.Sound("/home/pi/Music/Feeling/goodbye_f.wav")
soundChannel9 = pygame.mixer.Channel(2)
soundList[8] = sound9
soundChannelList[8] = soundChannel9
sound10 = pygame.mixer.Sound("/home/pi/Music/Feeling/need_break.wav")
soundChannel10 = pygame.mixer.Channel(3)
soundList[9] = sound10
soundChannelList[9] = soundChannel10
sound11 = pygame.mixer.Sound("/home/pi/Music/Feeling/thank_you_f.wav")
soundChannel11 = pygame.mixer.Channel(4)
soundList[10] = sound11
soundChannelList[10] = soundChannel11
sound12 = pygame.mixer.Sound("/home/pi/Music/Feeling/sunny.wav")
soundChannel12 = pygame.mixer.Channel(5)
soundList[11] = sound12
soundChannelList[11] = soundChannel12
print "Soundboard Ready."

# Main loop to print a message and play a sound every time a pin is touched.
print 'Press Ctrl-C to quit.'
last_touched = cap.touched()
while True:
    current_touched = cap.touched()
    # Check each pin's last and current state to see if it was pressed or released.
    for i in range(12):
        # Each pin is represented by a bit in the touched value.  A value of 1
        # means the pin is being touched, and 0 means it is not being touched.
        pin_bit = 1 << i
        # First check if transitioned from not touched to touched.
        if current_touched & pin_bit and not last_touched & pin_bit:
            print '{0} touched!'.format(i)
            soundChannel = soundChannelList[i-1] 
            sound = soundList[i-1]
            print'Sound played'
        # Next check if transitioned from touched to not touched.
        if not current_touched & pin_bit and last_touched & pin_bit:
            print '{0} released!'.format(i)
    # Update last state and wait a short period before repeating.
    last_touched = current_touched

    # Alternatively, if you only care about checking one or a few pins you can 
    # call the is_touched method with a pin number to directly check that pin.
    # This will be a little slower than the above code for checking a lot of pins.
    #if cap.is_touched(0):
    #    print 'Pin 0 is being touched!'

Finally, adjust the volume of the speakers using the following command at the terminal prompt:

amixer  sset PCM,0 90%

which will adjust the volume by the indicated percentage of its total capacity (e.g., 90% in the above example).

Here’s a video of the setup used to make two pieces of fruit have a conversation:

Note: If you are getting weird static sounds out of the speaker connected to your analog audio output, it’s probably because of the power source. Try different ones and hopefully you will find one that doesn’t’ affect the audio performance too much. Another possibility is that you have not plugged in your speaker properly. Check it again before giving up on your speakers!

Finally, if you want this code to be loaded at startup, follow these instructions.



Toronto Mini Maker Faire 2014

As we put the final touches on our two tables, I can hear people entering the main hall of Toronto Reference Library on this November Saturday morning for the first day of Toronto Mini Maker Faire. Yes, people are up early and ready to explore and get the latest news on local Makers, most recent 3D printers and scanners, wearable computers, crafts augmented with electronics (or not) and many other fun and crazy ideas!

Toronto Mini Maker Faire 2014

Toronto Mini Maker Faire 2014

This is the second time we are at the Toronto Mini Maker Faire and we have come a long way since last year when we were mainly spectators. Granted, on one of the days last year, I (with my collaborator, Natalie Comeau) demoed an early version of our wearable computing design, HugBug, at the Faire, but other than that we were observing and getting acquainted with the Toronto Maker scene. Last year, I visited with participants from an experimental MakeShop I was facilitating at York. This year, several of the participants from the MakeShop were also at the Faire but in a different capacity: as Makers presenting their work.

Screen Shot 2014-11-26 at 8.36.05 AM

Our tables at Maker Faire

“Making” in this context, refers to a specific way of fabricating custom prototypes and designs using hands-on methods that can range from using embedded electronics and 3D printing to using craft materials and methods, usually with an emphasis on open-source and Do-it-Yourself (DIY) designs and with a view to share knowledge and experience with fellow “Makers”. This attitude has become widely popular in the last decade through publications such as the Make Magazine and events such as Maker Faire, as well as, the proliferation of thousands of fabrication and Maker spaces, hack labs, and online forums and communities around the world. Making, with roots in older notions of “Hacking” and “Tinkering”, has gained such recognition as to be dubbed as the “new industrial revolution” and a “movement” that promises to democratize design and fabrication (see Chris Andersons, Makers: the New Industrial Revolution). Our Making activities have grown much over the last year and our presence at the Faire reflects this: in addition to two tables, we are also running an ambitious workshop for Makers with disabilities and their parents and caregivers. We are showcasing several projects that are examples of cross disciplinary collaborations. In this post, I will briefly describe these projects but before that I would like to briefly describe my experience at Maker Faire in general. This year, I mainly stay at our tables, describing and demoing our projects and meeting a lot of amazing people. While we have a great team of knowledgable and energetic volunteers at the table who would cover me if I want to take a break, I am having such a great time sharing experiences with visitors that it is difficult to leave! Also, I have worked on all the projects that we have on the table in some capacity and it is such a pleasure to present them to an open-minded audience. Because of this (and sadly!), I haven’t seen many projects other than our own. However, my brief detours around the Faire made me realize the range of projects from the usual embedded computing platforms (mainly Arduinos in many shapes and sizes, as well as, Raspberry Pis), to many shades of 3D scanning and printing, to hacked toys (in the creative sense of the word!), to non-digital puppets and even a hitchhiking robot that has crossed Canada!

Fun Projects at Maker Faire: (clockwise from top left) mixMotion, Puppets Cool, Cyberpunk/homemade mechanical nut cracker

Fun Projects at Maker Faire: (clockwise from top left) mixMotion, Puppets Cool, Cyberpunk/homemade mechanical nut cracker

The free and public location of the Faire at the Toronto Reference Library is fantastic: accessible and inviting to the general public (as Making should be)! Over the weekend, a whopping 10’000 people were estimated to visit the Faire! In my experience, I talked to so many interesting people that at the end of the Faire, I had an information overload and had to take a couple of days completely off to reflect and digest all the input! As I mentioned, this year we have two tables, one showcasing a range of projects from our lab, Graphics and Media at York (GaMaY), and the other, showing our colleague and collaborator, Ray Feraday‘s work, with especial focus on our collaboration on DIY open-source assistive technology. On his table, Ray has set up a pong game developed using the Scratch programming language, running on Raspberry Pi and controlled using home-made cardboard controllers using Makey Makey, all put into a cut kitchen table top surface. It is a hit with children! Screen Shot 2014-11-26 at 8.35.13 AM TalkBox TalkBox is an open-source DIY communication board for children or adults with no voice. This is truly a classic Maker project: the idea was first conceived by Ray, an inspiring and inventive special education teacher working with the TCSDB, who I met last year at Maker Faire. He had created a customizable communication board using the Makey Makey and conductive tape and was interested in improving the design and making it more mobile and affordable by replacing the Makey Makey and attached computer with a touch sensor and Raspberry Pi (an embedded computer). We formed a team with Toni Kunic, a great programmer and a computer engineering student at York and Melanie, my supervisor, and met over many weekends to work out and implement a new design for TalkBox. The result is a DIY communication board that can be used to store voice samples and associate them with touch buttons that play them back once touched. TalkBox costs less than $80 and is open-source (both hardware instructions and software code are shared freely with users). For more information please see this link.

Ray demoing TalkBox at Maker Faire

Ray demoing TalkBox at Maker Faire

Unexpectedly, we won an award for TalkBox at MakerFaire! Bridgable, an innovative design firm in Toronto, awarded us with the Bridging the Gap Award. This award was given to a team that demonstrated “a clear connection between a user need and product benefit; an extraordinary translation between research and design, concept and prototype or material/technology and execution; and an appropriately accessible and well communicated presentation”. We are very happy to accept this award as it provides us with further recognition of the idea of making “Making” accessible that we have had in mind with this project.

The Bridging the Gap Award

Bridging the Gap Award

MakeTalk Workshop Since the beginning of our collaboration on the TalkBox project, we had envisioned a process through which children with disabilities, their teachers and caregivers could put together the kits themselves. We wanted to make the design available and the process of Making accessible. The vision of making Making accessible has been discussed before  by inspiring assistive technology advocates and digital activists, Amy Hurst and Shaun Kane (see their paper here).  Additionally, we wanted to make a process in which we explored alternative ways to fund and deploy the technology to the people who need it most. To these ends we partnered with the Tetra Society of North America which is a volunteer-based organization that brings together student volunteers, retired engineers and inventive users with disabilities and their families together to modify, create and deploy assistive technology solutions. Tetra has a chapter at York University with many student volunteers. We decided to ask for their help in running a workshop at Maker Faire and also getting funding to create a series of TalkBoxes. The president of the York chapter, Brandon, is also a graduate student at our lab and along with another volunteer coordinator, Zareen, helped with recruiting, coordinating and training volunteers. (Brandon is also great at soldering, so he helped prepare the soldering part of the TalkBox kits too!)

Pre-workshop preparations

Pre-workshop preparations

In preparation of the workshop, we ran a series of dry runs at our lab in the weeks preceding the Faire. We recruited the help of Yana Boeva, a graduate student in Science and Technology Studies at York who has experience in Making and creating instructibles. Our amazing Tetra volunteers helped with every stage of the process: from preparing the material to running the workshops. Melanie designed a customized case for the Raspberry Pi which she miraculously (read, through many hours spent in the lab!) managed to print in time for the participants at the Faire.

3D printed modified Raspberry Pi cases

3D printed modified Raspberry Pi cases

At the Faire, we provided a kit consisting of all the parts and instructions to 7 groups of users with disabilities and their parents/guardians/care givers. I decided to stay the tables while most of the rest of the team were busy at the 2-hour workshop, so I could only tell from their elated faces and happy expressions how rewarding the process was.

MakeTalk Workshop: Kit, process, outcome

MakeTalk Workshop: Kit, process, outcome

Putting the kits together is not trivial and yet everyone at the workshop managed to put them together and get them working. I was reminded that one of the joys of working with people with disabilities is that they have a lot of patience and see value in process rather than just wanting quick outcomes.

HugBug HugBug is a playful wearable interface designed to teach children about digital design. I developed it with Natalie Comeau. I used it in a workshop in Mexico (described here) and presented it at the Design and Emotion Conference in Bogota. I have simplified the design and made it more robust and straightforward. HugBug is very successful at Maker Faire with many children and adults trying it out.

Colin trying out HugBug

Colin trying out HugBug

The next step for this project is to connect HugBug to biosignals (I am currently working on this project with our colleague, Manuel, from University of Seville) so that it would externalize various states onto its LED lights (e.g., stressed vs. relaxed).

Manuel connecting HugBug to Biosignals

Manuel connecting HugBug to biosignals

Rafigh Rafigh is a living media interface to encourage children to engage in learning and therapeutic activities. It consists of a real mushroom colony connected to a micro controller that controls the amount of water administered to it. At Maker Faire I have an early prototype of Rafigh with dried mushrooms for demonstration purposes. I am currently conducting a study with the working prototypes in situ at participants’ homes. For more information on Rafigh please see this link.

Magic Wand Magic Wand is a project of the Wandmakers, a team of three engineering students, Sonal Ranjit, Chitiiran Krishna Moorthy and Kajendra Seevananthan, designed and fabricated their own Harry Potter-inspired wand after attending the MakeShop sessions I facilitated last year. The Wandmakers created a custom 3D model of the wand, which they printed and outfitted with a microcontroller, accelerometer and laser light, such that specific movements of the wand would be translated into “spells” that would cast a laser light.

Magic Wand

Magic Wand

Magic Wand was first presented at the TEI’14 student design competition earlier this year and Sonal joined us at Maker Faire to present the Wand to many interested children (and adults). They had printed the instructions on beautiful rice paper that people took home as a souvenir. Their 3D model is open-source and available for free download on Thingiverse. It has been quite popular with fans and since last February when it was uploaded has been downloaded more than 1500 times!

Children trying Magic Wand

Children trying Magic Wand

Synchrum Synchrum is a tangible interface to facilitate audience participation through rhythmic collaboration. It was inspired by the Tibetan prayer wheel and aims to capture its physicality and performativity. A light source is attached to the top of the object and a weight rotates under the light. By detecting the number of blocks in the light by a sensor we calculate and communicate the rotation speed of the device wirelessly to a computer that can then coordinate its speed with other Synchrum units. Using Synchrum, members of an audience can synch with each other and collaborate to bring about a change in a performance or environmental factor (such as ambient sound, music or light). In a previous performance, members of an audience collaborated together to remove virtual chains from a performer. Synchrum was designed in collaboration with Alexander Moakler and Assaf Gadot and was presented at TEI’12 and UIST’12 conferences. Please see this document  for more information.


Our Team This year our team consisted of Melanie Baljko (my supervisor and co-director of GaMaY), Ray Feraday (our community partner, a special education teacher and an inventor), Brandon Haworth (a PhD candidate at our lab with focus on serious games and Making), Toni Kunic (software and design developer for TalkBox), Sonal Ranjit (co-designer of the Magic Wand), Yana Boeva (instrcutible designer for the TalkMake workshop), Manuel Merino Monge (visiting researcher from University of Seville, working on HugBug), Natalie Comeau (co-designer of HugBug), Catherine Duchastel de Montrouge (Science and Technology Studies), Colin Ruan (Computer Science) and myself. Additionally, we got a lot of help from the York University Chapter of Tetra Society volunteers who tirelessly helped run the MakeTalk workshop, as well as, helping present at the tables. These included: Zareen, Steven, Syed, Nina, Laura, Vassil, Greene and Nitzi. Additionally, Jeannette Da Luz from TCDSB helped create digital content during the Faire, and Glenn Barnes and Hamed Dar from Tetra Society of North America, helped with setting up and running the tables. Thanks so much everyone and great job!