Rafigh (Companion)

Rafigh-highresIn an earlier post, I promised to talk more about my main PhD project, Rafigh, in more detail in the future. I presented Rafigh at the Tangible, Embedded and Embodied Interaction (TEI’14) conference and the CHI Conference on Human Factors in Computing Systems (CHI’14) this year, both as demos in the Arts Track and Interactivity sections of the conferences (respectively) and in the paper session of CHI. So, I think the time has come to describe it in more detail here.

Rafigh (“companion” or “buddy” in Persian and Arabic) is a living media interface for support of behavioural intervention (e.g., speech intervention) for children. It consists of two parts: the software part is a suit of therapeutic games and learning activities (currently, a reading activity and several speech exercises) on an iPad (or desktop/laptop computer) that we want to encourage the child to engage in; the hardware part, consists of a custom-designed box that contains a living mushroom colony and an irrigation system, connected to the iPad wirelessly.

Screen Shot 2014-05-13 at 9.21.33 AM

The mushroom colony is completely non-toxic and is made primarily from used coffee grounds injected with live mushroom spores. The mushrooms, when properly irrigated, will grow to their full size in about two weeks (the speed and length of growth depends on the mushroom variety and in our experiments has varied from 10 to 14 days). The irrigation system consists of an automated power switch connected to an aquarium water pump and controlled by an Arduino microcontroller that is connected to the iPad via wireless radio. Thus, the frequency and duration of irrigation can be controlled by the iPad application. In the current implementation, the more time the child spends on doing the activities, the more water will be given to the mushrooms. The amount of water administered is within a range to avoid overwatering or under watering the mushrooms.

The idea of developing a project where children would practice their speech and other language skills through computer games and digital media was inspired by knowing about how much children are engaged by interactive digital activities and games. A new wave of “gamification” has swept through many areas of digital design, from learning projects to therapeutic activities. No doubt interactive technologies have the capacity to engage children for extended hours (just imagine all the hours spent on Angry Birds around the world!) but for me the question is not whether technology can engage children but rather, engage them to do what?

The motivation behind incorporating a living being into the interface was inspired by observing two children in my community who within the short span of 6 months, interacted (on a daily basis) with a dog, a cat, a bearded dragon, two gerbils, turtles, plants, worms and crickets! I think the message for me would have been hard to miss: Children are fascinated by nature and living beings. Additionally, they enjoy taking care of and being responsible for other beings. In the case of the children I was observing, this focus on interacting with living beings and nature might have been more intense than your average urban family (perhaps because of their parents’ interest in nature), but inspiration does not have to be generalizable in order to lead to ideas that can make an impact.

This idea led to a “why not?” moment where I asked: why not design a game that explores dynamics other than high scores or badges (which are fine as mechanisms to encourage game players in certain contexts but have long monopolized gaming incentives)? Perhaps, we can design for technologies that make us more caring and responsible. Technologies that empower children to take care of other living beings in a low-risk scenario. Given the increasingly urban environments in which our children live in and grow up in, I think we are in timely need of ways of reconnecting them to nature and other modes of life.

Additionally, many of the children I work with have various disabilities which makes it harder for them to care for pets (such as dogs or cats), so it would be good if we could develop an interface that allows children to take care of a living being even if they have disabilities by mixing their intentionality with more automaticity. Previous research has shown that having pets can increase children’s confidence and provide for many learning opportunities.


The next step in the evolution of these ideas was what animal or plant to choose to care for? I looked at several possibilities from goldfish to sea monkeys (!) and realized that in general animals are hard to control and plants are too slow. So I settled on mushrooms, not only because they grow faster and can be controlled by adjusting water (which is relatively easy) but also because they are edible. This adds a poetic aspect to the project. The children not only can care for a living being but they are also growing food for their facilities. Perhaps, in the long run, this would nurture the idea that the children can care for others in the community when they grow up.

There are many things I love about mushrooms: not only have they fascinated artists from John Cage to Bjork but (at least the ones that I work with) embody beautiful lessons about nature. The mushrooms that I work with (oyster mushrooms) grow to their prime before sporing and starting to shrivel. If they are not harvested at the peak of their prime, they will start to become smaller and eventually dry out (no matter how much water is given) and so it is a nice demonstration of nature’s cyclic patterns where life and death intermingle. For the children, the mushrooms do not die in a  traumatic and unfortunate manner: they transform into food we can eat.

Screen Shot 2014-05-13 at 9.21.49 AM

During the two conferences where I had to demo Rafigh, I learned a few lessons about working with living material: mainly that you have to have patience and respect for life. When you travel with living mushrooms or set them up for an installation, you can’t change their natural pace or pattern of growth. Sometimes, their growth slows down because of environmental conditions or being moves across continents (can mushrooms get jet lagged!?), sometimes you need special permission to travel with them and usually you get to eat your interface after a conference 🙂

I look at these characteristics as interesting research material. For example, when it comes to patience, how many games teach us patience these days? In a world, obsessed with speed, I think we should consider slowing down our speed. Notions of time are different between digital and living material. You can usually pause or turn off a game, but can you pause your life or your friends? In Rafigh, you can’t pause the mushrooms once you start interacting with them. Suddenly both your action or inaction are choices that affect your interaction. You can pause the iPad game but that does not pause life. I think these would be useful lessons and interesting dynamics to examine during user interaction with Rafigh. In a way, this line of thought reminds me of Arjuna and Krishna’s conversation in Bhagavad Gita, but that’s another story!


On the design side, I used laser cutting to develop a custom box to house the mushrooms. Laser cutting is a technique by which you can cut patterns in material like wood and cardboard by using a laser. It is a surprisingly powerful method whereby you can create a lot of cool and structurally strong material. I am specifically including a design aspect in my work, as I find design thinking and methodology extremely interesting. With help from Michael Longford, from the Design Department at York, I was able to create a box that has compartments that separate the electronic components from the organic material, reducing the risk of damage through moisture and humidity. Additionally, I am able to show that it is possible to make customizable prototypes of the system which look more like finished “products” than research prototypes. In a previous user experiment, one of my prototypes got destroyed because the child was curious and wanted to take the components apart, a great trait in a child but not very good for my prototype! So I decided to make the structure stronger and more fail-safe.

In the new version, I have also separated the iPad and the mushroom colony more explicitly. The main reason is that I want to explore the use of the system in a setting where the children share the experience with other people who do not live with them (e.g., grandparents, cousins, …) and do activities at home and make the mushrooms grow remotely. This can be achieved by connecting the mushrooms to the internet (via for example a Raspberry Pi) and controlling the irrigation off-site. It would be interesting to see how having this tangible living medium affects relationships.

The next stage is to make a more robust prototype and start testing with one or two participants to see what the mechanisms of the interaction would be.




“Our finger moves!”: An interdisciplinary 3D modelling and robotics workshop for high school students

I love collaborations! In a world where the explosion of specialized knowledge means having a deep understanding of any field requires years of study and reflection, it is essential to collaborate with others, in order to be able to benefit from a collective set of skills and perspectives, spread across many people.

Our team at work

Our team at work

In a recent Youth Workshop jointly run by the Centre for Innovation and Data Driven Design (CIVDDD) and the Information and Communications Technology Council (ICTC), we had a chance to do just that. An interdisciplinary team of mechanical engineers and computer scientists specializing in the diverse areas of robotics, computer graphics and human-computer interaction, collaborated to design and conduct a series of three workshops for high school students. The workshops consisted of hands-on group activities in which each team was to learn how to customize and change a 3D model for 3D printing, how to control basic movements using servos and micro controllers, and, finally, how to combine these to move their 3D printed models in a group installation.

I had collaborated with Barbara Whitmer, the CIVDDD project manager who was leading the team on another set of workshops for high school students in the Fall that had proven successful and had given us another opportunity to run workshops with high school students. This time we were more ambitious and aimed for hands-on workshops in which the students could experience 3D modelling and printing and robotics in a meaningful way. This involved coming up with a creative and engaging activity for the workshop participants, bringing together expertise from several different fields, figuring out the logistics of what should be done when and the sequence, “narrative” and outcome of each workshop.

After a few brainstorming sessions, Barbara, came up with a fun idea: how about having student teams each play with the 3D model of a single finger, customize them within constraints, print them out and attach them to a pre-made hand whose movements could be controlled by servos? This was a great idea and would not only give the students a chance to be creative and try out cutting-edge tools and techniques but also see the outcome of their efforts in a tangible and physical way at the end of the workshop.

The next question was how to do it? Tom Young, a PhD student in Computer Science who has expertise in robotics and embedded electronics joined our team and came up with several ideas of what to cover during the robotic workshop. This included showing students how to implement simple movement using Arduino microcontrollers.


After consulting with several of the engineering faculty, Dr. Pouya Rezai and two of his graduate students, Ramtin Ardeshiri and Jacob Leung also joined the team. They designed the gears and shaft that was to be the base of the fingers and translate the movement of the servo to the movement of the fingers. Everything (except the servo and microcontrollers) was to be 3D printed.

On the 3D design side, we had the help of Ryan Schmidt of Autodesk who is the creator of the powerful MeshMixer program. I collaborated with him to modify an existing hand model such that it could be emptied and scaled to house the servo and gears. We also extracted finger models from it for participants to play with and customize.

Our team worked relentlessly behind the scenes for many weeks to make the project happen. But the results were amazing, having the participants create their own customized and tangible objects and see them setup in a collective project was great!


The first workshop was led by Ryan who introduced MeshMixer and taught the 5-member student teams how to use it to make modified 3D fingers. The outcome of this workshop was a series of 10 customized finger models that were cleaned-up and prepared for printing by Ryan. We had a series of 5 printers from MakeLab which started printing the fingers right away.


In the second workshop, Tom, introduced the students to basic robotics concepts and showed them how to connect a servo to Arduino and how to program it. At the end of the workshop, the students could control servo movements programatically or by using the keyboard.


Finally, at the last session, the fingers were assembled onto the two hand setup that Ramtin, Jacob and Tom had prepared previously. The final outcome looked weird and wonderful: a multicoloured hand with strangely shaped fingers that were designed on the spot and conjured out of thin air!

Lassonde Tech Faire_3D Printed Robotic Hand_Design Team_25Apr2014

I really enjoyed the collaboration and would also like to thank Josh Sideris and Hugh Chesser for their help and ideas. Here’s a link to another review of the event.

Days later in Montreal, at a reception for the Graphics Interface conference which was held at a wax museum, I saw a number of wax hand models that reminded me of our workshop!20140508_192714