Final Product

The final prototype includes Emily (fully interactive) and the switch box (to simualte turning on and off the air conditioner and heater, and change outdoor temperature). Emily measures the indoor temperature, and compares it to the outdoor temperature. If she deems it inefficient she changes colour, changes her facial expression, vibrates, speaks, and sings (depending on the level of escalation she is up to).

Emily has three states: Normal, Hot, and Cold (as seen below).

Intended Interaction Video

Live Demo

Basic Overall Logic

Emily will sit in the living room of a house. She will constantly monitor the temperature of the household and compare it to the temperature outside. When she notices that the behaviour is inefficient, she will change to red or blue (depending on whether the temperature inside is too hot or cold), change her facial expression, and begin annoying the user by complaining when they walk past. These complaints will be selected randomly depending on the state she is in. If no one walks past within 10 minutes, she will begin singing either "Everybody Hurts", "Man in the Mirror", or "Killing Me Softly" to try to annoy the user in a funny way to come out to the living room and correct the behaviour. If the behaviour isn't changed, she will start vibrating and screaming. This alarm sequence will continue until someone turns her off by holding down the buttons on both her arms, and will then thank the user and return to her happy facial expression and green colouring. She will continue monitoring and if the temperature hasn't returned to an acceptable level in 10 minutes she will go back to the beginning and start this process again.

For the purposes of being able to demo this prototype at the exhibit, this logic has been changed from 10 minutes to 30 seconds.

Temperature Logic

The logic used to categorise inefficient temperatures are as follows:

• When no appliances are turned on, the temperature is never deemed inefficient.
• When the air conditioner is on, it will be inefficient if the indoor temperature is below 23 or if the outdoor temperature is less than 24.
• When the heater is on, it will be inefficient if the indoor temperature is greater than 24 or if the outdoor temperature is greater than 22.

Emily's Responses

Complaints

These are a few of the complaints Emily will say if she is in a hot or cold state and someone walks past.

Songs

These are a few of the songs Emily will sing if no one walks past to hear her complain.

Yelling

These are a few of the escalated complaints Emily will say if the user still hasn't changed their behaviour (i.e. turned off the air conditioner or heater).

Technical & Physical Design

Technical Build

Features:

• Powered by esp32
• Detects temperature using a digital temperature sensor module
• Detects user presence through ultra-sonic distance sensor
• Converts text-to-speech using Parallax Emic2 text-to-speech synthesise Module (using dectalk algorithms)
• Shows facial expression on tjctm24024-spi (240x320) screen
• Light changes through an led neopixel strip
• Vibrates using a vibrating mini-motor
• Turning off alarm detection through buttons on both arms
• Switches to simulate turning on the air conditioner and heater
• Potentiometer to simulate outdoor temperature

Key Features on Board

Key Libraries Used:

• FastLED: used to control led neopixel strip
• Dallas Temperature: used to talk to temperature sensor
• One Wire: also used to talk to temperature sensor
• Tjpg Decoder: used to send images to screen
• TFT eSPI: used to talk to the screen over SPI
• Arduino-Timer: used to trigger events (e.g. measure temperature)
• OneButton: used for buttons on arms

Physical Build

Main Prototype

The physical build is made out of a custom laser cut design on acrylic.

The build includes the outside shell, as well as internal structures that hold all the components together. In particular there are shelves to hold the screen, speaker, and main board.

External Switch Board

I also created a box to store the switches and potentiometer.

Design Process

Overview

The design process focused on the discovery multiple research questions surrounding the topic, including:

1. Create a prototype that effectively implements all intended features.
2. Create a form that is help create an emotional connection with Emily.
3. Utilise negative reinforcement techniques to annoy users into change.
4. Create an emotional connection between the user and Emily.

User Testing Findings

Negative Reinforcement

I conducted exploratory interviews with four people, with the focus on: what responses annoy people when using a device (e.g. their phone); how to find a balance between annoying the user too much and not enough (and if having an emotional connection helps this); and attributes in current devices or movie characters that makes you like them. From this, it was found that most users found loud and constant noises, vibrations, and difficult to use technologies (e.g. apps) most annoying. All users agreed that they would discontinue using a product if it was too annoying to use, however, one user revealed that if they purchased the product with the intention to promote a behaviour change, the annoyance would be a helpful reminder. They also suggested that if it was playful and fun to interact with it the annoyance could be fun too. When describing features that make you emotionally connected to a cartoon, all users described specific facial features (e.g. big eyes) and cute voices. From this, I decided that having a screen with facial expressions and something text-to-speech functionality was important to create the desired emotional connection. I also decided to focus on constant noise and vibrations would be the key negative reinforcement features. I decided to take a "passive aggressive" approach to these noises through complaints to further add to Emily's personality and act as negative reinforcement. I researched into text-to-speech software that could do this. I found a popular one called DeckTalk that allowed different tones to be played and found a module (Emic2 Text-To-Speech Synthesizer Module) that could be connected to my esp-32 (along with a speaker and amplifier) to play these sounds. I also found a library full of DecTalk songs that I could add to my prototype. I decided to use "Everybody Hurts" by R.E.M as a funny way to get the users attention if they haven't walked past Emily to hear her complain.

Environmentally Conscious Decision-Making

I conducted another set of interviews with five people that aimed to understand people’s environmentally-conscious decisions. All participants stated that they had left their air conditioner on for much longer than they needed with 80% stating that this was the most important behaviour they wanted to change. One respondent explained that they often feel guilty about leaving the air conditioner on but leave it on anyway as no one is there to judge them. All participants agreed that their main method for saving electricity is turning off lights when they leave a room or the house. These interviews confirmed my initial assumption that the most important behaviour to focus on for this prototype is air conditioner and heater usage.

Prototype Testing

I also condudcted a round of user testing after my first prototype deliverable, getting general feedback on the prototype as well as suggestions for a method of turning Emily off and whether the "annoying" features are annoying enough. From this, I found that a lot of people were confused about the logic used to determine whether the temperature inside was inefficient. For this prototype, I was just comparing the two temperatures and if they were too different, it was inefficient (assuming similar temperatures is actually energy efficient, as the appliance doesn't have to work as hard). I decided to change this logic and add in a switch to simulate turning the air conditioner and heater on and off. This way I could then check whether an appliance was on and then how far off the government suggested temperature this setting is. This would also eliminate any false positives as if both were off, the temperature can never be deemed inefficient. This round of user testing also confirmed my method of turning Emily off by "holding her arms" and touching two touch pins on each of her arms. However, when I implemented this, I found that the touch pins were very unstable, so I decided to change to holding down a button on each of her arms instead. This method works well for the target user group as it is simple for both children and adults and easy to integrate into their everyday lives.

Technology

I started constructing my prototype using the Arduino, however through user testing and further discovery into the interaction elements, I decided an esp-32 would be required for the prototype. This was because the screen and speakers would have difficulty running on the Arduino due to the limited space on the device for images and sounds. I started with the M5 Stack as I thought it might be easier to program and set up. However, once I got the screen working, I tried to move the other features across and this was very difficult due to its compact nature. I then decided it was best just to go with a plain esp-32. Although it was quite a bit more complicated than just the Arduino, these features were necessary for the project objectives.

Form Material

I spent a lot of time exploring different alternatives for the Emily’s form. There were 3 main requirements for the form:

1. Transparent enough to let light through
2. Opaque enough to keep the wiring hidden
3. Solid enough to stand on its own

I started with Protoputty, as I could create a unique shape that would help promote an emotional connection with Emily. However, through extended experimentation with the material, I found it was too difficult to get it thin enough to let light through while also maintaining structural integrity to stand alone. In the end, I decided on laser cutting the individual pieces using white acrylic. This method passed all the requirements and would allow more time to design the object rather than trying to just sculpt it in the 15 minutes Protoputty allows. I used boxes.py to create a basic template and altered this to cater for the bended arms and holes for the different features.

Form Design

When our team first proposed the idea, we decided that Emily would take the form of a ball. However, because Emily is intended to change behaviour through negative reinforcement, she needed to have a personality to mitigate the long term effects on continued use. Through discovery into creating an emotional connection, I decided that more personality needs to be portrayed in the form. I decided to research into existing cartoon characters that people felt connected to, and I immediately thought of Groot. Groot has so much personality even though he only says 3 words. I decided to use features like a facial expression and voice to add to this personality that I wanted Emily to have. I decided that a cactus would match all of the set criteria as it is easily identifiable, relates to the environment, and is playful and cute (working well for the target user group and domain). I have played around with many different shapes to best represent a cactus, with each iteration shown below.

Reflection

The 'Actual' v.s The 'Ideal'

Due to the time and resource constraints that arose from changing a four-person project into an individual project, I had to limit the scope to ensure I could effectively focus on the interaction experience. With more time I would have loved to explore other energy-related sensors to further explore ways Emily can help the household. The current interaction experience is somewhat limited and the user may lose interest after a while. In the future, I would like to look into ways to keep the family engaged through a leader board or something else of that nature.

Other Works in the Domain

Other energy monitoring devices, such as the Sense Energy Monitor, focus on just presenting the data in an app or on-screen. They encourage behaviour change through setting goals for you and your family. However, these methods require the user to constantly engage with the device to have any effect but also shows very detailed data for those who want it. Emily is intended for a slightly different audience as she encourages an everyday interaction to promote a long term behaviour change. By being placed in the living room, she can constantly interact with the family and act as a reminder to be more energy efficient. She requires little effort from the user to engage with her until they are being inefficient. This way, she can use negative reinforcement to change their behaviour rather than the regular method of just presenting the data in an effective way, which is a lot easier to ignore.

Overall Studio Theme

The studio theme this semester was "designing for playful and open ended interactions in everyday life: human values in physical computing". This theme can be broken down into smaller parts to assess Emily's effectiveness.

• Playful: Through Emily's form and her playful interactions with the family with passive aggressive complaints when they walk past, singing, and changing facial expressions.
• Open-Ended: There is no set way to interact with Emily and through a variety of complaints she can choose from, she can create an open-ended experience with the user.
• Everyday Life: As Emily constantly monitors the energy consumption in the household, she can be interacted with everyday.
• Human Values: Sustainability is a value all humans strive for.

Desired Outcomes

I have been able to create a prototype that successfully meets the desired outcomes set for the project. Emily accurately monitors household's indoor temperature and effectively compares this to the outdoor temperature (simulated). Even though the outdoor temperature is simulated, having Emily powered by an esp-32 meant that connecting to a weather API in the future would be quite simple. This was not done for this prototype as it was much more effective to show all of Emily's abilities by being able to easily change the outdoor temperature. She knows when the air conditioner or heater is on to make informed decisions around the energy efficiency behaviour currently occuring. Emily can be easily integrated into the family's life due to her small and portable nature, allowing her to be transferred to any area of the house. Emily is easy to understand as all users that have interacted with Emily can identify the state she is in and what to do to make her happy again. Emily meets the needs of the target user group and has been tested by multiple age groups to ensure they can all turn Emily off. As Emily is still in the early phases of development and has yet to be tested over a long period, its unclear whether she will be able to impact users' long term decision making around energy consumption or if she is too annoying to encourage continued use. However, all users that were observed, also found Emily very annoying but thoroughly enjoyed the interaction with her and wanted to continue this interaction. In particular, they enjoyed the singing and facial expressions, describing her as "cute" and "funny". Although users did enjoy this, I would have liked to have looked further into the facial expressions and how I could animate her to make the interaction less static. With more time, I would have also liked to explore more customisation in her phrases. For example, I added the "Davidson family" into one of the sayings but as the text-to-speech module just requires plain text to work, it would have been cool to allow the user to input their name to customise their Emily further.

Copyright © 2020 Lucy Davidson

Built on Bootstrap