CONCEPT

  • Team Project
  • Individual Focus

Team domain

Fast-paced life makes people more and more impetuous, and people will also generate more and more pressure because of social factors. These pressures may come from school, work, or family, and have a bad influence on their mood and even affect their mental health. In order to solve this problem, the problem space studied by our team is to relieve everyday stress. As can be found in the background research, music can help soothe the mind and body and can achieve the purpose of relieving stress, and different types of music will have different effects on people’s emotions. Among many types of music, the impact of natural sounds or everyday sounds on stress relief is remarkable. Based on this, the aim of the project is to design a highly playable and innovative interactive device that can help people interact with daily sound. What we want to do is to make music into something that can be touched in daily life, although people without music foundation can also make their own music and enjoy the fun of music. Our design allows users to interact with daily sounds by recording, replaying, and mixing multiple pieces of everyday sound, in the process of interaction, cool visual effects will be presented to make people more immersed in the pleasure of interacting with daily sounds to achieve the purpose of decompression.

Target audience

The name of our product is called “Sound Lab”. It can be regarded as an interactive sound processing device that can help people reduce stress. These are three types of our target audience:

1. People who are interested in daily sounds

2. People who have a lot of stress in life

3. Non-professionals who are interested in music production

Functions

The whole device consists of five main functions, they are collecting and recording sound, playing sound, storing sound, mixing sound, and deleting sound. The shape of the product will be the same as the commonly used tools in the chemical laboratory, that is, droppers, test tubes, jars, flasks, trash cans, and the interactive form of the product will also simulate the way chemical experiments are usually carried out.

Ideally, the entire project can achieve the following effects:

The dropper is a sound collector. The user can record the sound by placing the dropper close to the sound source and squeezing the latex head of the dropper. The brightness of the light in the dropper can show the working status of the test tube.

The test tube is a sound storage and playback device. By touching the inner wall of the test tube with the dropper, the sound collected in the dropper can be transferred to the selected test tube, and the light in the test tube represents whether the test tube has any content stored. The user can play back the stored music by shaking the test tube, and the corresponding visual effect will be generated according to the frequency of the music. After the transfer occurs, the test tube will be emptied into an empty test tube.

The jar is a storage of fixed melody. It can be seen as a preset melody memory; users can choose to add daily sounds to the established melody for music creation to ensure the harmony of music production. The audio in the test tube can be transferred to the flask by pouring for subsequent operations.

The flask is a generator of sound mixing. Users can select their favorite sound clips and transfer them into the flask by dumping. The remixing effect is achieved by shaking the flask, and a series of visual special effects (light changes, smoke effects) will accompany the remixing.

The trash can is a sound deleting device. The user can delete the stored content by dumping the test tube or the flask to the trash can for the next music production.

Our team produces the project in the form of group cooperation. In the team, I am mainly responsible for the design of the test tube part (that is, sound storage, playback and transmission function). The test tube part can be regarded as the link of the entire interactive product. It can not only accept the audio collected by the dropper part, but also transmit different audio to the flask. My personal focus is how to make the function of music playback in a more interesting and interactive form to achieve a high degree of playability. In the design process, I focused on the following considerations: visual effects, aesthetic design, material selection, interaction form, code implementation and related user research.

Ideally, the test tube part can achieve the following effects:

It can receive and store the sound transmitted by the dropper. The dropper with the recording can store the sound by touching the inner wall of the test tube. When a sound is stored in the test tube, the built-in light of the test tube will light up, indicating that the test tube has stored a music clip.

It can play back the stored sound and produce different visual effects according to the frequency of the music. The sound stored in the test tube can be played back by shaking, what appears with the sound is a waveform that varies according to the sound being played. The design is to allow people to have a more immersive experience with everyday sounds.

It can transmit sound. People can transfer the stored music by dumping the test tube. When dumping ends, the original audio storage location will be transferred.

PRODUCT

  • Overall product
  • Individual part
  • video demonstration

The final product is a daily sound interactive device that creates a chemical laboratory as a scene. Similar to the expected effect, our products achieve most of the content mentioned in the concept. In fact, the functions implemented by our products are as follows:

The user can record the sound by pressing the test tube and can use the dropper head to touch the inner wall of the test tube to complete the sound transmission.
Users can play back multiple stored sounds by shaking different test tubes or jars. At the same time, depending on the music being played, different waveforms will be displayed on the large screen according to different audio. (Due to technical limitations, we have not realized connecting Arduino, Python and Unity at the same time. The realization of this part of the function can only be achieved by manually entering the path of the file in Unity, I will introduce this part in detail in the technical part).
The user can dump the test tube/jar to transfer the sound to the flask and shake the flask to achieve the sound mixing.
At present, due to technical limitations and the lack of related sensors, we have not completed the sound deletion and the special effects of smoke in the flask.
As an alternative, after the end of the mix, we set up a simple interactive effect based on the user's feelings. Users can interact with their friends through this small device.
People can evaluate the mixing music by shaking different handles representing different attitude. Users can invite friends to rate their music, When friends like this music, they can shake the smiling face, the system will send out the sound of the applause. When friends dislike the sound, they can shake the bitter face system will play a sad special effects sound and the sound in the flask will be deleted.

I am responsible for the design of the test tube in the entire design team. The functions currently implemented by the test tube are sound storage, sound playback, and sound transmission. To be more specific, my actual functions and some visual effects of my part are as follows：

My teammate has already done the sound recording function, for the sound that has been recorded in the test tube, the sound can be poured in by using the dropper gently touching the inner wall of test tube.
there is sound stored in the test tube, the LED in the test tube will be always on.
The sound stored in the test tube can be recalled at any time. Simply by shaking the test tube with the sound to complete the sound playback.
In order to make the interaction process more interesting, I used Unity to create a small program that presents different visual effects according to different sound frequencies. Unfortunately, this applet cannot be connected to our overall project. Although the dynamic effect can be changed with the change of audio, the applet cannot directly read the path address of the selected audio to achieve intelligent playback. I chose to leave an input path box empty in the unity page box to simulate the different effects of audio files with different paths. Apart from this, as a data transmission tool, the test tube can transfer the sound stored in the test tube to the flask to prepare for the sound mixing.

Technology (Test tube part)

  • Technical form
  • Physical form


Arduino + Python

I will use the combination of Arduino and python to make the test tube code. Arduino is used to realize the interactive form and realize the related visual effects by reading the values of different digital signals/analog signals. Python is used to receive back-end information, connect to Arduino through a serial port, and complete the functions related to this project by receiving Arduino signals.

Unity (used for making visual effect for the sound playback)

Implementation principle: I use a plugin: “audiovisualizer” to achieve the audio visualization. Since there is currently no suitable way to link unity with this project, I made an interface that accepts user input, stores the input string as a path, clicks the button to call its bound event, plays audio, and generates an effect.

The test tubes are made of glass laboratory equipment. Because the glass test tube is transparent, the wires and sensors hidden in the test tube will be easily found by the user, which is not good in appearance. We use acrylic paint to color the equipment used to have a good covering effect.

Each test tube contains:
a pressure sensor: Check if there is pressure on the inner wall of the test tube. to determine whether the analog signal reads the pressure value to control the interactive effect of the dropper touching the inner wall of the test tube
a tilt switch: to detect whether the test tube is tilted. Once the tilt switch of the test tube works, the transmission occurs, at the same time, Arduino will send a transmit command, and the background python will detect the command to perform data transmission operations
a vibration sensor: to detect whether the test tube vibrates, Arduino will send a play test tube Command, the background python monitors that the command will play the specified audio file according to what user have selected
a LED light: displays the status of the test tube, which represents whether there is sound stored in the test tube.

Design Process

  • low-fidelity prototype
  • medium-fidelity prototype
  • high-fidelity prototype

The first important node of the project is the presentation of the group in week4. At this stage, we determined the theme of the group as musical metrics and did a series of background research based on this theme. At this phase, we determined that our problem spaces that is to use music to relieve people’s stress in life and proposed the proposal of the group idea. Our initial idea was to use chemical experiment concepts to simulate sound processing, enabling people to interact with everyday sounds. According to the relevant feedback of proposal, there are some suggestions that provide good guidance for our subsequent project improvement.

The most obvious change we made based on the feedback from the previous period is the concept of a music framework is proposed. At the user research stage, users stated that they had no knowledge of musical theory, so it was difficult to ensure that the music they made was pleasing to the ear, and they were more willing to add sound clips to the established tunes, which helped to create more beautiful music. We added the concept of the jar which is used to store the established melody to the design of the second round of prototype, in order to ensure the harmony of phonology to a large extent.

The second important time node is the mid-term prototype, as shown on the left, this is the product we used to test at the time. During that period, we mainly completed most of the project's functions, and at the same time took relevant videos as supplements and introduced our products in detail. In the mid-term prototype test phase, we harvested relevant suggestions in terms of functionality and vision, which played an important role in our third round of iteration.

In terms of functions, users think that the audio presentation is not so obvious and can consider the use of waveform charts as inspiration for music presentation. In the follow-up prototype, we consider using Unity to make a music waveform effect diagram, using audio visualization to make the audio have a clearer presentation. In terms of aesthetics, people think that too many wires will make the whole product look messy, leading to the loss of interest in the product. In the subsequent iteration, we replaced the original project production materials and produced related brackets to make the lines involved in the project as invisible as possible.

The above-mentioned problems commonly mentioned in user research are our final iteration improvement directions.

The high-fidelity prototype is the final delivery product shown in the exhibition. Based on the feedback from the second stage, we made a perfection of the appearance and function of the product. The most notable is the change in the appearance of the product and the addition of new forms of interaction during the sound playback. Users think that too many wires are exposed to the outside, which affects the appearance of the product and reduces people's expectations for the product. The final product presentation will replace the previous complicated wire connection. We have perfected the appearance of the product, hiding circuit boards and complex wires in invisible boxes as much as possible. In addition, because the test tube is transparent, the sensor will be exposed to the public. Based on this, we use acrylic paint to paint the bottle body and draw a tick mark to simulate the real visual effect.

Another obvious change is the addition of visual effects to make users more immersed in the thrill of interacting with everyday sounds. The inspiration for this change is user feedback from the mid-term prototype. By summing up the feedback and suggestions, the user hopes that the music will be more diverse and more obvious, so that the music will be presented to the user in a more intuitive form. So we thought of the idea of audio visualization. Audio and visual effects are combined to catch users' eyes with cool visual effects, thereby increasing users' interest in our product.

In addition, you can see that a new function to interact with friends has been added to the product. This is based on user feedback about the lack of collaborative experience of the device. The user hopes that this is not only a device that operates daily sounds by one person but also an interactive device that allows everyone to participate in it. Therefore, in the design of the final project, we introduced the function of scoring between friends so that more people can join the daily sounds. In the process of interaction, feel the fun brought by daily sounds

Outcomes

Overall, I am satisfied with the final result of our group. Despite being affected by the epidemic, learning and life have become less convenient than usual, our team still completed the project as much as possible, and carried out the final improvement before the final exhibition, and in the final online exhibition, the students and teachers who watched our products in our live broadcast room also gave positive comments. I will complete my reflection from the following aspects.

'Actual' to'Ideal'

Our team develops the project in the form of teamwork. Although we have realized the main functions, the actual product still deviates from the expected effect. Most of the deviations are mainly due to technical reasons, as well as the emergence of uncontrollable factors caused by the impact of the epidemic. Through three main iterations, we have designed a device that allows users to better interact with daily sounds, and also completed the realization of the main functions. Reviewing the overall design ideas, our products still have the following defects:
1. In our original plan, the dropper was used as a radio device. It is wirefree, and users can take it to record their favorite sounds anywhere. Although we have connected an extension cord, it can be used in a limited range, but it still cannot provide users with a more intelligent experience. Similarly, due to the installation of many sensors, there are still many wires connected to the rest of the project. Although we tried many ways to hide the wires, it was still unavoidable that the nakedness caused the visual unsightly. How to beautify products to a greater extent, as well as intelligent products will be the goal and improvement direction of future research.
2. Unity has not successfully connected with Arduino and python. In the original plan we planned to create visual effects for the sound playback, but unfortunately we have no way to connect it with python and arduino. Although the generated visual effects are very cool, at present, it is only possible to display the visual effects by manually inputting the path. Overcoming this technical barrier is also the direction of future efforts.
3. I did not find a suitable sensor to make the sound deletion function. In our original plan there was the concept of a trash can. As soon as the instrument is dumped over the trash can, the sound will be deleted. At present, this part does not think of a suitable technical implementation method, but uses other methods to achieve the deletion effect of music.
4. The product does not completely get rid of the computer, and achieves a highly intelligent effect. In our original plan to use raspberry pi to replace the computer, and because of time constraints and limited materials on hand, we still use the computer to complete the relevant functions.

Relevance to theme

To course

Course theme is mainly about novel physical interactions. Our products are highly consistent with the course theme. Unlike traditional interaction methods (such as keyboard and mouse interaction), our products simulate the operation of the chemical laboratory, providing users with a variety of novel interaction forms such as squeezing, touching, dumping, shaking, etc. In order to give users a better operating experience.

To studio

Studio theme refers to an interesting, highly playable, innovative feature that can produce music for daily life. Our products meet the above requirements.
First, the combination of music synthesis and chemistry experiment is a very new concept. The user also showed his curiosity when he saw the device. The design idea is very attractive.
Second, our project is very close to life. In an ideal situation, people can take a dropper to collect various sounds in daily life, and take them home to interact with daily sounds. The place where the sound is mixed can also be a more common place in daily life such as homes, companies, etc. The visual effects and auditory effects produced by playing sounds can play a good role in soothing emotions.
Third, the product is highly playable and interactive, and also gives users a high degree of freedom. Users can not only record any sound they like, but also mix their favorite music clips. When using the device, the user can not only interact with the device, but also enjoy a collaborative experience with friends.

Human value

As an interactive device for daily sound processing, Sound lab can make music more into people's lives. Users can alleviate the tension and anxiety of life through the interaction with daily sounds. The product is simple to operate and has a high degree of freedom. Users can "mix" their own DIY music according to their favorite, which adds a lot of fun to the daily boring life. From the feedback of user tests, they believe that the device can greatly reduce the stress of life, and they are more attentive to the voice of life.

User experience

In the stage of final product display, users can quickly know the concept of our project through introduction, and can quickly get started. They think that the method of using a combination of visual effects and sound is very novel, and it is very unique to use daily sound as an entry point. Users think that in this way, everyone will pay more attention to the surrounding life and also play a good role in relieving stress. In addition, users can complete the mixing operation without instructions. They also appreciated the different visual effects we designed. I can think of our products as suitable for user experience.