My original project inspiration was a Low-resolution display based on a cocktail umbrella. In my imagination, it may be a interesting interactive device that can give different feedback based on people's movements. But unfortunately, this needs work to meet the brief. It is an interactive decoration, so I finally gave up this idea. Choose to work with my best friend Shawn to complete his Inspiration.
On week four, our team conducted an online Presentation. We can see that many people are very interested in this project. Some classmates put forward some personal suggestions for our project, while others give us some details that we did not consider, such as safety issues. All these comments and suggestions form the basis of our work in the past two months. We have written these suggestions into Proposal .
Considering that our potential users are visually impaired, we need to design a safety alarm device to reduce the probability of harm to users. Secondly, voice recognition is a factor that can significantly improve the user experience of our project. Users only need to speak something by using their smartphones. The device can automatically complete some operations. These two contents are mainly designed and completed by me. Secondly, I am the decision-maker of our group, so I also participated in some other works, such as infrared guns and 3D sound effects. In the following content, I will detail the completion of our entire project.
After going through the presentation and analyzing feedback, I started to design the circuit diagram of our project. Our initial idea was to use some laser modules (KY-008) and some photoresistors to make an Arduino laser tripwire. When the laser is blocked, the resistance of the photoresistor will change. Then we can analyze the value to determine whether the user has hit the target. However, due to the influence of COVID-19, we have not received the laser module, so we cannot show our first version of the prototype before May.
So We have to re-discuss other possible options. Considering that when our users use the long swords may cause some potential risks, Shawn proposed to modify the Arduino's remote control to turn it into a pistol. We discussed the feasibility of this solution and consulted the tutor. It may be able to replace our original design, so I redesigned the second circuit diagram.
I am mainly responsible for testing the feasibility of any 3d audio production. Cindy and Bonnie are accountable for this part. The mention of 3D sound effects reminded me of ASMR. I am very impressed by the unique and realistic effect of this audio, so I thought about using Adobe Audition to synthesize ordinary sounds into ASMR-like sounds. So I tried to put two identical mono files into different channels. After synthesis, you can hear the different sounds from the left and right headphones when you put on the headphones. As long as we placed the sound document on the left channel. The user can recognize the position of the sound through the headset. So that we can complete the sound in the left and right directions. Through the analysis function of Adobe Audition( right part of the image). We can view the virtual position of the sound. In this way, by adjusting some parameters, you can get a sound in front of you. So based on these principles, we can make 3D surround sound.
We designed the infrared gun to replace the laser module as plan B. We use the Arduino Remote Control PCB as the main part of the gun. First, we removed the carbon pads of the remote control and found that there are many holes on the PCB to facilitate wiring. I tried using two wires to connect one of the buttons to the trigger of the toy gun, and soldered a brand new Infrared LED to the PCB. After testing, it works.
By collecting feedback and surveys from inspiration Presentation (through the visually impaired group on Facebook), we realized that we ignored safety issues in the original design. I rechecked our Proposal; we did not mention anything related to it. So we hold a group in mid-April to discuss how to design a device about safety.
I propose to use an ultrasonic sensor as the main module of a safety system because it is a device with almost the same principle as a bat—the ultrasonic sensor basing on sonar to determine the distance between the object and the sensor. And the detection range is wide, more than 4 meters with high accuracy. Therefore, this sensor may suitable for our project.
However, the ultrasonic sensor has a disadvantage; that is, the effective angle of the sensor is less than 15°, which may cause a dead angle. So I chose to place three ultrasonic sensors in one direction at the same time so that the detection angle exceeds 60 degrees, covering at least 16 square meters of area. Besides, the ranging distance of the ultrasonic sensor ranges from 2 cm to 400 cm/1 inch to 13 feet. I set the trigger distance of voice feedback to 2cm to 100cm as long as the user stands within this distance. The buzzer will continue to work until the user returns to a safe area( >100cm).
Learn More
When designing the first voice control prototype, I envisaged using the app called Arduino Voice Control as a terminal to receive voice commands. After testing all the code, I found that this app needs the support of Google Voice. But at this stage, we can just use Google Voice in the United States and Canada. So this voice control is only theoretically feasible, but we have not tested it.
From the circuit design point of view, the difference from the first version is the addition of a Bluetooth module. We can use a mobile phone to control three relays after connecting to Bluetooth (there is only one on the circuit diagram). Unlike the first time based on Google voice service. This time we chose to write some code to upload to Arduino and then control our equipment remotely through other apps. In the following content, I will explain in detail this part.
The safety alert circuit maintains the original design. I readjusted the trigger distance of the ultrasonic sensor from the original 100cm to the current 150cm, because after communicating with Shawn. In order to ensure the safety of the user. We temporarily removed the speaker located behind the user. So far, we only have three different directions: front, left, and right. The purpose is to enable the user to stand in the center of the field. By increasing the trigger distance, you can detect earlier whether the player has moved too much in these three directions.
I also connected the VCC (5v) of the ultrasonic sensor to the Arduino through a relay. In this way, the safety alert can be switched remotely via Bluetooth.
When improving the safety alarm, we took some feedback from the first prototype test. Almost all of users mentioned that our buzzer sounds annoying. So I rewrote the code of buzzer. From continuous sound to frequency sound.
After the first version prototype of voice control, we know that the Google Voice service is not available in Australia. Therefore I intend to use the Internet of Things-based service to design voice recognition functions. I have check some information about this kind of service. I found that Blinker is a home IoT app similar to the Home app on ios devices. We can use it to connect the smart devices via Bluetooth or wifi, and It is highly editable, including Interface and function settings. So this software is much suitable than 'Arduino voice control' for our project.
This time I choose JDY-16 as our Bluetooth module. It has several advantages, low power consumption，Bluetooth 4.0. These two advantages allow us to connect Bluetooth terminals more steadily. We don't need to worry about too much power consumption that can cause Arduino to drive too many modules (I have encountered too many modules on Arduino before, and Arduino cannot Stable power supply).
As can be seen from Circuit Design2, I connected two Arduino UNO by using a relay. Relay is used as a switch to connect the Ultrasonic power cord (VCC), so that I can connect the Arduino to control the VCC switch through the mobile phone to complete the remote operation.
Based on this, we can apply to any devices that need to be turned on remotely, such as the infrared receive, and our speakers.
void button1_callback(const String &state)
{
if (state == BLINKER_CMD_ON)
{
digitalWrite(7, HIGH); //relay on
BLINKER_LOG("Toggle on!");
Button1.color("#FFFFFF");
Button1.print("on");
}
else if (state == BLINKER_CMD_OFF)
{
digitalWrite(7, LOW); //relay off
BLINKER_LOG("Toggle off!");
Button1.color("#FFFFFF");
Button1.print("off");
}
}
void setup()
{
Serial.begin(115200);
BLINKER_DEBUG.stream(Serial);
pinMode(7, OUTPUT);
digitalWrite(7, LOW);
Blinker.begin(2, 3);
Button1.attach(button1_callback);
}
After the online exhibition, many people think that we have reasonably used different technologies to complete our project. And it looks very cool. Notably, we use voice control instead of traditional buttons or other physical interaction, which can facilitate the barrier-free use of this device for visually impaired people. Some people think that in the process of completing this project. We can consider using some technologies to protect the safety of users, which is a heartwarming thing. Of course, our project still has many shortcomings. For example, our tutor mentioned that not everyone likes zombies. Some people are afraid of this sound. Therefore, we need more user research to expand the user group and provide convenience for more visually impaired people.
Overall, I am satisfied with the work I am responsible for and the final product made through the efforts of our team. The current product was playable. It means that we didn't build a castle in the sky. Almost all our plans on the report are completed, and they are shown through the project. We can spend more time designing the storyline and adding some other features to make this device more interesting.
I knew the information about "ASMR" about three years ago. Its realistic sound feedback left a deep impression on me, which is why I originally proposed to use 3D surround sound (simulated ASMR). After all, the sound is one of the critical factors for improving the user experience at this stage. But the obvious thing is that we use too many computer voices and it sounds weird.
Regarding safety alarm, I always think that ultrasonic sensors are not the best choice. Although it can accurately calculate the distance between the person and the sensor, its disadvantages are also visible. For example, I had to use three ultrasonic sensors at the same position to avoid dead ends. This is not a delicate design, and sometimes the three ultrasonic sensors will interfere with each other. And we found in the actual test that there is some time delay in the triggering of the ultrasonic sensor. For me, laser tripwire alarm may be a better choice. Just like the scene in the movie, when people block the laser, the alarm will sound the first time with almost no delay. And the laser emitter is tiny, and will not cause any obstacles to the player's activities. However, because of the impact of the COVID-19. We did not receive the laser module, so the ultrasonic sensor was finally used. Our product experience should be better.
Regarding voice control, voice control should be a smart device based on Google voice, which is more stable through wifi connection and does not have to worry about the transmission distance. For some other reason, we can not use Google voice service in Australia and finally use the Bluetooth module as the wireless control medium. And our software experience is not very good, for example, the mobile phone needs more than 8s to recognize our voice commands, which is too long. I can use Siri to complete our voice control. All we need is just a Gateway, but it is costly in Australia. So we gave up this idea.
Although we have some regrets based on user experience, it is clear that the process was incredibly rewarding. Our project can solve the problem space.