Understanding emotions as a business task is the challenge software development companies often face today. Technology progresses at a fast pace, and the future from good old science fiction is upon us. Emotion recognition is one of the discoveries that seemed to be impossible a decade ago and is a matter of technology to implement now.
Following current hot trends and in cooperation with our partners from Belarus, we have built a portable system of real-time emotion recognition via facial expressions for beauty salons. The article discovers practical technical aspects, and also it touches the privacy issue. A reader may find what we’ve managed to achieve, how our emotion recognition system is applied, and which points require improvements, and finally, what potential of our emotion recognition system remains uncovered.
Emotion recognition on the service of a beauty industry
The birth of an idea
Businesses care about customer engagement to gain trust and loyalty and thus return a client. To be a client-centric business is to serve your customers the way they feel special, in other words, to rouse emotions. No surprise that in the era of automation, emotion recognition becomes a technology. In a competitive chase, an effective business may need a tool to evaluate satisfaction from the buyer’s experience. Emotion recognition systems can help a company to provide better services. Such software can capture and filter the customers’ feelings impartially, gather, process, and analyze the data, structure, and systemize results in accordance with business purposes.
A few years ago, to learn a new trend of face recognition and to fulfill the thirst of knowledge, our experts started experimenting with toolkits and frameworks. The best way to develop a skill is by doing. We’ve tried different application areas of computer vision on practice:
-
Emotion Recognition
-
Computer Vision
-
Object Counting
-
ID & Document Verification
-
Car Numbers recognition, and
Hosting our client in SYTOSS’s office in Kharkiv, we shared our research, and that conversation set the start of a new project.
Business tasks to solve by recognizing emotions
Orientation on customer engagement and personal experience is the trend and the only possible method of doing business for some industries, especially for services like beauty salons, sports, SPAs, and others. Building connections and boosting satisfaction are the additional tasks of specialists providing the services, like barbers, nailers, coaches, or fitness instructors. The usual practice for business owners is to award those masters who work on increasing engagement. But how to evaluate clients’ satisfaction in an unbiased, quick, and effective manner?
In our case, the emotion recognition software is intended for beauty salons, so the initial requirements were as follows:
-
Easy-to-use, a receptionist and a business owner should be able to handle software painlessly.
-
The real-time operation, a beauty salon client should not wait, and an owner may receive current data.
-
Portable hardware, which can be moved and reorganized anytime and anywhere without complex mounting or assembling.
-
Reasonable pricing, an average beauty salon could afford emotion recognition implementation.
-
And, privacy compliance, the system should not store photos and clients’ sensitive information about clients without their permission.
Taking into account the requirements, our AI experts investigated the field and offered the solution, consisting of affordable hardware available at the market, and custom-developed software using open-source toolkits. The demand for real-time operation narrowed our lookout of tools significantly, and we had to exclude popular cloud solutions by Amazon and Microsoft from the comparison.
A portable kit of emotion recognition hardware:
-
a laptop and
-
one 2 Megapixel IP network camera.
Emotion recognition tools and frameworks:
-
OpenVINO Toolkit by Intel,
-
Algorithm accelerators, and,
-
C++.
Emotion recognition software in operation
Buyer's journey. A receptionist’s post is usually the first point of communication when a client enters a beauty salon, and the last one when it leaves. A receptionist greets clients, asks them about their appointments, has small talks, and invites a client to the appointed specialist. After the procedure ends, a client goes back to a receptionist post to pay for a service, or to schedule the next appointment. That is why the post of a receptionist is an excellent choice to install hardware for the emotion recognition system.
Hardware mounting. As a result of different tests and trials, we’ve found that the distance between the IP camera and a client should not exceed two meters. At the bigger distance, the precision of recognition results suffers, as the system zooms shots to detect a face. Also, lightning conditions are essential for the quality of the photos. Direct sun or lights may blind the camera, and on the opposite, the poorly lighted room may cause the decline of photos quality as well. As soon as emotions are recognized in real-time, a laptop from the middle price range is enough to process the data input from up to ten cameras.
Software at work. A 2 megapixel IP camera makes up to 15 shots of a client in a second. The emotion recognition system marks a client with the ID number, and four neural networks start processing the visual data.