EMOJI MEETINGS
EMOJI MEETINGS: A HAND INTERACTION TESTING SYSTEM FOR TODAY'S VIDEO CALLS
With the nature of work shifting and remaining online, video chat communication for most of us has just become part of our daily lives.
Consequently, researchers have identified psychological issues like ‘Zoom fatigue’ due to several reasons, some examples being:
- An excessive amount of close-up eye contact with multiple people at once
- Facing their own on-screen reflection for long periods
- Reduced mobility through a fixed field of view, limiting their usual nonverbal cues.
That’s where the Solarflare Launch team jumped in to test out some intuitive object hand interactions within video calls. Why? To allow people to trigger responses with less disruption, higher engagement, and reduce their overall cognitive load.
How We hacked IT
Using Google’s AI, MediaPipe, our apprentice developed a hand gesture recognition system that would track the user’s x and y landmark coordinates through their webcam and trigger intuitive emoji reactions.
Want to explore the possibilities of new technologies? Get in touch to explore ideas.
By firstly expanding upon the MediaPipe JavaScript base code, points on each joint of the hand acted as landmarks relative to the HTML canvas. Through being able to determine the measurement between multiple landmarks on the hand, he was able to write conditional statements that would allow the user’s hand gesture to be recognised with a high degree of accuracy every frame/second.
In the final step, once the gesture was determined, a string was returned of the emoji, with a separate script written to handle the falling animation and follow the user’s hand position.
The result
With our designer creating a branded set of bespoke emojis we tested out the augmented reality add-on with the wider team. Take a look a the outcome below.
As platforms, companies, and brands continue to explore hybrid working as well as their digital twins, we’re faced with that recurring question of what will be that communication sweet spot.
To try out the filter for yourself, take a look at this page on your desktop!
We’re always looking to expand our network of creative collaborators, from technical specialists to people that work in more traditional art forms. LAUNCH exists to blend the barriers between creative practices and new tech. Get in touch to explore ideas.
