EMOJI MEETINGS: A HAND INTERACTION TESTING SYSTEM FOR TODAY'S VIDEO CALLS
With the nature of work shifting and remaining online, video chat communication for most of us has just become part of our daily lives.
Consequently, researchers have identified psychological issues like ‘Zoom fatigue’ due to several reasons, some examples being:
- An excessive amount of close-up eye contact with multiple people at once
- Facing their own on-screen reflection for long periods
- Reduced mobility through a fixed field of view, limiting their usual nonverbal cues.
That’s where the Solarflare Launch team jumped in to test out some intuitive object hand interactions within video calls. Why? To allow people to trigger responses with less disruption, higher engagement, and reduce their overall cognitive load.
How We hacked IT
Using Google’s AI, MediaPipe, our apprentice developed a hand gesture recognition system that would track the user’s x and y landmark coordinates through their webcam and trigger intuitive emoji reactions.
In the final step, once the gesture was determined, a string was returned of the emoji, with a separate script written to handle the falling animation and follow the user’s hand position.
With our designer creating a branded set of bespoke emojis we tested out the augmented reality add-on with the wider team. Take a look a the outcome below.
As platforms, companies, and brands continue to explore hybrid working as well as their digital twins, we’re faced with that recurring question of what will be that communication sweet spot.
To try out the filter for yourself, take a look at this page on your desktop!