Covid-19 has shown us how many surfaces we touch regularly—a potential source of germs. In the workshop Project Untouched1, we focused on finding solutions for this problem using machine learning.
Inspired by Microgestures2, Soli3, and Kinetic Touchless4, we decided to create hands-free interactions in the public area. Especially in public bathrooms, the faucets are already triggered via infrared for water, but how can one change the temperature?
With a simple prototype, we propose a simple solution:
Using Tensorflow.js Handpose5 we detect keypoints of the hand and its horizontal position. Once detected, the user can form a fist to “grab” the slider and then move it along the axis to adjust the temperature. Ambient lighting behind the sink indicates the current temperature and slider range.
But what are we talking about, try it out yourself!
We can also detect different hand poses, but instead of using a pixel system, we use the more efficient vector points of handpose. For this we chain the output of Tensorflow Handpose with another machine learning algorithm—KNN (i.e. K-Nearest Neighbors) using Tensorflow.js KNN Classifier6. This could be the foundation for multiple gestures to use in a variety of projects.
Try it out yourself! Train at least two handposes to switch between them.
This project is a result of the Untouched Project Workshop together with Hongkig University, hosted by Serena Cangiano, Jae Yeop Kim, Sook Yeon Kim, and Matteo Loglio. It was held in Februrary 2021.