Untouched: Faucet

A project by Tim Pulver and Olivier Brückner

The Project

Covid-19 has shown us how many surfaces we touch regularly—a potential source of germs. In the workshop Project Untouched1, we focused on finding solutions for this problem using machine learning.

Inspired by Microgestures2, Soli3, and Kinetic Touchless4, we decided to create hands-free interactions in the public area. Especially in public bathrooms, the faucets are already triggered via infrared for water, but how can one change the temperature?

With a simple prototype, we propose a simple solution:

The Solution

Using Tensorflow.js Handpose5 we detect keypoints of the hand and its horizontal position. Once detected, the user can form a fist to “grab” the slider and then move it along the axis to adjust the temperature. Ambient lighting behind the sink indicates the current temperature and slider range.

But what are we talking about, try it out yourself!

Multiple Gestures

We can also detect different hand poses, but instead of using a pixel system, we use the more efficient vector points of handpose. For this we chain the output of Tensorflow Handpose with another machine learning algorithm—KNN (i.e. K-Nearest Neighbors) using Tensorflow.js KNN Classifier6. This could be the foundation for multiple gestures to use in a variety of projects.

Try it out yourself! Train at least two handposes to switch between them.

This project is a result of the Untouched Project Workshop together with Hongkig University, hosted by Serena Cangiano, Jae Yeop Kim, Sook Yeon Kim, and Matteo Loglio. It was held in Februrary 2021.

  1. Untouched Project—A workshop by Matteo Loglio and Serena Cangiano
  2. Micro Gestures By Janis Walser Cofalka and Max Mertens, February 2017
  3. Project Soli by Google
  4. Kinetic Touchless by Stuck
  5. Tensorflow.js Handpose by Google
  6. Tensorflow.js KNN Classifier by Google