Designing for Hands, Gestures, and VR

We’ve all enjoyed movies where the heroes control futuristic interfaces by gesturing in the air. With today’s technology these are now becoming a reality – but as a UX designer, how do you go about designing an interface like this? Where would you start?

At Ultraleap, we’re tasked with establishing UX design best practice for hand-tracking – technology that lets you control VR experiences and touchless displays by gesturing in the air, using only your hands. This enables a new breed of digital experience like touch-free retail displays, or immsersive pilot and surgeon training without any need for controllers.

Gesturing with your hands is a whole new way of interacting with digital products, and the rulebook is still being written. How does a UI panel work when it has three dimensions? How do you use a webpage when you’re standing two feet away from it? And how do we ensure a good experience for all, when everyone’s body, movements and expectations are different?

In this talk, I’ll explain how the team at Ultraleap are figuring out the next big step in interaction design. I’ll show real prototypes from real projects, and give a candid explanation of what we’ve learned so far, and how we went about it – as well as answering your questions.

By the end, you’ll understand where interaction research has got to today, the design principles for building a gestural interface, and have a good idea where to begin as a UX designer creating your first touchless experience.

International View Full Schedule
Interest Area: International Date: May 2, 2023 Time: 10:30 am - 11:15 am

Matt Corrall
Design Director • Ultraleap