Meta Reality Labs

The FRL Pathfinder team took Hand Tracking from a research technology to a product shipped on Quest, unlocking new possibilities for both developers and users. Our work was the subject of a key talk at OC6, and became a set of public design guidelines that enable developers to quickly make best use of this new input.

The research phase of this work explored new interaction concepts and interfaces natively designed for hand input. This was followed with focussed studies on key interactions like targeting and manipulating. Together these were used to define the interaction model, reactive components, and visual feedback that makes Hands on Quest feel useable and powerful.

As part of the AR OS Shell team for Orion, I led augment orchestration, defining how spatial applications interact with the system, with each other, and with the physical world. Working across system design, interaction prototyping, and concept film making, I produced a series of prototypes and films that became the primary design briefs for a team of nearly 50, defining core OS elements that are shipping on Orion today.

The FRL Pathfinder team worked with Mixed Reality technology teams to build a productivity proof of experience showcasing smart physical surfaces, multiple inputs, and new use cases. Several elements of this work have been adopted and are planned for release in future products. Footage above is public material indicative of internal work.