Avatar Kinect
The Kinect, the first mass market focused sensor device paved the way for technology and experiences that we take for granted today (such as the sensor suites on cars, and VR and AR HMDs). While the Xbox team focused on using Kinect to craft fun and compelling experiences where you and others were able to puppet an avatar, they ignored one of the most compelling components of feeling like your avatar was truly a reflection of you. My team in Microsoft Research (an innovation incubation group) was focused on next generation telepresence modalities and had just shipped Microsoft’s first face tracking SDK. This enabled us to bring real-time facial expression and emotion to Xbox avatars, creating a richer experience. You can see the potential for expressive avatars reflected in today’s social worlds like Meta’s Horizon Worlds and Reality Labs work, as well as Microsoft’s Mesh platform.
My role on the project was focused on creative direction & design which included much of the ideation for the user experience as well as art direction for the project. Our internal art team numbered 7 people, with a small portion of work (e.g., illustration, audio, and some animations) outsourced. You can see an early proof of concept video someone posted to the web here. (That person was NOT the “Art Director,” or even the main responsible technical artist who made the adoption of the existing Xbox avatar design system work for our tech). Yes, I’m a bit salty about that. :)
A highlight for the team (and product) was Microsoft CEO Steve Ballmer not only announcing the title at CES, but also appearing as an avatar himself, powered by our technology. Nice eyebrows there, Steve.
A key element of direction on this project was maintaining not just the look and feel of the Xbox brand (especially as part of the Kinect launch), but also product quality of experience that met the bar for 1st party products (a high bar to meet for the Xbox brand).
Here are few detail images from the product development, from our original face tracking prototype, concept art by Jenna Smith commissioned for one of the stages, and various screenshots of some of the environments used.