Faceware Technologies

History

Image Metrics, founded in 2000, is a provider of facial animation and motion capture technology within the video game and entertainment industries. In 2008, Image Metrics offered a beta version of its facial animation technology to visual effects and film studios. The technology captured an actor’s performance on video, analyzed it, and mapped it onto a CG model. The release of the beta allowed studios to incorporate the facial animation technology into internal pipelines rather than going to the Image Metrics studio as they had in the past. The first studio to beta test Image Metric’s software in 2009 was the visual effects studio Double Negative out of London.

In 2010, Image Metrics launched the facial animation technology platform Faceware. Faceware focused on increasing creative control, efficiency and production speed for animators. The software could be integrated into any pipeline or used with any game engine. Image Metrics provided training to learn the Faceware platform. The first studio to sign on as a Faceware customer was Bungie, which incorporated the software into its in-house production. Image Metrics acquired FacePro in 2010, a company that provided automated lip synchronization which could be altered for accurate results, and Image Metrics integrated the acquired technology into its facial animation software. Also in 2010, Image Metrics bought Character-FX, a character animation company. Character-FX produced tools for use in Autodesk’s Maya and 3DS Max which aide in the creation of character facial rigs using an automated weighting transfer system that rapidly shifts facial features on a character to create lifelike movement.

Image Metrics raised $8 million in funding and went public through a reverse merger in 2010 with International Cellular Industries. Image Metrics became wholly owned by International Cellular industries, which changed its name and took on facial animation technology as its sole line of business. Faceware 3.0 was announced in March 2011. The upgrade included auto-pose, a shared pose database, and curve refinement. Image Metrics led a workshop and presentation about Faceware 3.0 at the CTN Animation Expo 2011 titled "Faceware: Creating an Immersive Experience through Facial Animation." Faceware’s technology was displayed at Edinburgh Interactive in August 2011 to show its ability to add player facial animation from webcam or Kinect sensor into a game in real time.

Image Metrics sold the Faceware software to its spinoff company, Faceware Technologies, in January 2012. Following the spinoff, Faceware Technologies focused on producing and distributing its technology to professional animators. The technology was tested through Universities, including the University of Portsmouth.

Faceware launched its 3D facial animation tools, software packages Faceware Analyzer and Faceware Retargeter with the Head-Mounted Camera System (HMCS). Analyzer tracks and processes live footage of an actor and Retargeter transfers that movement onto the face of a computer-generated character. The Head-Mounted Camera System is not required to use the software. Six actors can be captured simultaneously.

Faceware Live was shown for the first time at SIGGRAPH 2013. It was created to enable the real-time capture and retargeting of facial movements. The live capture of facial performance can use any video source to track and translate facial expressions into a set of animation values and transfer the captured data onto a 3D animated character in real time. In 2014, Faceware released Faceware Live 2.0. The update included the option to stream multiple characters simultaneously, instant calibration, improved facial tracking, consistent calibration, and support for high-frame-rate cameras.

In 2015, Faceware launched a plugin for Unreal Engine 4 called Faceware Live. The company co-developed the plugin with Australia-based Opaque Multimedia. It makes motion capture of expressions and other facial movements possible with any video camera through Faceware's markerless 3D facial motion capture software.

In 2016, Faceware announced the launch of Faceware Interactive, which is focused on the development of software and hardware that can be used in the creation of digital characters with whom real people can interact.

Partners

Faceware Technologies partnered with Binari Sonori in 2014 to develop a video-based localization service. Also in 2014, Faceware Technologies entered a global partnership with Vicon, a company focused on motion capture. The partnership would focus on developing new technology to expand into full-body motion capture data. The first step of the integration was to make the Faceware software compatible with Vicon’s head rig, Cara, to allow data acquired from Cara to be processed and transferred into Faceware products.

Overview

Faceware Technologies has two main aspects of facial animation software.

Faceware Analyzer is a stand-alone single-camera facial tracking software that converts videos of facial motion into files that can be used for Faceware Retargeter. The Lite version of the software can automatically track facial movements which can be applied to 3D models with Faceware Retargeter. The Pro version can perform shot specific custom calibrations, import and export actor data, auto indicate tracking regions, and has server and local licensing options. The data captured by Faceware Analyzer is then processed in Faceware Retargeter.

Faceware Retargeter 4.0 was announced in 2014. Faceware Retargeter uses facial tracking data created in Analyzer to create facial animation in a pose-based workflow. The upgrade has a plug-in for Autodesk animation tools, advanced character expression sets, visual tracking data, shared pose thumbnails, and batch processing. The Lite version of the Retargeter software transfers actor’s performances onto animated characters and reduces and smooths key frames. The Pro version includes custom poses, intelligent pose suggestions, shared pose libraries, and the ability to backup and restore jobs.

Faceware Live aims to create natural looking faces and facial expressions in real-time. Any video source can be used with the software’s one-button calibration. The captured video is transferred onto a 3D animated character. This process combines image processing and data streaming to translate facial expressions into a set of animation values.

Faceware has hardware options that can be rented or purchased. Available hardware is the entry level GoPro Headcam Kit and the Professional Headcam System. The Indie Facial Mo-cap package includes hardware, a camera and headmount, and the tools to use it.

Selected works

Faceware software is used by companies such as Activision-Blizzard, Bethesda, Ubisoft, Electronic Arts, Sony, and Microsoft. Rockstar Games used the software in games such as Grand Theft Auto V and Red Dead Redemption and Bungie used Faceware in games including Destiny and Halo: Reach. Faceware has also been used in other games like XCOM2, Dying Light: The Following, Hitman, EA Sports UFC 2, Fragments for Microsoft’s HoloLens, DOOM, Mirror’s Edge Catalyst, Kingsglaive, F1 2016, ReCore, Destiny: Rise of Iron, Mafia III, Call of Duty Infinite Warfare, Killzone:Shadow Fall, NBA 2K10-2K17, Sleeping Dogs, Crysis 2 and 3, and in movies like The Curious Case of Benjamin Button and Robert Zemeckis's The Walk.